Opened 7 years ago

Closed 6 years ago

#10103 closed enhancement (fixed)

Support building FF24 from Gitian

Reported by: mikeperry Owned by: mikeperry
Priority: High Milestone:
Component: Applications/Tor bundles/installation Version:
Severity: Keywords: ff24-esr
Cc: gk Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We should add way to build the new alpha TBBs from a separate versions file from Gitian, so we can build both FF24 and FF17 from the same Gitian builder checkout.

gk also has some patches we should merge when this is ready.

Child Tickets

TicketTypeStatusOwnerSummary
#9828defectclosederinnFirefox 24 ESR needs at least Python 2.7 and libgstreamer dev packages on the 10.04 VM
#9829defectclosederinnFirefox ESR 24 does need a newer compiler than gcc 4.2
#9830defectclosederinnmingw-w64 compilation of Firefox 24 ESR is broken
#9837defectclosederinnmake package is broken for TBB 3.0 based on Firefox 24 ESR
#9858defectclosederinnunzip breaks bundling TBBs based on Firefox 24 ESR
#9896defectclosederinnDebugInfo files are not properly created with Firefox 24 ESR
#10139defectclosederinn--enable-strip is broken in ESR24 builds for Mac OS X
#10159defectclosederinnomni.ja differs in ESR24 Gitian builds
#10161defectclosederinnrecord-inputs.sh is complaining when building ESR 24 alpha bundles
#10201defectclosederinnFirefox ESR 24 sometimes hangs during exit on Mac OS
#10235defectclosederinnPackaging (JavaScript) resources into omni.ja files is non-deterministic in ESR24
#10252defectclosedTBB based on ESR 24 is not working on Windows
#10320defectclosedFirefox ESR 17 is not compiling with the new OS X cross-compiler

Change History (15)

comment:1 Changed 7 years ago by mikeperry

gk - Can you make this ticket the parent of all of your FF24-related build patches? That way we can try to get this all done together?

comment:2 Changed 7 years ago by mikeperry

Ok, the work in the parent ticket is done. I created the ability to have a parallel 'versions.alpha' file to build alpha packages using separate Makefile rules:
https://gitweb.torproject.org/builders/tor-browser-bundle.git/commitdiff/01c6ce6472cbd341367df9b856aba08e46d51eb5

Let me know if there are issues with that that would make merging the child tickets more difficult. It seems like it should work for everything that doesn't involve patches to the actual descriptors (though perhaps we can provide them with a 'versions' file as a gitian input with various build hints, too).

comment:4 Changed 7 years ago by gk

Just having an alpha versions file seems not enough here as there will be substantial and sometimes incompatible changes in some of the descriptors between ESR17 and ESR24. This could easily lead to a mess or a lot of extra work if we want to support building with ESR17 and ESR24 in parallel. What about having alpha descriptors as well? This seems to be the cleanest approach to me. We could then just pass the "alpha" flag to the make files which in turn would grab the proper versions file and descriptors.

comment:5 Changed 7 years ago by gk

On a second thought adding another bunch of files that need to get updated and merged from time to time might be overkill. So what's the use-case here? If it should make the life easier in a transition period from one ESR to another then I think we can indeed omit the alpha descriptors as the dev needs to patch the build process anyway even with an alpha versions file (there is e.g. no tor-browser-24.1.0esr-1 tag and Gitian is therefore yelling at me). If that should be a more permanent thing/for other people with different requirements I would still argue for alpha descriptors or something similar.

comment:6 Changed 7 years ago by mikeperry

We only have a month until we need to switch over to ESR24. We should just start building ESR17 from the ESR24 descriptors. Basically the only differences will be python and a new compiler for MacOS, right?

After this transition, the stable and alpha releases will both be based on ESR24 for some time. This will make using a different versions file even more straightforward.

comment:7 Changed 7 years ago by mikeperry

Ok, I just ran into something that definitely will cause differences in our gitian-bundle.yml descriptors: The prefs file now lives inside tor-browser_en-US/Browser/browser/omni.ja instead of in the jar one level up.

We could still pass in a 'versions' file as input that tells us which omni.ja to update, I guess?

comment:8 Changed 7 years ago by mikeperry

Ok, I committed a hack that should work around the omni.ja issues.

comment:9 in reply to:  7 ; Changed 7 years ago by gk

Replying to mikeperry:

Ok, I just ran into something that definitely will cause differences in our gitian-bundle.yml descriptors: The prefs file now lives inside tor-browser_en-US/Browser/browser/omni.ja instead of in the jar one level up.

Looks like #9837, no? If so, why not fixing it at the browser level and leaving the descriptors alone?

comment:10 in reply to:  6 Changed 7 years ago by gk

Replying to mikeperry:

We only have a month until we need to switch over to ESR24. We should just start building ESR17 from the ESR24 descriptors. Basically the only differences will be python and a new compiler for MacOS, right?

No. See #9837 and #9858. Although the workaround for the latter should be working for ESR17 as well. Apart from that I planned to get rid of the hacky links we set in the Firefox descriptor for Mac OS X. Mozilla has already a patch for it on trunk which I planned to backport for ESR24. Thus, we'd need either different descriptors for ESR17 and ESR24 or we need to keep the links or we need to backport the fix for ESR17 as well.

After this transition, the stable and alpha releases will both be based on ESR24 for some time. This will make using a different versions file even more straightforward.

Okay, then lets try to avoid alpha descriptors. We'll see how far we get with it.

comment:11 in reply to:  9 Changed 7 years ago by gk

Replying to gk:

Replying to mikeperry:

Ok, I just ran into something that definitely will cause differences in our gitian-bundle.yml descriptors: The prefs file now lives inside tor-browser_en-US/Browser/browser/omni.ja instead of in the jar one level up.

Looks like #9837, no? If so, why not fixing it at the browser level and leaving the descriptors alone?

I just saw that is not enough. While the patch there is fixing the make package step it does not fix the breakage in the bundle descriptors...

comment:12 Changed 7 years ago by gk

Here is another issue with the do-not-use-alpha-descriptors-approach: The new Mac OS X cross-compiler won't be able to compile ESR17 without at least two additional patches for ESR17 (let alone that we need to backport some of the patches needed for ESR24): a) the workaround mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=887645 and b) the patch for https://bugzilla.mozilla.org/show_bug.cgi?id=855790

comment:13 Changed 7 years ago by gk

Oh, and, of course, there is the issue that the current Mac cross-compiler is a 32bit one but the new one a 64bit one (the 32bit toolchain was not able to compile ESR24) needing now a 64bit VM.

comment:14 Changed 7 years ago by mikeperry

Keywords: MikePerry201311 removed
Summary: Support building both FF24 and FF17 from GitianSupport building FF24 from Gitian

comment:15 Changed 6 years ago by gk

Resolution: fixed
Status: newclosed

Closing, finally.

Note: See TracTickets for help on using tickets.