Build our own cctools for macOS cross-compilation
Ray Donnelly has been working on merging his OSX cross compiler patches into the crosstools-ng project, which should allow us to more easily build his compilers from source instead of relying on a binary blob.
Jotting down Ray's instructions in case anyone wants to try this:
On Linux it is possible now to build the equivalent of what toolchain4 provided (for OSX only - iOS still has some build issues)
You should be able to clone: https://github.com/diorcety/crosstool-ng.git
.. checkout the cctools-llvm branch, and enter:
./configure --prefix=<SOMEWHERE>/ct-ng-build
make
make install
export PATH="${PATH}":<SOMEWHERE>/ct-ng-build
mkdir <SOMEWHEREELSE>/ct-ng-final
cd <SOMEWHEREELSE>/ct-ng-final
ct-ng list-samples
ct-ng i686-apple-darwin11 (or i686-apple-darwin10)Check the differences between samples/i686-apple-darwin11 and samples/i686-apple-darwin10 too, the main one being that 10 copies the sysroot into the final build whereas 11 doesn't. This means you need to pass --sysroot to the compilers built with 11 but in theory don't need to for the 10 ones (of course 10 is set up to use MacOSX10.6.sdk while 11 is set up to use MacOSX10.7.sdk).
We've been adding clang to the project most recently which has caused us to disabled llvm-gcc for now and the addition of clang isn't ready yet (it is nearly there but I am going on holiday for 2 weeks after this week so I'm not sure if we will get it finished before then - likely not).
So, all the above instructions are still valid; go with commit 7d555f284b6977e64640a30bcec77597580d3049 if you can. Any problems give me a shout. FWIW, llvm/clang is only now coming on-line for Darwin software anyway, and vanilla gcc-4.2 (i.e. not even llvm-gcc-4.2) is much more reliable for building OSX software. Seems Apple focussed more on iOS than OSX for clang.
Let me know how it goes. We've started to submit some patches upstream to crosstool-ng this last week, but I have a feeling it could be a drawn out process. We will regularly merge the other way the meantime though.
Activity
Trac:
Cc: N/A to g.koppen@jondos.de
Mike, are you using llvmgcc or normal gcc from toolchain4 at present? If normal gcc, then I'd recommend that you guys build from the latest of the cctools-llvm branch to get the latest fixes (llvmgcc is disabled at present, IMHO it was never very reliable, even the official Apple binaries). If you update to the latest then clang can be built now, however this is very untested so far and if you look in the TODO file in the root folder you'll get a better picture of some things that may still be problematic with clang.
Georg, there's no reason to use darwin11 instead of darwin10. When I started this work it was with a darwin11 SDK so it's purely a matter of habit for me. The merge to crosstool-ng is being done by Yann Diorcet and myself; I handle the darwin11 samples and he does the same for the darwin10 ones.
A compiler built with the darwin11 SDK (MacOSX10.7.sdk) can be used fine to build software for darwin10 using -mmacosx-version-min=10.5, However feel free to base your version on i686-apple-darwin10 (e.g. flosoft's MacOSX10.6.sdk) and the i686-apple-darwin10 sample instead.
I recommend studying the two crosstool.config files:
samples/i686-apple-darwin10/crosstool.config and samples/i686-apple-darwin11/crosstool.config
.. these are just samples of course. You could make your own, e.g. darwin-gitian. The final compiler prefixes don't depend on the sample folder name but rather the options specified in the crosstool.config file itself.
Some options that should be noted are:
CT_DARWIN_SDK_PATH="${HOME}/MacOSX10.7.sdk" .. this is where you placed your OSX SDK.
CT_DARWIN_COPY_SDK_TO_SYSROOT=n .. setting this to y means that some headers and dylibs from the OSX SDK are copied into the installation prefix. Doing this means you do not need to pass -sysroot to the compiler on the commandline, with the down-side being that you'd not be able to distribute the built toolchain due to copyrights and the OSX SDK license.
CT_LLVM_V_3_3=y .. specifies the version of LLVM (and clang) to use. LLVM is compiled into cctools for LTO support and also, obviously for clang. Toolchain4 used LLVM 2.7 which is quite old at this stage. 3.3 is the latest release. I'm more interested in modernising things than using old versions (though reliability trumps all other concerns).
CT_DEBUGGABLE_TOOLCHAIN=n .. expect to use 20GB if you set this to y. The built tools (and also those installed) are built at -O0 -ggdb and left un-stripped.
I think it would be sensible for me to setup exactly the same environment you are using. Is this Ubuntu? If so, can you point me to the exact ISO and any scripts you use to prepare it for development? I will then make a new VM.
If you've got some logs (build.log) detailing the "fun bugs" please post them.
Trac:
Username: mingwandroid-
Something else I should mention is that the old GCC in the crosstool-ng port doesn't yet support the -arch and -Xarch flags (whereas toolchain4 does).
This is because those flags are actually handled by driverdriver.c which is an Apple-specific program that's compiled outside of the normal GCC/llvmgcc compilation and when run calls the real GCC once for each arch specified (using the standard -m32 or -m64 flag). It then calls lipo to merge the objects for each arch together into a fat object/binary.
I'm not sure how the tbb build system for Mac invokes gcc; I would hope it is easy to change it from using "-arch i386" to "-m32" and/or "-arch x86_64" to "-m64", of course, if you build fat binaries in a single invocation then this is a bigger problem.
Having said that, adding the driverdriver program is high priority and something I expect to do tomorrow.
Finally, clang 'natively' supports these flags but I'd be (pleasantly) surprised if switching tbb to build with clang was trivial.
Trac:
Username: mingwandroid -
Replying to mingwandroid:
Mike, are you using llvmgcc or normal gcc from toolchain4 at present? If normal gcc, then I'd recommend that you guys build from the latest of the cctools-llvm branch to get the latest fixes (llvmgcc is disabled at present, IMHO it was never very reliable, even the official Apple binaries).
Normal gcc and yes, I am using the latest cctools-llvm branch.
Georg, there's no reason to use darwin11 instead of darwin10. When I started this work it was with a darwin11 SDK so it's purely a matter of habit for me. The merge to crosstool-ng is being done by Yann Diorcet and myself; I handle the darwin11 samples and he does the same for the darwin10 ones.
A compiler built with the darwin11 SDK (MacOSX10.7.sdk) can be used fine to build software for darwin10 using -mmacosx-version-min=10.5, However feel free to base your version on i686-apple-darwin10 (e.g. flosoft's MacOSX10.6.sdk) and the i686-apple-darwin10 sample instead.
That was actually the reason why I asked. We use already flosoft's MacOSX10.6.sdk and I hoped to avoid using the 10.7 SDK to get a compiler to use that one with the 10.6 SDK... Good.
I recommend studying the two crosstool.config files:
samples/i686-apple-darwin10/crosstool.config and samples/i686-apple-darwin11/crosstool.config
.. these are just samples of course. You could make your own, e.g. darwin-gitian. The final compiler prefixes don't depend on the sample folder name but rather the options specified in the crosstool.config file itself.
Thanks I'll think about doing an own while I digest that whole system.
I think it would be sensible for me to setup exactly the same environment you are using. Is this Ubuntu? If so, can you point me to the exact ISO and any scripts you use to prepare it for development? I will then make a new VM.
The gitian build system is working with (at least) the current LTS (12.04). I order to build the Mac TBB you do the following
git clone https://git.torproject.org/builders/tor-browser-bundle.git cd tor-browser-bundle/gitian make prepInstall all the stuff you are asked to and run
make prepagain until you have all the necessary sources (i.e. until |make prep| does not give you any errors anymore) Then do
./mkbundle-mac.shand it should give you (after a while) Mac TBBs in a new directory (currently 3.0-alpha-4). Anyway, if you have issues with setting it up. Just write me a mail and we'll sort these things out.
If you've got some logs (build.log) detailing the "fun bugs" please post them.
So I have found so far three issues (disclaimer: I am not sure yet if there is anything you can do about as this might actually be things belonging to the source packages used in building the compiler):
-
I hit intermittent GMP configure failures due to newly created files being older than distributed ones. (log: cross_mac_gmp1) That is likely due to the gitian build environment but it would be good to somehow avoid that. I am wondering whether the reason here is some old version of GMP? I have been building mingw-w64 from source for a while with a similar gitian environment but never hit this problem.
-
'groff' was missing (the check in the configure script is there but although 'groff' is needed for building the script is not returning with an error if it is not available) (cross_mac_build1). Installing the necessary package probably solves the issue.
-
a Makefile.config(?) is missing? (cross_mac_build2).
Now, what bothers me most, though, is that Firefox 24 is not working with a gcc4.2 anymore (at least Mozilla is saying that in the configure script and I don't have a reason to doubt that). I am under the impression that your cross-compilers are (by default) gcc4.2, no? If so, what are our options here? Are there non-interactive ways to change the gcc version?
-
-
Trac:
cross_mac_gmp1 -
Trac:
cross_mac_build1 -
Trac:
cross_mac_build2 -
Replying to gk:
The gitian build system is working with (at least) the current LTS (12.04). I order to build the Mac TBB you do the following {{{ git clone https://git.torproject.org/builders/tor-browser-bundle.git cd tor-browser-bundle/gitian make prep }}} Install all the stuff you are asked to and run {{{ make prep }}} again until you have all the necessary sources (i.e. until |make prep| does not give you any errors anymore) Then do {{{ ./mkbundle-mac.sh }}} and it should give you (after a while) Mac TBBs in a new directory (currently 3.0-alpha-4). Anyway, if you have issues with setting it up. Just write me a mail and we'll sort these things out.
I forgot to add that the relevant files for cross-compiling the TBB code are gitian-tor.yml and gitian-firefox.yml in tor-browser-bundle/gitian/descriptors/mac. This should give you a hint on how your cross-compilers are currently used in the gitian build setup. If you have questions about that as well, drop me a mail.
-
Replying to gk:
Replying to mingwandroid:
Mike, are you using llvmgcc or normal gcc from toolchain4 at present? If normal gcc, then I'd recommend that you guys build from the latest of the cctools-llvm branch to get the latest fixes (llvmgcc is disabled at present, IMHO it was never very reliable, even the official Apple binaries).
Normal gcc and yes, I am using the latest cctools-llvm branch.
Georg, there's no reason to use darwin11 instead of darwin10. When I started this work it was with a darwin11 SDK so it's purely a matter of habit for me. The merge to crosstool-ng is being done by Yann Diorcet and myself; I handle the darwin11 samples and he does the same for the darwin10 ones.
A compiler built with the darwin11 SDK (MacOSX10.7.sdk) can be used fine to build software for darwin10 using -mmacosx-version-min=10.5, However feel free to base your version on i686-apple-darwin10 (e.g. flosoft's MacOSX10.6.sdk) and the i686-apple-darwin10 sample instead.
That was actually the reason why I asked. We use already flosoft's MacOSX10.6.sdk and I hoped to avoid using the 10.7 SDK to get a compiler to use that one with the 10.6 SDK... Good.
I recommend studying the two crosstool.config files:
samples/i686-apple-darwin10/crosstool.config and samples/i686-apple-darwin11/crosstool.config
.. these are just samples of course. You could make your own, e.g. darwin-gitian. The final compiler prefixes don't depend on the sample folder name but rather the options specified in the crosstool.config file itself.
Thanks I'll think about doing an own while I digest that whole system.
I think it would be sensible for me to setup exactly the same environment you are using. Is this Ubuntu? If so, can you point me to the exact ISO and any scripts you use to prepare it for development? I will then make a new VM.
The gitian build system is working with (at least) the current LTS (12.04). I order to build the Mac TBB you do the following
Should I use the 32bit version of the LTS?
{{{ git clone https://git.torproject.org/builders/tor-browser-bundle.git cd tor-browser-bundle/gitian make prep }}} Install all the stuff you are asked to and run {{{ make prep }}} again until you have all the necessary sources (i.e. until |make prep| does not give you any errors anymore) Then do {{{ ./mkbundle-mac.sh }}} and it should give you (after a while) Mac TBBs in a new directory (currently 3.0-alpha-4). Anyway, if you have issues with setting it up. Just write me a mail and we'll sort these things out.
If you've got some logs (build.log) detailing the "fun bugs" please post them.
So I have found so far three issues (disclaimer: I am not sure yet if there is anything you can do about as this might actually be things belonging to the source packages used in building the compiler):
- I hit intermittent GMP configure failures due to newly created files being older than distributed ones. (log: cross_mac_gmp1) That is likely due to the gitian build environment but it would be good to somehow avoid that. I am wondering whether the reason here is some old version of GMP? I have been building mingw-w64 from source for a while with a similar gitian environment but never hit this problem.
In general, how does gitian handle filestamp dependencies? I'm under the impression that it uses .so injection to force specific time functions to return the same thing, given that, I'm a little confused. Is there a mechanism to force some fixed negative delta onto files created by specific tools (tar and patch come to mind)?
- 'groff' was missing (the check in the configure script is there but although 'groff' is needed for building the script is not returning with an error if it is not available) (cross_mac_build1). Installing the necessary package probably solves the issue.
Ok this sounds like an upstream crosstool-ng problem. I will put it on the TODO list for now.
- a Makefile.config(?) is missing? (cross_mac_build2).
Now, what bothers me most, though, is that Firefox 24 is not working with a gcc4.2 anymore (at least Mozilla is saying that in the configure script and I don't have a reason to doubt that). I am under the impression that your cross-compilers are (by default) gcc4.2, no? If so, what are our options here? Are there non-interactive ways to change the gcc version?
4.2.1 was as the last GCC that Apple provided. They heavily patched this version (and made llvmgcc from it) and they never updated beyond that (understandable given the level and nature of patching, of course they could've opted to make their changes cleanly and worked with GCC core developers to upstream but I suspect GCC was something they worked on grudgingly). They switched from GCC to clang, so when you say that Firefox 24 is not working with gcc4.2 anymore, I can only assume that for Mac, they switched to building with clang - or maybe llvmgcc, if you can determine the exact details of this it would be helpful, also can you find out it requires libc++? When do you plan to switch over to Firefox 24?
I will download your git repo, give it a test and study your logs at lunchtime. You didn't say whether you use the -arch flag or not, but no problem I can take a look.
Trac:
Username: mingwandroid -
Trac:
cross_mac_vm -
Trac:
Username: mingwandroid -
Trac:
Username: mingwandroid -
I added two logs, the first is actually targeting ARM (iPhone), so you probably should ignore that one for now.
Was your log complete? In mine, "[INFO ] Installing cctools for host" doesn't happen until line 42686 whereas it's the first line in your log. I'm guessing you snipped your log to avoid size limits and so that it can be referred to via URLs? I think having your complete log would be helpful if this is the case.
From your log: "[CFG ] configure: WARNING: llvm-config (/home/firefox/x-tools/i686-apple-darwin10/bin/i686-apple-darwin10-llvm-config) not found, will use LLVM 2.7 defaults" is interesting. It indicates that LLVM wasn't built or installed correctly.
Trac:
Username: mingwandroid -
Replying to mingwandroid:
I added two logs, the first is actually targeting ARM (iPhone), so you probably should ignore that one for now.
Was your log complete? In mine, "[INFO ] Installing cctools for host" doesn't happen until line 42686 whereas it's the first line in your log. I'm guessing you snipped your log to avoid size limits and so that it can be referred to via URLs? I think having your complete log would be helpful if this is the case.
From your log: "[CFG ] configure: WARNING: llvm-config (/home/firefox/x-tools/i686-apple-darwin10/bin/i686-apple-darwin10-llvm-config) not found, will use LLVM 2.7 defaults" is interesting. It indicates that LLVM wasn't built or installed correctly.
Attached is the full build log (cross_mac_vm_full.tar.bz2).