Nightly builds are failing with compiler crash in binutils for Linux 64bit

[gk]

Nightly builds are broken for a while now due to a compiler crash while building binutils for 64 bit Linux. See e.g.:

g++ -DHAVE_CONFIG_H -I.  -I. -I./../include -I./../elfcpp -DLOCALEDIR="\"/home/ubuntu/install/binutils/share/locale\"" -DBINDIR="\"/home/ubuntu/install/binutils/bin\"" -DTOOLBINDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/bin\"" -DTOOLLIBDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/lib\""   -W -Wall    -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=reduced_debug_output.o -g -O2 -MT reduced_debug_output.o -MD -MP -MF .deps/reduced_debug_output.Tpo -c -o reduced_debug_output.o reduced_debug_output.cc
make[4]: Leaving directory `/home/ubuntu/build/binutils-2.24/binutils'
make[3]: Leaving directory `/home/ubuntu/build/binutils-2.24/binutils'
make[2]: Leaving directory `/home/ubuntu/build/binutils-2.24/binutils'
g++ -DHAVE_CONFIG_H -I.  -I. -I./../include -I./../elfcpp -DLOCALEDIR="\"/home/ubuntu/install/binutils/share/locale\"" -DBINDIR="\"/home/ubuntu/install/binutils/bin\"" -DTOOLBINDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/bin\"" -DTOOLLIBDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/lib\""   -W -Wall    -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=reloc.o -g -O2 -MT reloc.o -MD -MP -MF .deps/reloc.Tpo -c -o reloc.o reloc.cc
g++-4.4.real: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <file:///usr/share/doc/gcc-4.4/README.Bugs> for instructions.
make[4]: *** [object.o] Error 1
make[4]: *** Waiting for unfinished jobs....

or

g++ -DHAVE_CONFIG_H -I.  -I. -I./../include -I./../elfcpp -DLOCALEDIR="\"/home/ubuntu/install/binutils/share/locale\"" -DBINDIR="\"/home/ubuntu/install/binutils/bin\"" -DTOOLBINDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/bin\"" -DTOOLLIBDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/lib\""   -W -Wall    -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=reloc.o -g -O2 -MT reloc.o -MD -MP -MF .deps/reloc.Tpo -c -o reloc.o reloc.cc
mv -f .deps/merge.Tpo .deps/merge.Po
g++ -DHAVE_CONFIG_H -I.  -I. -I./../include -I./../elfcpp -DLOCALEDIR="\"/home/ubuntu/install/binutils/share/locale\"" -DBINDIR="\"/home/ubuntu/install/binutils/bin\"" -DTOOLBINDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/bin\"" -DTOOLLIBDIR="\"/home/ubuntu/install/binutils/x86_64-unknown-linux-gnu/lib\""   -W -Wall    -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=resolve.o -g -O2 -MT resolve.o -MD -MP -MF .deps/resolve.Tpo -c -o resolve.o resolve.cc
g++-4.4.real: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <file:///usr/share/doc/gcc-4.4/README.Bugs> for instructions.
make[4]: *** [layout.o] Error 1
make[4]: *** Waiting for unfinished jobs....

[gk]

Two questions:
1) What qemu process is active (if there is still one) after the crash?
2) What happens if we compile with -j1?

[gk]

Okay, let's try a bigger thing with bug_14730 (​https://gitweb.torproject.org/user/gk/tor-browser-bundle.git/commit/?h=bug_14730&id=c3d85924939d65d45104c19f6683c2a288d612d9) in my public tor-browser-bundle repo. ln5: Could you just use this branch (instead of master) and start nightly builds as usual?

[gk]

FWIW: bug_14730 did not solve the problem. The crash happened again but there are hints about not having enough memory on the nightly build machine as the cause.

[gk]

Seems to be due to an underequipped machine. :)