Opened 8 years ago

Last modified 6 months ago

#6623 needs_revision defect

--enable-static-tor cannot succeed

Reported by: tmpname0901 Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.2.3.20-rc
Severity: Normal Keywords: tor-relay, autotools, build, link, static, 032-unreached-backport, 035-deferred-20190115, 041-proposed, 033-unreached-backport
Cc: Actual Points:
Parent ID: Points:
Reviewer: nickm Sponsor:

Description

Build environment: CentOS v5.8/x86_64 with GCC v4.4.6.

Use of the --enable-static-tor cannot result in a successful build, at least not with GLibC v2.5.

Use of this flag causes the --static switch to be placed on the compiler command line when running tests from the configure script. So when the script tries to verify, say, a working libevent library, the test fails. Then the script falsely believes that the intended test (e.g. finding a linkable libevent lib) has failed. The actual failure is the inability to staticly link GLibC.

Here's as example:

$ gcc44 -o /tmp/conftest -static -I/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/include -I${top_srcdir}/src/common -L/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib -L/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib /tmp/conftest.c -lpthread -ldl -lrt -levent
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/evutil.c:1105: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(evutil.o): In function `evutil_unparse_protoname':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/evutil.c:758: warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(event.o): In function `gettime':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/event.c:366: undefined reference to `clock_gettime'
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(event.o): In function `detect_monotonic':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/event.c:336: undefined reference to `clock_gettime'

Ouch! Guess it couldn't find a working libevent.a, right? Wrong!

What linker couldn't do is staticly link GLibC's runtime libraries. Remove that --static from the command line and the configure script's test succeeds in linking against the specified static libevent library.

Tor can be successfully linked against static zlib/openssl/libevent libraries but it cannot be staticly linked against gLibC. This makes --enable-static-tor pointless, a least for Linux builds.

Child Tickets

TicketStatusOwnerSummaryComponent
#26464newHello71Static cross-compiling for Windows is brokenCore Tor/Tor
#27802newOpenSSL 1.1.0 issue during static linkCore Tor/Tor

Change History (33)

comment:1 Changed 8 years ago by nickm

Milestone: Tor: 0.2.3.x-finalTor: 0.2.4.x-final

Putting this in 0.2.4; I want to clean up our library search and build system heavily, but the way it stands now, it's way too fragile to fix in 0.2.3.x without having a high risk of breaking stuff.

comment:2 Changed 8 years ago by nickm

Keywords: tor-relay added

comment:3 Changed 8 years ago by nickm

Component: Tor RelayTor

comment:4 Changed 8 years ago by nickm

Milestone: Tor: 0.2.4.x-finalTor: 0.2.5.x-final

comment:5 Changed 7 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.???

comment:6 Changed 6 years ago by nickm

Milestone: Tor: 0.2.???Tor: 0.2.7.x-final

These may be worth looking at for 0.2.7.

comment:7 Changed 6 years ago by nickm

Status: newassigned

comment:8 Changed 6 years ago by nickm

Keywords: 027-triaged-1-out added

Marking triaged-out items from first round of 0.2.7 triage.

comment:9 Changed 6 years ago by nickm

Milestone: Tor: 0.2.7.x-finalTor: 0.2.???

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

comment:10 Changed 4 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:11 Changed 4 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:12 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:13 Changed 3 years ago by nickm

Keywords: 027-triaged-in added

comment:14 Changed 3 years ago by nickm

Keywords: 027-triaged-in removed

comment:15 Changed 3 years ago by nickm

Keywords: 027-triaged-1-out removed

comment:16 Changed 3 years ago by nickm

Status: assignednew

Change the status of all assigned/accepted Tor tickets with owner="" to "new".

comment:17 Changed 3 years ago by nickm

Keywords: autotools build link static added
Severity: Normal

comment:18 Changed 2 years ago by sysrqb

Status: newneeds_review

Okay, I have a branch for this. The reason this was failing is because library order was partially reversed for linking. There are three remaining issues that need some clarification. Please see branch bug6623 in my public user repo.

However, overall, I wonder if there is any benefit in supporting static linking at this point. (To be clear, I don't actually have a use for this)


The backtrace unit tests fail because the symbols are not shown/available. This needs further investigation.

Statically linked:

$ ./src/test/test-bt-cl crash

============================================================ T= 1537399469
Tor died: Caught signal 11
[0x4074c9]
[0x400df8]
[0x400df8]
[0x400e4d]
[0x400e9d]
[0x400eed]
[0x4007d3]
[0x41aac6]
[0x41aec1]
[0x400cca]
Aborted (core dumped)
$ ./src/test/test-bt-cl assert
Sep 19 23:25:07.037 [err] tor_assertion_failed_(): Bug: src/test/test_bt_cl.c:46: crash: Assertion 1 == 0 failed; aborting. (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug: Assertion 1 == 0 failed in crash at src/test/test_bt_cl.c:46. Stack trace: (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x4075a5] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x404294] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400e29] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400e4d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400e9d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400eed] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x4007d3] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x41aac6] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x41aec1] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400cca] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Aborted (core dumped)

vs. Dynamically linked:

$ ./src/test/test-bt-cl crash

============================================================ T= 1537404853
Tor died: Caught signal 11
./src/test/test-bt-cl(+0xa2a9)[0x576acdd052a9]
./src/test/test-bt-cl(crash+0x48)[0x576acdcfebd8]
./src/test/test-bt-cl(crash+0x48)[0x576acdcfebd8]
./src/test/test-bt-cl(oh_what+0x1d)[0x576acdcfec2d]
./src/test/test-bt-cl(a_tangled_web+0x1d)[0x576acdcfec7d]
./src/test/test-bt-cl(we_weave+0x1d)[0x576acdcfeccd]
./src/test/test-bt-cl(main+0xa3)[0x576acdcfe9d3]
/lib64/libc.so.6(__libc_start_main+0xea)[0x726f7219d88a]
./src/test/test-bt-cl(_start+0x2a)[0x576acdcfeaaa]
Aborted (core dumped)
$ ./src/test/test-bt-cl assert
Sep 20 00:47:26.187 [err] tor_assertion_failed_(): Bug: src/test/test_bt_cl.c:46: crash: Assertion 1 == 0 failed; aborting. (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug: Assertion 1 == 0 failed in crash at src/test/test_bt_cl.c:46. Stack trace: (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(log_backtrace_impl+0x45) [0x5b334ff78385] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(tor_assertion_failed_+0x94) [0x5b334ff75074] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(crash+0x79) [0x5b334ff71c09] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(oh_what+0x1d) [0x5b334ff71c2d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(a_tangled_web+0x1d) [0x5b334ff71c7d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(we_weave+0x1d) [0x5b334ff71ccd] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(main+0xa3) [0x5b334ff719d3] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xea) [0x7fcc6186a88a] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(_start+0x2a) [0x5b334ff71aaa] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Aborted (core dumped)

The util/pwdb unit test is failing because of the static linking (I think) - see next section, too:

Statically linked:

$ ./src/test/test util/pwdb
util/pwdb: [forking] 
============================================================ T= 1537406368
Tor 0.3.5.0-alpha-dev (git-274ad5d643e969a6) died: Caught signal 11
[0x85d1e9]
/lib64/libnss_systemd.so.2(+0x18f5d)[0x75a3c39bef5d]
/lib64/libnss_systemd.so.2(+0x18f5d)[0x75a3c39bef5d]
/lib64/libnss_systemd.so.2(+0x26d90)[0x75a3c39ccd90]
/lib64/libnss_systemd.so.2(_nss_systemd_getpwnam_r+0x3b0)[0x75a3c39cea50]
[0xab5775]
[0xab53aa]
[0x842f29]
[0x6645c1]
[0x6c0856]
[0x6c0ba1]
[0x6c115c]
[0x407105]
[0xa4dff6]
[0xa4e3f1]
[0x40765a]
[Lost connection!] 
  [pwdb FAILED]
1/1 TESTS FAILED. (0 skipped)

vs. Dynamically linked:

$ ./src/test/test util/pwdb
util/pwdb: [forking] OK
1 tests ok.  (0 skipped)

During linking, the following messages are emitted:

  CCLD     src/app/tor
/usr/lib64/libcrypto.a(fips.o): In function `verify_checksums':
(.text+0x4fb): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-fs.a(dir.o): In function `check_private_dir':
/home/user/tor/src/lib/fs/dir.c:216: warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-fs.a(userdb.o): In function `tor_getpwnam':
/home/user/tor/src/lib/fs/userdb.c:80: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-fs.a(userdb.o): In function `tor_getpwuid':
/home/user/tor/src/lib/fs/userdb.c:110: warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-net.a(resolve.o): In function `tor_addr_lookup':
/home/user/tor/src/lib/net/resolve.c:102: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib64/libcrypto.a(b_sock.o): In function `BIO_gethostbyname':
(.text+0x71): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/user/repos/libevent/.libs/libevent.a(evutil.o): In function `evutil_unparse_protoname':
/home/user/repos/libevent/evutil.c:911: warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/user/repos/libevent/.libs/libevent.a(evutil.o): In function `evutil_parse_servname':
/home/user/repos/libevent/evutil.c:883: warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

I configured and tested the branch with:

$ ./autogen.sh && ./configure --disable-asciidoc --enable-pic --enable-android --enable-fatal-warnings --enable-static-openssl --with-ssl-dir=/usr/lib64 --enable-static-zlib --with-zlib-dir=/usr/lib64 --enable-static-libevent --with-libevent-dir=../libevent/.libs --enable-static-tor

On Fedora I installed openssl-static, zlib-static, glibc-static. I compiled libevent (release-2.1.8-stable) from source.

comment:19 Changed 2 years ago by teor

Keywords: 029-backport 032-backport 033-backport 034-backport added
Milestone: Tor: unspecifiedTor: 0.3.5.x-final

I am not sure whether to put this fix in 0.3.5 or 0.3.6, but it would be nice to have a working static binary in our LTS release.

It might also be good to backport these fixes, if they aren't too big,

comment:20 Changed 2 years ago by dgoulet

Reviewer: nickm

comment:21 Changed 2 years ago by nickm

I'd actually like to deprecate the TOR_SEARCH_LIBRARY nonsense if possible, and not add its use for zlib. (I don't understand why this patch starts using TOR_SEARCH_LIBRARY for zlib, but I assume it's necessary? Can we use pkg-config instead?)

When I try to use this patch with --enable-static-tor, it fails during configuration with

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Am I doing something wrong?

comment:22 in reply to:  21 ; Changed 2 years ago by sysrqb

Replying to nickm:

I'd actually like to deprecate the TOR_SEARCH_LIBRARY nonsense if possible, and not add its use for zlib. (I don't understand why this patch starts using TOR_SEARCH_LIBRARY for zlib, but I assume it's necessary? Can we use pkg-config instead?)

Hrm, it shouldn't. I simply moved the zlib check from being after the openssl checks to being before it (because openssl depends on zlib).

When I try to use this patch with --enable-static-tor, it fails during configuration with

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Am I doing something wrong?

Nope, that is possible. I ran into that, too. Do you have glibc-static installed? -lc is probably resulting in an error due to the lack of a static library. I should've added a more helpful error message for this case. I can do that.

comment:23 in reply to:  22 Changed 2 years ago by nickm

Replying to sysrqb:

Replying to nickm:

I'd actually like to deprecate the TOR_SEARCH_LIBRARY nonsense if possible, and not add its use for zlib. (I don't understand why this patch starts using TOR_SEARCH_LIBRARY for zlib, but I assume it's necessary? Can we use pkg-config instead?)

Hrm, it shouldn't. I simply moved the zlib check from being after the openssl checks to being before it (because openssl depends on zlib).

Oh! I didn't notice that this was moved code. Okay, no need to change stuff in this patch.

When I try to use this patch with --enable-static-tor, it fails during configuration with

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Am I doing something wrong?

Nope, that is possible. I ran into that, too. Do you have glibc-static installed? -lc is probably resulting in an error due to the lack of a static library. I should've added a more helpful error message for this case. I can do that.

A better error message would be helpful, yes!

comment:24 Changed 2 years ago by dgoulet

Status: needs_reviewneeds_revision

comment:25 Changed 2 years ago by teor

Keywords: 032-unreached-backport added; 032-backport removed

0.3.2 is end of life, so 032-backport is now 032-unreached-backport.

comment:26 Changed 22 months ago by nickm

Keywords: 035-deferred-20190115 041-proposed added
Milestone: Tor: 0.3.5.x-finalTor: unspecified

Marking a number of 0.3.5 tickets as possible, maybe even a good idea, for later. Possibly backportable, some of them. But not currently things to do as part of 0.3.5 stabilization.

comment:27 Changed 20 months ago by teor

Keywords: 033-backport removed

These open, non-merge_ready tickets can not get backported to 0.3.3, because 0.3.3 is now unsupported.

comment:28 Changed 20 months ago by teor

Keywords: 033-backport-unreached added

Hmm, I guess they should still get 033-backport-unreached

comment:29 Changed 17 months ago by nickm

Keywords: 034-backport removed

Removing 034-backport from all open tickets: 034 has reached EOL.

comment:30 Changed 15 months ago by teor

Keywords: 033-unreached-backport added; 033-backport-unreached removed

Fix 033-unreached-backport spelling.

comment:31 Changed 10 months ago by teor

Keywords: 029-backport removed

0.2.9 is no longer maintained as of 1 January 2020. Therefore, we won't be backporting anything to it.

(None of these tickets are merge_ready, so we can just delete the backport tag.)

comment:32 Changed 6 months ago by werd

I want to build tor, not download it from somewhere else. My computer is on wifi so that prevents me from being an acceptable relay or bridge, best I can tell. But after I copy over the binaries I made, to a computer with ethernet to the router, and run tor I get this:

dyld: Library not loaded: @rpath/libclang_rt.asan_osx_dynamic.dylib
  Referenced from: /path/to/tor/bin/tor
  Reason: image not found
Abort trap: 6

So it seems tor is relying on a library burried deep in xcode.

I've already built zlib, libevent, and libressl/openssl as static. My tor configure line informs tor of this. But as soon as I add --enable-static-tor and ./configure, the checking stops (as noted above in comment 21) here:

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Based on my reading of this ticket and others, it seems there is something out of order, but my attempts to mimic what others said worked for them, has not worked for me.

I changed the order as noted in the patch here: #27802, (lines 169 & 222), but the same error happened.

comment:33 in reply to:  32 Changed 6 months ago by teor

Replying to werd:

I want to build tor, not download it from somewhere else. My computer is on wifi so that prevents me from being an acceptable relay or bridge, best I can tell. But after I copy over the binaries I made, to a computer with ethernet to the router, and run tor I get this:

dyld: Library not loaded: @rpath/libclang_rt.asan_osx_dynamic.dylib
  Referenced from: /path/to/tor/bin/tor
  Reason: image not found
Abort trap: 6

So it seems tor is relying on a library burried deep in xcode.

This is the clang address sanitizer library. We don't recommend using it for relays or bridges, because it makes them crash on some non-fatal errors.

You can disable this library by removing --enable-expensive-hardening or --enable-fragile-hardening from your configure command-line.

I've already built zlib, libevent, and libressl/openssl as static. My tor configure line informs tor of this. But as soon as I add --enable-static-tor and ./configure, the checking stops (as noted above in comment 21) here:

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Based on my reading of this ticket and others, it seems there is something out of order, but my attempts to mimic what others said worked for them, has not worked for me.

I changed the order as noted in the patch here: #27802, (lines 169 & 222), but the same error happened.

Does macOS ship with a static C library?

As far as I remember, it's not possible to build a truly static binary on macOS, because some of the lower-level OS libraries are dynamic. But you can get close.

And I think you're actually trying to solve a slightly different problem, which is transferring a binary from one macOS to another. That shouldn't require a static tor, Tor Browser compiles a single binary that works on multiple macOS versions.

Have you tried passing -mmacosx-version-min=version to your C compiler? That will disable any features that aren't available on newer macOS.

Have you tried building all the dynamic libraries, and then copying them across with the binary? You may need to set the library paths correctly in the binary, or set LD_LIBRARY_PATH before launching it.

You might also be happier if you use a package manager like HomeBrew to build from source on the wired Mac. It should have package definitions and debugging output that let you modify and verify the build.

Note: See TracTickets for help on using tickets.