Opened 6 years ago

Last modified 3 weeks ago

#6623 needs_revision defect

--enable-static-tor cannot succeed

Reported by: tmpname0901 Owned by:
Priority: Medium Milestone: Tor: 0.3.5.x-final
Component: Core Tor/Tor Version: Tor: 0.2.3.20-rc
Severity: Normal Keywords: tor-relay autotools build link static 029-backport 032-backport 033-backport 034-backport
Cc: Actual Points:
Parent ID: Points:
Reviewer: nickm Sponsor:

Description

Build environment: CentOS v5.8/x86_64 with GCC v4.4.6.

Use of the --enable-static-tor cannot result in a successful build, at least not with GLibC v2.5.

Use of this flag causes the --static switch to be placed on the compiler command line when running tests from the configure script. So when the script tries to verify, say, a working libevent library, the test fails. Then the script falsely believes that the intended test (e.g. finding a linkable libevent lib) has failed. The actual failure is the inability to staticly link GLibC.

Here's as example:

$ gcc44 -o /tmp/conftest -static -I/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/include -I${top_srcdir}/src/common -L/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib -L/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib /tmp/conftest.c -lpthread -ldl -lrt -levent
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/evutil.c:1105: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(evutil.o): In function `evutil_unparse_protoname':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/evutil.c:758: warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(event.o): In function `gettime':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/event.c:366: undefined reference to `clock_gettime'
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/dependencies/lib/libevent.a(event.o): In function `detect_monotonic':
/home/rpmbuild/BUILD/tor-0.2.3.20-rc/libevent-2.0.19-stable/event.c:336: undefined reference to `clock_gettime'

Ouch! Guess it couldn't find a working libevent.a, right? Wrong!

What linker couldn't do is staticly link GLibC's runtime libraries. Remove that --static from the command line and the configure script's test succeeds in linking against the specified static libevent library.

Tor can be successfully linked against static zlib/openssl/libevent libraries but it cannot be staticly linked against gLibC. This makes --enable-static-tor pointless, a least for Linux builds.

Child Tickets

TicketStatusOwnerSummaryComponent
#26464needs_informationHello71Static cross-compiling for Windows is brokenCore Tor/Tor
#27802newOpenSSL 1.1.0 issue during static linkCore Tor/Tor

Change History (24)

comment:1 Changed 6 years ago by nickm

Milestone: Tor: 0.2.3.x-finalTor: 0.2.4.x-final

Putting this in 0.2.4; I want to clean up our library search and build system heavily, but the way it stands now, it's way too fragile to fix in 0.2.3.x without having a high risk of breaking stuff.

comment:2 Changed 6 years ago by nickm

Keywords: tor-relay added

comment:3 Changed 6 years ago by nickm

Component: Tor RelayTor

comment:4 Changed 6 years ago by nickm

Milestone: Tor: 0.2.4.x-finalTor: 0.2.5.x-final

comment:5 Changed 5 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.???

comment:6 Changed 4 years ago by nickm

Milestone: Tor: 0.2.???Tor: 0.2.7.x-final

These may be worth looking at for 0.2.7.

comment:7 Changed 4 years ago by nickm

Status: newassigned

comment:8 Changed 4 years ago by nickm

Keywords: 027-triaged-1-out added

Marking triaged-out items from first round of 0.2.7 triage.

comment:9 Changed 4 years ago by nickm

Milestone: Tor: 0.2.7.x-finalTor: 0.2.???

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

comment:10 Changed 2 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:11 Changed 22 months ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:12 Changed 17 months ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:13 Changed 17 months ago by nickm

Keywords: 027-triaged-in added

comment:14 Changed 17 months ago by nickm

Keywords: 027-triaged-in removed

comment:15 Changed 17 months ago by nickm

Keywords: 027-triaged-1-out removed

comment:16 Changed 17 months ago by nickm

Status: assignednew

Change the status of all assigned/accepted Tor tickets with owner="" to "new".

comment:17 Changed 17 months ago by nickm

Keywords: autotools build link static added
Severity: Normal

comment:18 Changed 4 weeks ago by sysrqb

Status: newneeds_review

Okay, I have a branch for this. The reason this was failing is because library order was partially reversed for linking. There are three remaining issues that need some clarification. Please see branch bug6623 in my public user repo.

However, overall, I wonder if there is any benefit in supporting static linking at this point. (To be clear, I don't actually have a use for this)


The backtrace unit tests fail because the symbols are not shown/available. This needs further investigation.

Statically linked:

$ ./src/test/test-bt-cl crash

============================================================ T= 1537399469
Tor died: Caught signal 11
[0x4074c9]
[0x400df8]
[0x400df8]
[0x400e4d]
[0x400e9d]
[0x400eed]
[0x4007d3]
[0x41aac6]
[0x41aec1]
[0x400cca]
Aborted (core dumped)
$ ./src/test/test-bt-cl assert
Sep 19 23:25:07.037 [err] tor_assertion_failed_(): Bug: src/test/test_bt_cl.c:46: crash: Assertion 1 == 0 failed; aborting. (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug: Assertion 1 == 0 failed in crash at src/test/test_bt_cl.c:46. Stack trace: (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x4075a5] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x404294] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400e29] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400e4d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400e9d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400eed] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x4007d3] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x41aac6] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x41aec1] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 19 23:25:07.037 [err] Bug:     [0x400cca] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Aborted (core dumped)

vs. Dynamically linked:

$ ./src/test/test-bt-cl crash

============================================================ T= 1537404853
Tor died: Caught signal 11
./src/test/test-bt-cl(+0xa2a9)[0x576acdd052a9]
./src/test/test-bt-cl(crash+0x48)[0x576acdcfebd8]
./src/test/test-bt-cl(crash+0x48)[0x576acdcfebd8]
./src/test/test-bt-cl(oh_what+0x1d)[0x576acdcfec2d]
./src/test/test-bt-cl(a_tangled_web+0x1d)[0x576acdcfec7d]
./src/test/test-bt-cl(we_weave+0x1d)[0x576acdcfeccd]
./src/test/test-bt-cl(main+0xa3)[0x576acdcfe9d3]
/lib64/libc.so.6(__libc_start_main+0xea)[0x726f7219d88a]
./src/test/test-bt-cl(_start+0x2a)[0x576acdcfeaaa]
Aborted (core dumped)
$ ./src/test/test-bt-cl assert
Sep 20 00:47:26.187 [err] tor_assertion_failed_(): Bug: src/test/test_bt_cl.c:46: crash: Assertion 1 == 0 failed; aborting. (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug: Assertion 1 == 0 failed in crash at src/test/test_bt_cl.c:46. Stack trace: (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(log_backtrace_impl+0x45) [0x5b334ff78385] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(tor_assertion_failed_+0x94) [0x5b334ff75074] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(crash+0x79) [0x5b334ff71c09] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(oh_what+0x1d) [0x5b334ff71c2d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(a_tangled_web+0x1d) [0x5b334ff71c7d] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(we_weave+0x1d) [0x5b334ff71ccd] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(main+0xa3) [0x5b334ff719d3] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xea) [0x7fcc6186a88a] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Sep 20 00:47:26.188 [err] Bug:     ./src/test/test-bt-cl(_start+0x2a) [0x5b334ff71aaa] (on Tor 0.3.5.0-alpha-dev 274ad5d643e969a6)
Aborted (core dumped)

The util/pwdb unit test is failing because of the static linking (I think) - see next section, too:

Statically linked:

$ ./src/test/test util/pwdb
util/pwdb: [forking] 
============================================================ T= 1537406368
Tor 0.3.5.0-alpha-dev (git-274ad5d643e969a6) died: Caught signal 11
[0x85d1e9]
/lib64/libnss_systemd.so.2(+0x18f5d)[0x75a3c39bef5d]
/lib64/libnss_systemd.so.2(+0x18f5d)[0x75a3c39bef5d]
/lib64/libnss_systemd.so.2(+0x26d90)[0x75a3c39ccd90]
/lib64/libnss_systemd.so.2(_nss_systemd_getpwnam_r+0x3b0)[0x75a3c39cea50]
[0xab5775]
[0xab53aa]
[0x842f29]
[0x6645c1]
[0x6c0856]
[0x6c0ba1]
[0x6c115c]
[0x407105]
[0xa4dff6]
[0xa4e3f1]
[0x40765a]
[Lost connection!] 
  [pwdb FAILED]
1/1 TESTS FAILED. (0 skipped)

vs. Dynamically linked:

$ ./src/test/test util/pwdb
util/pwdb: [forking] OK
1 tests ok.  (0 skipped)

During linking, the following messages are emitted:

  CCLD     src/app/tor
/usr/lib64/libcrypto.a(fips.o): In function `verify_checksums':
(.text+0x4fb): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-fs.a(dir.o): In function `check_private_dir':
/home/user/tor/src/lib/fs/dir.c:216: warning: Using 'getgrgid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-fs.a(userdb.o): In function `tor_getpwnam':
/home/user/tor/src/lib/fs/userdb.c:80: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-fs.a(userdb.o): In function `tor_getpwuid':
/home/user/tor/src/lib/fs/userdb.c:110: warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
src/lib/libtor-net.a(resolve.o): In function `tor_addr_lookup':
/home/user/tor/src/lib/net/resolve.c:102: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib64/libcrypto.a(b_sock.o): In function `BIO_gethostbyname':
(.text+0x71): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/user/repos/libevent/.libs/libevent.a(evutil.o): In function `evutil_unparse_protoname':
/home/user/repos/libevent/evutil.c:911: warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/user/repos/libevent/.libs/libevent.a(evutil.o): In function `evutil_parse_servname':
/home/user/repos/libevent/evutil.c:883: warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

I configured and tested the branch with:

$ ./autogen.sh && ./configure --disable-asciidoc --enable-pic --enable-android --enable-fatal-warnings --enable-static-openssl --with-ssl-dir=/usr/lib64 --enable-static-zlib --with-zlib-dir=/usr/lib64 --enable-static-libevent --with-libevent-dir=../libevent/.libs --enable-static-tor

On Fedora I installed openssl-static, zlib-static, glibc-static. I compiled libevent (release-2.1.8-stable) from source.

comment:19 Changed 4 weeks ago by teor

Keywords: 029-backport 032-backport 033-backport 034-backport added
Milestone: Tor: unspecifiedTor: 0.3.5.x-final

I am not sure whether to put this fix in 0.3.5 or 0.3.6, but it would be nice to have a working static binary in our LTS release.

It might also be good to backport these fixes, if they aren't too big,

comment:20 Changed 4 weeks ago by dgoulet

Reviewer: nickm

comment:21 Changed 3 weeks ago by nickm

I'd actually like to deprecate the TOR_SEARCH_LIBRARY nonsense if possible, and not add its use for zlib. (I don't understand why this patch starts using TOR_SEARCH_LIBRARY for zlib, but I assume it's necessary? Can we use pkg-config instead?)

When I try to use this patch with --enable-static-tor, it fails during configuration with

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Am I doing something wrong?

comment:22 in reply to:  21 ; Changed 3 weeks ago by sysrqb

Replying to nickm:

I'd actually like to deprecate the TOR_SEARCH_LIBRARY nonsense if possible, and not add its use for zlib. (I don't understand why this patch starts using TOR_SEARCH_LIBRARY for zlib, but I assume it's necessary? Can we use pkg-config instead?)

Hrm, it shouldn't. I simply moved the zlib check from being after the openssl checks to being before it (because openssl depends on zlib).

When I try to use this patch with --enable-static-tor, it fails during configuration with

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Am I doing something wrong?

Nope, that is possible. I ran into that, too. Do you have glibc-static installed? -lc is probably resulting in an error due to the lack of a static library. I should've added a more helpful error message for this case. I can do that.

comment:23 in reply to:  22 Changed 3 weeks ago by nickm

Replying to sysrqb:

Replying to nickm:

I'd actually like to deprecate the TOR_SEARCH_LIBRARY nonsense if possible, and not add its use for zlib. (I don't understand why this patch starts using TOR_SEARCH_LIBRARY for zlib, but I assume it's necessary? Can we use pkg-config instead?)

Hrm, it shouldn't. I simply moved the zlib check from being after the openssl checks to being before it (because openssl depends on zlib).

Oh! I didn't notice that this was moved code. Okay, no need to change stuff in this patch.

When I try to use this patch with --enable-static-tor, it fails during configuration with

checking whether free(NULL) works... no
configure: error: Your libc implementation doesn't allow free(NULL), as required by C99.

Am I doing something wrong?

Nope, that is possible. I ran into that, too. Do you have glibc-static installed? -lc is probably resulting in an error due to the lack of a static library. I should've added a more helpful error message for this case. I can do that.

A better error message would be helpful, yes!

comment:24 Changed 3 weeks ago by dgoulet

Status: needs_reviewneeds_revision
Note: See TracTickets for help on using tickets.