Untange kludgey library detection, particularly for SSL forks

Split from #13415 (moved):

teor:

LibreSSL

I'm having trouble getting LibreSSL (2.1.2) to work with tor git on OS X 10.9.

Here are the issues I've found and fixed in the configure invocation:

configure --with-openssl-dir= detects the wrong bin/openssl if "$OPENSSL_DIR/bin/openssl" isn't in the path before all other openssl executables.
configure --enable-static-openssl requires LDFLAGS="$OPENSSL_DIR/lib":$LDFLAGS to link properly, at least on OS X. 

I'm pretty sure these issues will affect all (non-system/non-standard) SSLs.

Can we make configuring with non-system SSLs easier by prepending "

OPENSSL_DIR/bin" and "

OPENSSL_DIR/lib" to the PATH and LDFLAGS respectively?

BoringSSL

BoringSSL is even worse - it doesn't even have an openssl executable, only builds static libraries, and is a pain to configure correctly under our current config scripts.

I can't seem to stop it finding the system-supplied SSL, even when I provide it the BoringSSL directories.

I get the following warnings when I manually install BoringSSL into include/lib/bin dirs, and fake the openssl executable using the bssl executable:

(See #13815 (moved))

nickm:

(The build issues are another matter and should get their own ticket: Untangling our kludgey library detection has been something a bunch of people have wanted for a while.)

changed milestone to %Tor: unspecified

added build component::core tor/tor library-detection milestone::Tor: unspecified parent::6311 priority::high severity::normal status::new tor-build type::defect version::tor 0.2.6.1-alpha labels

On OS X 10.9, which has a system-installed OpenSSL 0.9.8, the EVP_aes_128_ctr function is missing when the system OpenSSL is used instead of LibreSSL.

Check out #1354 (moved) here, too

#6311 (moved) could fix this - Migrate TOR_SEARCH_LIBRARY to use pkg-config

Replying to teor:

#6311 (moved) could fix this - Migrate TOR_SEARCH_LIBRARY to use pkg-config

Using pkg-config for our library detection would also solve #10304 (moved).

Assigning #6311 (moved) as the parent of tasks that could be resolved by pkg-config

Trac:
Parent: #13415 (moved) to #6311 (moved)

From: https://lists.torproject.org/pipermail/tor-relays/2014-November/005812.html https://lists.torproject.org/pipermail/tor-relays/2014-November/005822.html

Seth:

I'm trying to build tor-0.2.5.10 from source against LibreSSL 2.1.1 on a
FreeBSD 9.3x jail system.

It fails with this message

---

CC src/tools/tor-gencert.o CCLD src/tools/tor-gencert src/common/libor-crypto.a(aes.o): In function aes_new_cipher': /usr/local/src/tor-0.2.5.10/src/common/aes.c:100: undefined reference to EVP_aes_128_ctr' *** [src/tools/tor-gencert] Error code 1

Stop in /usr/local/src/tor-0.2.5.10. *** [all] Error code 1

Stop in /usr/local/src/tor-0.2.5.10.

---

Has anyone has any luck building Tor against LibreSSL?

teor:

Yes, on OS X, but it wasn't easy, and it didn't bootstrap for me due to
SSL errors. Others have had more luck, but mostly on Linux AFAIK.

Do you perhaps have a system-installed OpenSSL 0.9.* which is lacking
EVP_aes_128_ctr?

See https://trac.torproject.org/projects/tor/ticket/13817 for a similar
failure, due to the following issues:

configure --with-openssl-dir= detects the wrong bin/openssl if
"

OPENSSL_DIR/bin/openssl" isn't in the path before all other openssl executables. configure --enable-static-openssl requires LDFLAGS="

OPENSSL_DIR/lib":$LDFLAGS to link properly, at least on OS X.

If you do run into runtime SSL errors, see this bug: https://trac.torproject.org/projects/tor/ticket/13816

Seth: Thanks for the information. I was able to get the latest git version of
Tor build against the libressl-2.1.1 pkg in a fresh FreeBSD 9x jail using
the following steps:

pkg install libressl autoconf git gmake gettext mkdir /usr/local/src;cd /usr/local/src;git clone
https://git.torproject.org/git/tor cd tor;sh autogen.sh;./configure --with-openssl-dir=/usr/local
--disable-asciidoc make;make install;tor

Here's the terminal output when launching it:

Nov 22 17:26:41.971 [notice] Tor v0.2.6.1-alpha-dev (git-336c856e52d211aa)
running on FreeBSD with Libevent 2.0.21-stable, OpenSSL LibreSSL 2.1 and
Zlib 1.2.8. Nov 22 17:26:41.971 [notice] Tor can't help you if you use it wrong! Learn
how to be safe at https://www.torproject.org/download/download#warning Nov 22 17:26:41.971 [notice] This version is not a stable Tor release.
Expect more bugs than usual. Nov 22 17:26:41.972 [notice] Configuration file "/usr/local/etc/tor/torrc"
not present, using reasonable defaults. Nov 22 17:26:41.987 [notice] Opening Socks listener on 127.0.0.1:9050 Nov 22 17:26:41.971 [notice] Tor v0.2.6.1-alpha-dev (git-336c856e52d211aa)
running on FreeBSD with Libevent 2.0.21-stable, OpenSSL LibreSSL 2.1 and
Zlib 1.2.8. Nov 22 17:26:41.971 [notice] Tor can't help you if you use it wrong! Learn
how to be safe at https://www.torproject.org/download/download#warning Nov 22 17:26:41.971 [notice] This version is not a stable Tor release.
Expect more bugs than usual. Nov 22 17:26:41.972 [notice] Configuration file "/usr/local/etc/tor/torrc"
not present, using reasonable defaults. Nov 22 17:26:41.987 [notice] Opening Socks listener on 127.0.0.1:9050 Nov 22 17:26:41.000 [notice] Parsing GEOIP IPv4 file
/usr/local/share/tor/geoip. Nov 22 17:26:42.000 [notice] Parsing GEOIP IPv6 file
/usr/local/share/tor/geoip6. Nov 22 17:26:42.000 [warn] You are running Tor as root. You don't need to,
and you probably shouldn't. Nov 22 17:26:42.000 [notice] We were built to run on a 64-bit CPU, with
OpenSSL 1.0.1 or later, but with a version of OpenSSL that apparently
lacks accelerated support for the NIST P-224 and P-256 groups. Building
openssl with such support (using the enable-ec_nistp_64_gcc_128 option
when configuring it) would make ECDH much faster. Nov 22 17:26:42.000 [notice] Bootstrapped 0%: Starting Nov 22 17:26:43.000 [notice] Bootstrapped 5%: Connecting to directory
server Nov 22 17:26:43.000 [notice] Bootstrapped 10%: Finishing handshake with
directory server Nov 22 17:26:43.000 [notice] We weren't able to find support for all of
the TLS ciphersuites that we wanted to advertise. This won't hurt
security, but it might make your Tor (if run as a client) more easy for
censors to block. Nov 22 17:26:43.000 [notice] To correct this, use a version of OpenSSL
built with none of its ciphers disabled. Nov 22 17:26:44.000 [notice] Bootstrapped 15%: Establishing an encrypted
directory connection Nov 22 17:26:44.000 [notice] Bootstrapped 20%: Asking for networkstatus
consensus Nov 22 17:26:45.000 [notice] Bootstrapped 25%: Loading networkstatus
consensus Nov 22 17:26:47.000 [notice] I learned some more directory information,
but not enough to build a circuit: We have no usable consensus. Nov 22 17:26:48.000 [notice] Bootstrapped 40%: Loading authority key certs Nov 22 17:26:49.000 [notice] Bootstrapped 45%: Asking for relay descriptors Nov 22 17:26:49.000 [notice] I learned some more directory information,
but not enough to build a circuit: We need more microdescriptors: we have
0/6624, and can only build 0% of likely paths. (We have 0% of guards bw,
0% of midpoint bw, and 0% of exit bw.) Nov 22 17:26:50.000 [notice] Bootstrapped 50%: Loading relay descriptors Nov 22 17:26:53.000 [notice] Bootstrapped 55%: Loading relay descriptors Nov 22 17:26:54.000 [notice] Bootstrapped 60%: Loading relay descriptors Nov 22 17:26:54.000 [notice] Bootstrapped 65%: Loading relay descriptors Nov 22 17:26:55.000 [notice] Bootstrapped 70%: Loading relay descriptors Nov 22 17:26:55.000 [notice] Bootstrapped 75%: Loading relay descriptors Nov 22 17:26:55.000 [notice] We now have enough directory information to
build circuits. Nov 22 17:26:55.000 [notice] Bootstrapped 80%: Connecting to the Tor
network Nov 22 17:26:55.000 [notice] Bootstrapped 90%: Establishing a Tor circuit Nov 22 17:26:56.000 [notice] Tor has successfully opened a circuit. Looks
like client functionality is working. Nov 22 17:26:56.000 [notice] Bootstrapped 100%: Done

after upgrading the FreeBSD 9.3 LibreSSL pkg from 2.1.1 to 2.1.2 the above method now fails with the error:

src/common/tortls.c: In function 'find_cipher_by_id': src/common/tortls.c:1480: error: 'SSL_METHOD' has no member named 'get_cipher_by_char' src/common/tortls.c:1486: error: 'SSL_METHOD' has no member named 'get_cipher_by_char' *** [src/common/tortls.o] Error code 1

This happens whether trying to build Tor stable 0.2.5.10, Tor 0.2.6.1-alpha, or the latest version from git.

Trac:
Username: sysfu

Worth looking at during 0.2.7 triage IMO

Trac:
Milestone: Tor: unspecified to Tor: 0.2.7.x-final

Trac:
Status: new to assigned

Marking triaged-out items from first round of 0.2.7 triage.

Trac:
Keywords: N/A deleted, 027-triaged-1-out added

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

Trac:
Milestone: Tor: 0.2.7.x-final to Tor: 0.2.???

Milestone renamed

Trac:
Milestone: Tor: 0.2.??? to Tor: 0.3.???

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

Trac:
Milestone: Tor: 0.3.??? to Tor: unspecified
Keywords: N/A deleted, tor-03-unspecified-201612 added

Remove an old triaging keyword.

Trac:
Keywords: tor-03-unspecified-201612 deleted, N/A added

Trac:
Keywords: N/A deleted, 027-triaged-in added

Trac:
Keywords: 027-triaged-in deleted, N/A added

Trac:
Keywords: 027-triaged-1-out deleted, N/A added

Change the status of all assigned/accepted Tor tickets with owner="" to "new".

Trac:
Status: assigned to new

Trac:
Reviewer: N/A to N/A
Severity: N/A to Normal
Keywords: lorax deleted, tor-build build library-detection added
Priority: Medium to High
Sponsor: N/A to N/A

mentioned in issue #13977 (moved)

moved to tpo/core/tor#13817 (closed)