TLS compilation warnings and unit test failures

Have these been fixed?

  CC       src/test/src_test_test-test_tortls.o
src/test/test_tortls.c:1394:32: warning: unused variable 'wbuf_c'
      [-Wunused-variable]
  size_t rbuf_c=-1, rbuf_b=-1, wbuf_c=-1, wbuf_b=-1;
                               ^
src/test/test_tortls.c:1394:21: warning: unused variable 'rbuf_b'
      [-Wunused-variable]
  size_t rbuf_c=-1, rbuf_b=-1, wbuf_c=-1, wbuf_b=-1;
                    ^
src/test/test_tortls.c:1394:43: warning: unused variable 'wbuf_b'
      [-Wunused-variable]
  size_t rbuf_c=-1, rbuf_b=-1, wbuf_c=-1, wbuf_b=-1;
                                          ^
src/test/test_tortls.c:1394:10: warning: unused variable 'rbuf_c'
      [-Wunused-variable]
  size_t rbuf_c=-1, rbuf_b=-1, wbuf_c=-1, wbuf_b=-1;

tortls/shutdown: 
  FAIL src/test/test_tortls.c:1988: assert(ret OP_EQ TOR_TLS_DONE): -8 vs 0
  [shutdown FAILED]
tortls/renegotiate: OK
tortls/finish_handshake: OK
tortls/handshake: OK
tortls/write: OK
tortls/read: 
  FAIL src/test/test_tortls.c:2106: assert(ret OP_EQ TOR_TLS_CLOSE): -8 vs -3
  [read FAILED]
tortls/server_info_callback: OK
tortls/is_server: OK
tortls/assert_renegotiation_unblocked: Oct 06 04:19:17.673 [err] void tor_assertion_failed_(const char *, unsigned int, const char *, const char *): Bug: src/common/tortls.c:1761: tor_tls_assert_renegotiation_unblocked: Assertion 0 != (options & SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION) failed; aborting. (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug: Assertion 0 != (options & SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION) failed in tor_tls_assert_renegotiation_unblocked at src/common/tortls.c:1761. Stack trace: (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     0   test                                0x002b0e84 log_backtrace + 68 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     1   test                                0x002bf077 tor_assertion_failed_ + 183 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     2   test                                0x002db2ad tor_tls_assert_renegotiation_unblocked + 93 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     3   test                                0x00156076 test_tortls_assert_renegotiation_unblocked + 70 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     4   test                                0x0019f44a testcase_run_one + 442 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     5   test                                0x0019fb88 tinytest_main + 584 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     6   test                                0x0019ef23 main + 675 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
Oct 06 04:19:17.673 [err] Bug:     7   libdyld.dylib                       0x922416d9 start + 1 (on Tor 0.2.8.0-alpha-dev 2ac5e5f61742738a)
FAIL src/test/test (exit status: 134)

changed milestone to %Tor: 0.2.8.x-final

added 026-backport 027-backport component::core tor/tor milestone::Tor: 0.2.8.x-final priority::medium resolution::fixed status::closed type::defect version::tor unspecified labels

I'm using libressl @2.2.2 from MacPorts.

OpenSSL 1.0.2d produces a different set of warnings:

 CC       src/common/tortls.o
src/common/tortls.c:1114:35: warning: implicit declaration of function
      'TLS_method' is invalid in C99 [-Wimplicit-function-declaration]
  if (!(result->ctx = SSL_CTX_new(TLS_method())))
                                  ^
src/common/tortls.c:1114:35: warning: incompatible integer to pointer conversion
      passing 'int' to parameter of type 'const SSL_METHOD *' (aka 'const struct
      ssl_method_st *') [-Wint-conversion]
  if (!(result->ctx = SSL_CTX_new(TLS_method())))
                                  ^~~~~~~~~~~~
/opt/local/include/openssl/ssl.h:2131:40: note: passing argument to parameter
      'meth' here
SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth);
                                       ^
src/common/tortls.c:1388:21: warning: incompatible integer to pointer conversion
      initializing 'const SSL_METHOD *' (aka 'const struct ssl_method_st *')
      with an expression of type 'int' [-Wint-conversion]
  const SSL_METHOD *m = TLS_method();

And

  CC       src/common/src_common_libor_crypto_testing_a-tortls.o
src/common/tortls.c:1114:35: warning: implicit declaration of function
      'TLS_method' is invalid in C99 [-Wimplicit-function-declaration]
  if (!(result->ctx = SSL_CTX_new(TLS_method())))
                                  ^
src/common/tortls.c:1114:35: warning: incompatible integer to pointer conversion
      passing 'int' to parameter of type 'const SSL_METHOD *' (aka 'const struct
      ssl_method_st *') [-Wint-conversion]
  if (!(result->ctx = SSL_CTX_new(TLS_method())))
                                  ^~~~~~~~~~~~
/opt/local/include/openssl/ssl.h:2131:40: note: passing argument to parameter
      'meth' here
SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth);
                                       ^
src/common/tortls.c:1388:21: warning: incompatible integer to pointer conversion
      initializing 'const SSL_METHOD *' (aka 'const struct ssl_method_st *')
      with an expression of type 'int' [-Wint-conversion]
  const SSL_METHOD *m = TLS_method();
                    ^   ~~~~~~~~~~~~

I'm using clang:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

Apparently these warnings only appear on i386.

Things work for me with 1.0.2d from homebrew on a 64-bit arch. I wonder what's going on here.

I wonder where TLS_method is getting declared for me. I don't see it in /usr/local/opt/openssl/include/openssl/*

(Is this 0.2.8-only, or is 0.2.7 affected too?)

Trac:
Keywords: N/A deleted, 027-backport 026-backport added

Hang on. No version of OpenSSL has ever had TLS_method. That should be TLSv1_method, right? And now that we require a recent openssl, we can assume it's there, right?

(Those test failures are probably 0.2.8-only: those tests are new here.)

I'm using Mac OS X with LLVM, OpenSSL from their git master (67202973cf55eaac021706c183377b8040cf0c20), compiled for 64 bit architecture. When attempting to compile current Tor master (c9cb5516ab3380f266c7f5db68899cb4cff49202), I get the following errors:

  CC       src/common/tortls.o
In file included from src/common/tortls.c:75:
./src/common/tortls.h:139:15: error: static declaration of
      'SSL_SESSION_get_master_key' follows non-static declaration
STATIC size_t SSL_SESSION_get_master_key(SSL_SESSION *s, uint8_t *out,
              ^
/usr/local/ssl/include/openssl/ssl.h:1659:15: note: previous declaration is here
__owur size_t SSL_SESSION_get_master_key(const SSL_SESSION *ssl,
              ^
src/common/tortls.c:2139:49: warning: passing 'const X509 *' (aka
      'const struct x509_st *') to parameter of type 'X509 *'
      (aka 'struct x509_st *') discards qualifiers
      [-Wincompatible-pointer-types-discards-qualifiers]
  if (!(ASN1_TIME_print(bio, X509_get_notBefore(cert)))) {
                                                ^~~~
/usr/local/ssl/include/openssl/x509.h:694:38: note: passing argument to
      parameter 'x' here
ASN1_TIME * X509_get_notBefore(X509 *x);
                                     ^
src/common/tortls.c:2147:48: warning: passing 'const X509 *' (aka
      'const struct x509_st *') to parameter of type 'X509 *'
      (aka 'struct x509_st *') discards qualifiers
      [-Wincompatible-pointer-types-discards-qualifiers]
  if (!(ASN1_TIME_print(bio, X509_get_notAfter(cert)))) {
                                               ^~~~
/usr/local/ssl/include/openssl/x509.h:696:36: note: passing argument to
      parameter 'x' here
ASN1_TIME *X509_get_notAfter(X509 *x);
                                   ^
src/common/tortls.c:2309:40: warning: passing 'const X509 *' (aka
      'const struct x509_st *') to parameter of type 'X509 *'
      (aka 'struct x509_st *') discards qualifiers
      [-Wincompatible-pointer-types-discards-qualifiers]
  if (X509_cmp_time(X509_get_notBefore(cert), &t) > 0) {
                                       ^~~~
/usr/local/ssl/include/openssl/x509.h:694:38: note: passing argument to
      parameter 'x' here
ASN1_TIME * X509_get_notBefore(X509 *x);
                                     ^
src/common/tortls.c:2314:39: warning: passing 'const X509 *' (aka
      'const struct x509_st *') to parameter of type 'X509 *'
      (aka 'struct x509_st *') discards qualifiers
      [-Wincompatible-pointer-types-discards-qualifiers]
  if (X509_cmp_time(X509_get_notAfter(cert), &t) < 0) {
                                      ^~~~
/usr/local/ssl/include/openssl/x509.h:696:36: note: passing argument to
      parameter 'x' here
ASN1_TIME *X509_get_notAfter(X509 *x);
                                   ^
4 warnings and 1 error generated.
make[1]: *** [src/common/tortls.o] Error 1
make: *** [all] Error 2

The above error appears because patch for #17082 (moved) exposed SSL_SESSION_get_master_key() function in tortls.h header file. This function may or may not be provided by OpenSSL. In the latest OpenSSL version, it is.

Quick patch for this: https://github.com/rl1987/tor/commit/b216340d75403571b8031baf5f63f751584470d1

However, I am still getting the above warnings about X509 * pointers AND I am getting more errors further down the road:

  CC       src/test/src_test_test-test_tortls.o
src/test/test_tortls.c:55:40: error: invalid application of 'sizeof' to an
      incomplete type 'SSL_METHOD' (aka 'struct ssl_method_st')
  SSL_METHOD *method = tor_malloc_zero(sizeof(SSL_METHOD));
                                       ^     ~~~~~~~~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/ssl.h:322:16: note: forward declaration of
      'struct ssl_method_st'
typedef struct ssl_method_st SSL_METHOD;
               ^
src/test/test_tortls.c:56:34: error: invalid application of 'sizeof' to an
      incomplete type 'SSL_METHOD' (aka 'struct ssl_method_st')
  memcpy(method, TLSv1_method(), sizeof(SSL_METHOD));
                                 ^     ~~~~~~~~~~~~
/usr/include/secure/_string.h:65:38: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                     ^
/usr/local/ssl/include/openssl/ssl.h:322:16: note: forward declaration of
      'struct ssl_method_st'
typedef struct ssl_method_st SSL_METHOD;
               ^
src/test/test_tortls.c:137:9: error: incomplete definition of type
      'struct ssl_method_st'
  method->num_ciphers = fake_num_ciphers;
  ~~~~~~^
/usr/local/ssl/include/openssl/ssl.h:322:16: note: forward declaration of
      'struct ssl_method_st'
typedef struct ssl_method_st SSL_METHOD;
               ^
src/test/test_tortls.c:446:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->rwstate = SSL_READING;
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:454:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->rwstate = SSL_READING;
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:462:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->rwstate = 0;
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:463:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->shutdown = SSL_RECEIVED_SHUTDOWN;
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:464:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->s3->warn_alert =SSL_AD_CLOSE_NOTIFY;
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:511:32: error: invalid application of 'sizeof' to an
      incomplete type 'X509' (aka 'struct x509_st')
  cert->cert = tor_malloc_zero(sizeof(X509));
                               ^     ~~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/ossl_typ.h:154:16: note: forward declaration of
      'struct x509_st'
typedef struct x509_st X509;
               ^
src/test/test_tortls.c:559:25: error: invalid application of 'sizeof' to an
      incomplete type 'X509' (aka 'struct x509_st')
  one = tor_malloc_zero(sizeof(X509));
                        ^     ~~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/ossl_typ.h:154:16: note: forward declaration of
      'struct x509_st'
typedef struct x509_st X509;
               ^
src/test/test_tortls.c:560:6: error: incomplete definition of type
      'struct x509_st'
  one->references = 1;
  ~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:154:16: note: forward declaration of
      'struct x509_st'
typedef struct x509_st X509;
               ^
src/test/test_tortls.c:561:25: error: invalid application of 'sizeof' to an
      incomplete type 'X509' (aka 'struct x509_st')
  two = tor_malloc_zero(sizeof(X509));
                        ^     ~~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/ossl_typ.h:154:16: note: forward declaration of
      'struct x509_st'
typedef struct x509_st X509;
               ^
src/test/test_tortls.c:562:6: error: incomplete definition of type
      'struct x509_st'
  two->references = 1;
  ~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:154:16: note: forward declaration of
      'struct x509_st'
typedef struct x509_st X509;
               ^
src/test/test_tortls.c:567:30: error: invalid application of 'sizeof' to an
      incomplete type 'SSL' (aka 'struct ssl_st')
  tls->ssl = tor_malloc_zero(sizeof(SSL));
                             ^     ~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:568:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION));
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:568:39: error: invalid application of 'sizeof' to an
      incomplete type 'SSL_SESSION' (aka 'struct ssl_session_st')
  tls->ssl->session = tor_malloc_zero(sizeof(SSL_SESSION));
                                      ^     ~~~~~~~~~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/ssl.h:324:16: note: forward declaration of
      'struct ssl_session_st'
typedef struct ssl_session_st SSL_SESSION;
               ^
src/test/test_tortls.c:569:11: error: incomplete definition of type
      'struct ssl_st'
  tls->ssl->session->peer = one;
  ~~~~~~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:176:16: note: forward declaration of
      'struct ssl_st'
typedef struct ssl_st SSL;
               ^
src/test/test_tortls.c:577:6: error: incomplete definition of type
      'struct x509_st'
  one->cert_info = tor_malloc_zero(sizeof(X509_CINF));
  ~~~^
/usr/local/ssl/include/openssl/ossl_typ.h:154:16: note: forward declaration of
      'struct x509_st'
typedef struct x509_st X509;
               ^
src/test/test_tortls.c:577:36: error: invalid application of 'sizeof' to an
      incomplete type 'X509_CINF' (aka 'struct x509_cinf_st')
  one->cert_info = tor_malloc_zero(sizeof(X509_CINF));
                                   ^     ~~~~~~~~~~~
./src/common/util.h:125:49: note: expanded from macro 'tor_malloc_zero'
#define tor_malloc_zero(size)  tor_malloc_zero_(size DMALLOC_ARGS)
                                                ^
/usr/local/ssl/include/openssl/x509.h:169:16: note: forward declaration of
      'struct x509_cinf_st'
typedef struct x509_cinf_st X509_CINF;
               ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[1]: *** [src/test/src_test_test-test_tortls.o] Error 1
make: *** [all] Error 2

Merged rl1987/bug17237_hax as a good beginning.

1eb838b30361b0dcc1e2b82815be25391d5a15f1 fixes the x509* part of this in 0.2.7.

With 20ec030d9b6ff0e403e37d2161f3e53dfd6dd622 I took a brute-force approach and disabled all the tests that were making trouble when OpenSSL 1.1 is present. (See the OPENSSL_OPAQUE flag.) Probably many of them can be fixed to work with openssl 1.1, but that's not something I have time to do this morning, and it's probably best done one-at-a-time.

I'm closing this as "works now", but opened #17253 (moved) to track attempts to re-enable these tests.

Trac:
Status: new to closed
Resolution: N/A to fixed

closed

moved to tpo/core/tor#17237 (closed)