Opened 5 weeks ago

Closed 3 weeks ago

#23551 closed defect (fixed)

src/common/compress.c:576: tor_compress_process: Non-fatal assertion

Reported by: cypherpunks Owned by: ahf
Priority: High Milestone: Tor: 0.3.1.x-final
Component: Core Tor/Tor Version: Tor: 0.3.1.5-alpha
Severity: Normal Keywords: 0.3.2.2-alpha-must
Cc: vvort@… Actual Points:
Parent ID: Points:
Reviewer: isis Sponsor:

Description

duplicate of #22719 ?

OS: HardenedBSD

Bootstrapped 100%: Done
Performing bandwidth self-test...done.
tor_bug_occurred_: Bug: src/common/compress.c:576: tor_compress_process: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed. (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed in tor_compress_process at src/common/compress.c:576. Stack trace: (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103e31d87 <log_backtrace+0x37> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103e488a1 <tor_bug_occurred_+0x131> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
\Bug:     0xe103e4dc7b <tor_compress_process+0xcb> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103e4d32c <tor_compress+0x21c> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103e4d7e8 <tor_uncompress+0x28> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103df4c88 <connection_dir_reached_eof+0x818> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103df448d <connection_dir_reached_eof+0x1d> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103dce0b8 <connection_handle_read+0xbb8> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103d19dd1 <connection_add_impl+0x1f1> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0x36578fda06f <event_base_assert_ok_nolock_+0xc9f> at /usr/local/lib/libevent-2.1.so.6 (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0x36578fd5f4e <event_base_loop+0x50e> at /usr/local/lib/libevent-2.1.so.6 (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103d1b834 <do_main_loop+0x594> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103d1dcdb <tor_main+0xcb> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103d19b99 <main+0x9> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)
Bug:     0xe103d089f0 <_start+0x180> at /usr/local/bin/tor (on Tor 0.3.1.5-alpha 83389502ee631465)

Child Tickets

Change History (16)

comment:1 Changed 5 weeks ago by nickm

Milestone: Tor: 0.3.1.x-final
Priority: MediumHigh

comment:2 Changed 5 weeks ago by nickm

It sure does look similar to #22719, though we thought we fixed that one 0.3.1.4-alpha

comment:3 Changed 5 weeks ago by Vort

Cc: vvort@… added

Same thing for 0.3.1.7 (custom build) and Windows 7:

Sep 19 12:33:13.743 [notice] Tor 0.3.1.7 running on Windows 7 with Libevent 2.0.22-stable, OpenSSL 1.0.2l, Zlib 1.2.11, Liblzma 5.2.3, and Libzstd 1.3.1.
...
Sep 19 23:36:30.000 [warn] tor_bug_occurred_(): Bug: compress.c:576: tor_compress_process: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed. (on Tor 0.3.1.7 )
Sep 19 23:36:30.000 [warn] Bug: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed in tor_compress_process at compress.c:576. (Stack trace not available) (on Tor 0.3.1.7 )

comment:4 Changed 4 weeks ago by Felixix

Tor 0.3.2.1-alpha (git-290274dbb5428bc5) running on FreeBSD with Libevent 2.1.8-stable, OpenSSL LibreSSL 2.5.4, Zlib 1.2.8, Liblzma 5.2.2, and Libzstd 1.3.1.
More info on the bug: method == Zstandard compressed, finish == 0,  *in_len == in_len_orig == 1136, *out_len == out_len_orig == 0

comment:5 Changed 4 weeks ago by nickm

Ohhhh. out_len_orig == 0 seems like the problem there...

comment:6 Changed 4 weeks ago by teor

I suggest we split that BUG() expression into its separate components to aid in diagnosing future issues.

Edit: oh, hang on, that won't work. Just ignore me.

Last edited 4 weeks ago by teor (previous) (diff)

comment:7 Changed 4 weeks ago by teor

It wouldn't surprise me if we have a zstd version incompatibility here. Or maybe truncated input data?

Truncated input data would definitely explain why zstd can't make any progress, but isn't returning an error code. In that case, it's our fault for not giving zstd enough data to work with. (And not failing the loop quietly?)

Because if it were corrupt data, it really should return an error:
http://facebook.github.io/zstd/zstd_manual.html#Chapter9

comment:8 Changed 4 weeks ago by nickm

I think that the reason it's making no progress is because out_len_orig was 0 -- there was no space in the output buffer when it started.

comment:9 Changed 4 weeks ago by ahf

Owner: set to ahf
Status: newassigned

comment:10 Changed 4 weeks ago by cypherpunks

this is hitting all my Debian 9 based relays:

Tor 0.3.1.7 (git-5fa14939bca67c23) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.1.0f, Zlib 1.2.8, Liblzma 5.2.2, and Libzstd 1.1.2.
 Bug: ../src/common/compress.c:576: tor_compress_process: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed. (on Tor 0.3.1.7 )
 Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed in tor_compress_process at ../src/common/compress.c:576. Stack trace: (on Tor 0.3.1.7 )
     /usr/bin/tor(log_backtrace+0x44) [0x557fdd02e194] (on Tor 0.3.1.7 )
     /usr/bin/tor(tor_bug_occurred_+0xb9) [0x557fdd047029] (on Tor 0.3.1.7 )
     /usr/bin/tor(tor_compress_process+0x135) [0x557fdd04ffa5] (on Tor 0.3.1.7 )
     /usr/bin/tor(+0x18e171) [0x557fdd050171] (on Tor 0.3.1.7 )
     /usr/bin/tor(tor_uncompress+0x31) [0x557fdd050631] (on Tor 0.3.1.7 )
     /usr/bin/tor(connection_dir_reached_eof+0x118c) [0x557fdcff266c] (on Tor 0.3.1.7 )
     /usr/bin/tor(+0x1089bc) [0x557fdcfca9bc] (on Tor 0.3.1.7 )
     /usr/bin/tor(+0x4d85e) [0x557fdcf0f85e] (on Tor 0.3.1.7 )
     /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5(event_base_loop+0x6a0) [0x7f2bebf2a5a0] (on Tor 0.3.1.7 )
     /usr/bin/tor(do_main_loop+0x29d) [0x557fdcf1098d] (on Tor 0.3.1.7 )
     /usr/bin/tor(tor_main+0x1c35) [0x557fdcf144d5] (on Tor 0.3.1.7 )
     /usr/bin/tor(main+0x19) [0x557fdcf0c189] (on Tor 0.3.1.7 )
     /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f2bea98c2b1] (on Tor 0.3.1.7 )
     /usr/bin/tor(_start+0x2a) [0x557fdcf0c1da] (on Tor 0.3.1.7 )

It does NOT affect my Debian 8 relays since they have no zstd:

Tor 0.3.1.7 (git-5fa14939bca67c23) running on Linux ..., and Libzstd **N/A.**

comment:11 Changed 4 weeks ago by nickm

Keywords: 0.3.2.2-alpha-must added

comment:12 Changed 3 weeks ago by isis

Reviewer: isis

Assigning myself as a reviewer since we want this is 0.3.2.2-alpha. I know very little to nothing of libzstd, so if someone else would also like to review that would be helpful. (Maybe teor?)

comment:13 Changed 3 weeks ago by ahf

Status: assignedneeds_review

I have a proposed fix for this issue. It can be found in the following branches:

comment:14 in reply to:  13 Changed 3 weeks ago by isis

Status: needs_reviewmerge_ready

Replying to ahf:

I have a proposed fix for this issue. It can be found in the following branches:


  • c2fac2c6 LGTM (yay test driven dev)
  • 44dc4b73 LGTM
  • c3b7f9d7 LGTM
  • a196fdb6 LGTM

FWIW this didn't require any libzstd knowledge to review, so I feel comfortable that my assessment is correct. If someone else still wants to also review, though, please feel free.

comment:15 Changed 3 weeks ago by nickm

Okay. Merged to maint-0.3.1 and master.

comment:16 Changed 3 weeks ago by nickm

Resolution: fixed
Status: merge_readyclosed
Note: See TracTickets for help on using tickets.