Opened 8 months ago

Last modified 3 weeks ago

#25245 needs_revision defect

Crash in assert_connection_ok when changing Exit options

Reported by: toralf Owned by: ahf
Priority: Very High Milestone: Tor: 0.3.5.x-final
Component: Core Tor/Tor Version: Tor: 0.3.3.2-alpha
Severity: Normal Keywords: crash, regression?, tor-exit, tor-relay, ipv6, 033-must, 033-triage-20180320, 033-included-20180320, 035-must, 035-triaged-in-20180711
Cc: Actual Points:
Parent ID: Points: 1
Reviewer: Sponsor:

Description

Got this today:

Feb 13 22:02:49.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5113: assert_connection_ok: Assertion (conn->type == CONN_TYPE_EXIT && conn->state == EXIT_CONN_STATE_RESOLVING) || connection_is_writing(conn) || conn->write_blocked_on_bw || (CONN_IS_EDGE(conn) && TO_EDGE_CONN(conn)->edge_blocked_on_circ) failed; aborting. (on Tor 0.3.3.2-alpha efc105716283bbdf)

Before I changed the config from

ExitRelay 1
IPv6Exit 1

to

#ExitRelay 1
#IPv6Exit 1

and a little bit later to

ExitRelay 0
IPv6Exit 0

and run a kill -HUP (after every config change)

Child Tickets

Change History (26)

comment:1 Changed 8 months ago by teor

Keywords: crash regression? tor-exit ipv6 added
Milestone: Tor: 0.3.3.x-final
Points: 1
Priority: MediumVery High

comment:2 Changed 8 months ago by teor

Keywords: tor-relay added
Summary: [err] tor_assertion_failed_(): Bug: src/or/connection.c:5113Crash in assert_connection_ok when changing Exit options

comment:4 Changed 8 months ago by rl1987

I tried untangling the assert statement for better debuggability:

Please try reproducing the crash bug with this Tor compiled from this branch. A stack trace would be nice as well. I wasn't able to make it crash on my local machine as it seems to be related to actual traffic being handled by your relay.

When exit node feature is disabled by force-reloading torrc, what exactly happens to pending exit connections? Do we cancel them properly?

Last edited 8 months ago by rl1987 (previous) (diff)

comment:5 Changed 8 months ago by rl1987

Status: newneeds_information

comment:6 Changed 8 months ago by toralf

Applied that patch and got immediately a crash of both of my Tor exit relays :

Mar 11 16:27:41.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5114: assert_connection_ok: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed; aborting. (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed in assert_connection_ok at src/or/connection.c:5114. Stack trace: (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x53) [0x564a1d49f513] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x99) [0x564a1d4bc559] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(assert_connection_ok+0x86b) [0x564a1d419e8b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(+0x152b4b) [0x564a1d45fb4b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x3329d) [0x7fd983b4329d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x244d0) [0x7fd983b344d0] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(event_base_loop+0x53f) [0x7fd983b3548f] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(do_main_loop+0x26c) [0x564a1d3616dc] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(tor_run_main+0x27d) [0x564a1d362d5d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(tor_main+0x49) [0x564a1d35bd79] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(main+0x28) [0x564a1d35baa8] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fd9826a75ad] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:27:41.000 [err] Bug:     /usr/bin/tor(_start+0x2a) [0x564a1d35bafa] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)

and

ar 11 16:28:07.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5114: assert_connection_ok: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed; aborting. (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed in assert_connection_ok at src/or/connection.c:5114. Stack trace: (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x53) [0x564fb8033513] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x99) [0x564fb8050559] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(assert_connection_ok+0x86b) [0x564fb7fade8b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(+0x152b4b) [0x564fb7ff3b4b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x3329d) [0x7f6ad207d29d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x244d0) [0x7f6ad206e4d0] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(event_base_loop+0x53f) [0x7f6ad206f48f] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(do_main_loop+0x26c) [0x564fb7ef56dc] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(tor_run_main+0x27d) [0x564fb7ef6d5d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(tor_main+0x49) [0x564fb7eefd79] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(main+0x28) [0x564fb7eefaa8] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f6ad0be15ad] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 16:28:07.000 [err] Bug:     /usr/bin/tor(_start+0x2a) [0x564fb7eefafa] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Last edited 8 months ago by toralf (previous) (diff)

comment:7 Changed 8 months ago by teor

Status: needs_informationnew

comment:8 Changed 8 months ago by rl1987

Okay, now line numbers do not match. Which one of the three new assert statements is at line 5114?

Alternatively, which git commit were you at before applying this?

comment:9 Changed 8 months ago by toralf

Well, I use the Gentoo package which has tor.0.3.3.3-alpha packed.
On that I do apply "git diff tor-0.3.3.3-alpha..tor-0.3.4.0-alpha-dev", the exact command is

torproject@mr-fox ~ $ cd ~/tor; git diff tor-0.3.3.3-alpha..tor-0.3.4.0-alpha-dev -- . ':!doc' ':!contrib' ':!changes' ':!ChangeLog' ':!Release*' ':!scripts' ':!.gitignore' ':!.travis.yml' ':!src/test/Makefile.nmake' ':!.gitlab-ci.yml' ':!.gitmodules' > /etc/portage/patches/net-vpn/tor/master.patch

and because your given patch starts with ebd1fda4b16bb169718 the Gentoo logic applies first your ebd* patch and then my patch called "master.patch".

I changed the name to "A-master.patch" so that your patch is applied on top of the upgrade from 0.3.3.3 to 0.3.4 and will test again.

Last edited 8 months ago by toralf (previous) (diff)

comment:10 Changed 8 months ago by toralf

And with the opposite order in applying first the 0.3.3.3->0.3.4 and then your patch I get this picture :

Mar 11 21:03:15.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5114: assert_connection_ok: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed; aborting. (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed in assert_connection_ok at src/or/connection.c:5114. Stack trace: (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x53) [0x55c4d18a3513] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x99) [0x55c4d18c0559] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(assert_connection_ok+0x86b) [0x55c4d181de8b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(+0x152b4b) [0x55c4d1863b4b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x3329d) [0x7f5327c1829d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x244d0) [0x7f5327c094d0] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(event_base_loop+0x53f) [0x7f5327c0a48f] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(do_main_loop+0x26c) [0x55c4d17656dc] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(tor_run_main+0x27d) [0x55c4d1766d5d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(tor_main+0x49) [0x55c4d175fd79] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(main+0x28) [0x55c4d175faa8] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f532677c5ad] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:03:15.000 [err] Bug:     /usr/bin/tor(_start+0x2a) [0x55c4d175fafa] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)

and

Mar 11 21:02:24.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5114: assert_connection_ok: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed; aborting. (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug: Assertion connection_is_writing(conn) || conn->write_blocked_on_bw failed in assert_connection_ok at src/or/connection.c:5114. Stack trace: (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x53) [0x559eb6f5e513] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x99) [0x559eb6f7b559] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(assert_connection_ok+0x86b) [0x559eb6ed8e8b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(+0x152b4b) [0x559eb6f1eb4b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x3329d) [0x7ff26e1a729d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x244d0) [0x7ff26e1984d0] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(event_base_loop+0x53f) [0x7ff26e19948f] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(do_main_loop+0x26c) [0x559eb6e206dc] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(tor_run_main+0x27d) [0x559eb6e21d5d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(tor_main+0x49) [0x559eb6e1ad79] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(main+0x28) [0x559eb6e1aaa8] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xfd) [0x7ff26cd0b5ad] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:02:24.000 [err] Bug:     /usr/bin/tor(_start+0x2a) [0x559eb6e1aafa] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)

comment:12 Changed 8 months ago by toralf

applying that patch too gives:

Mar 11 21:45:51.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5114: assert_connection_ok: Assertion 1 == 1 && (connection_is_writing(conn) || conn->write_blocked_on_bw) failed; aborting. (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug: Assertion 1 == 1 && (connection_is_writing(conn) || conn->write_blocked_on_bw) failed in assert_connection_ok at src/or/connection.c:5114. Stack trace: (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x53) [0x55d5ad38e513] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x99) [0x55d5ad3ab559] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(assert_connection_ok+0x86b) [0x55d5ad308e8b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(+0x152b4b) [0x55d5ad34eb4b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x3329d) [0x7f4f6a4be29d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x244d0) [0x7f4f6a4af4d0] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(event_base_loop+0x53f) [0x7f4f6a4b048f] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(do_main_loop+0x26c) [0x55d5ad2506dc] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(tor_run_main+0x27d) [0x55d5ad251d5d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(tor_main+0x49) [0x55d5ad24ad79] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(main+0x28) [0x55d5ad24aaa8] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f4f690225ad] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:45:51.000 [err] Bug:     /usr/bin/tor(_start+0x2a) [0x55d5ad24aafa] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)

and

Mar 11 21:46:26.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5114: assert_connection_ok: Assertion 1 == 1 && (connection_is_writing(conn) || conn->write_blocked_on_bw) failed; aborting. (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug: Assertion 1 == 1 && (connection_is_writing(conn) || conn->write_blocked_on_bw) failed in assert_connection_ok at src/or/connection.c:5114. Stack trace: (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x53) [0x56077cb26513] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x99) [0x56077cb43559] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(assert_connection_ok+0x86b) [0x56077caa0e8b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(+0x152b4b) [0x56077cae6b4b] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x3329d) [0x7f93f338f29d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(+0x244d0) [0x7f93f33804d0] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/lib64/libevent-2.1.so.6(event_base_loop+0x53f) [0x7f93f338148f] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(do_main_loop+0x26c) [0x56077c9e86dc] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(tor_run_main+0x27d) [0x56077c9e9d5d] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(tor_main+0x49) [0x56077c9e2d79] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(main+0x28) [0x56077c9e2aa8] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f93f1ef35ad] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)
Mar 11 21:46:26.000 [err] Bug:     /usr/bin/tor(_start+0x2a) [0x56077c9e2afa] (on Tor 0.3.4.0-alpha-dev d15a7a27bb9e49bd)

comment:13 Changed 7 months ago by rl1987

I figure this happens because we do not properly cleanup/close existing edge connections when ExitRelay is switched off. A solution would be to keep track of child connections that listeners spawn when accept() succeeds and close all of them when closing the listener.

Something like:

Does this make sense?

If this is correct approach I could finalise the patch into mergeable shape.

comment:14 Changed 7 months ago by nickm

Hm. We don't actually want to close all of the "Foo" connections when the "FooListener" is closed. If we did that, then our hibernation code would break, since it assumes that you we continue servicing existing connections after we've closed all the listeners.

I think instead the problem here is that there is something wrong with the tor_assert(). Can you help me understand how the problem here relates to the fix? That is, why does setting ExitRelay 0 make these connections trigger the assertion if they do not get closed?

comment:15 Changed 7 months ago by nickm

Keywords: 033-must added

comment:16 Changed 7 months ago by nickm

Keywords: 033-triage-20180320 added

Marking all tickets reached by current round of 033 triage.

comment:17 Changed 7 months ago by nickm

Keywords: 033-included-20180320 added

Mark 033-must tickets as triaged-in for 0.3.3

comment:18 in reply to:  14 Changed 7 months ago by rl1987

Let's ponder some more about this. Look at the above stack traces. Libevent calls back into tor, then the called function calls assert_connection_ok(), which explodes.

The function in question is conn_write_callback() in main.c. It calls connection_handle_write() which seems to succeed (otherwise it would attempt to close and null out outbuf_flushlen for connection in question).

However, right after that assert_connection_ok() fails. Do we have a race condition here? Is there one more exception case when edge connection cannot be writable despite outbuf_flushlen not being zero? Does libevent de-schedule EV_WRITE for already accepted connections when listener is closed?

comment:19 in reply to:  4 Changed 7 months ago by arma

Replying to rl1987:

I tried untangling the assert statement for better debuggability:

I think this patch is not right. That is, the new more debuggable version is a different set of asserts than the original.

comment:20 Changed 7 months ago by rl1987

I believe the following is logically equivalent (and logs more debug info):

This one checks if we have edge connection first and if so, checks if it is exit connection.

comment:21 Changed 7 months ago by ahf

Owner: set to ahf
Status: newassigned

comment:22 Changed 5 months ago by teor

Milestone: Tor: 0.3.3.x-finalTor: 0.3.5.x-final
Status: assignedneeds_revision

We can fix this in 0.3.5.

comment:23 Changed 4 months ago by rl1987

Idea: maybe we could use this to reproduce the crashbug: https://github.com/antitree/private-tor-network

comment:24 Changed 4 months ago by nickm

Keywords: 035-must added

comment:25 Changed 3 months ago by nickm

Keywords: 035-triaged-in-20180711 added

comment:26 Changed 3 weeks ago by nickm

Mark all 035-must tickets as "very high"

Note: See TracTickets for help on using tickets.