Opened 2 years ago

Closed 21 months ago

Last modified 18 months ago

#15776 closed defect (worksforme)

assert_circuit_ok: Assertion c == c2 failed; aborting.

Reported by: Spider.007 Owned by:
Priority: High Milestone: Tor: 0.2.8.x-final
Component: Core Tor/Tor Version: Tor: 0.2.5.12
Severity: Normal Keywords: 026-backport, 025-backport, crash, PostFreeze027, 027-backport, 2016-bug-retrospective
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Tor[22]: tor_assertion_failed_(): Bug: src/or/circuitlist.c:2115: assert_circuit_ok: Assertion c == c2 failed; aborting.
Tor[22]: Bug: Assertion c == c2 failed in assert_circuit_ok at src/or/circuitlist.c:2115. Stack trace:
Tor[22]: Bug: /usr/bin/tor(log_backtrace+0x41) [0x7f5f75f62bc1]
Tor[22]: Bug: /usr/bin/tor(tor_assertion_failed_+0x8c) [0x7f5f75f6ed4c]
Tor[22]: Bug: /usr/bin/tor(assert_circuit_ok+0x317) [0x7f5f75ef45c7]
Tor[22]: Bug: /usr/bin/tor(circuit_mark_for_close_+0x2e) [0x7f5f75ef4c7e]
Tor[22]: Bug: /usr/bin/tor(command_process_cell+0x499) [0x7f5f75f03019]
Tor[22]: Bug: /usr/bin/tor(channel_tls_handle_cell+0x20b) [0x7f5f75ee69db]
Tor[22]: Bug: /usr/bin/tor(+0xd1184) [0x7f5f75f24184]
Tor[22]: Bug: /usr/bin/tor(+0xc8175) [0x7f5f75f1b175]
Tor[22]: Bug: /usr/bin/tor(+0x31031) [0x7f5f75e84031]
Tor[22]: Bug: /usr/lib/libevent-2.0.so.5(event_base_loop+0x806) [0x7f5f754dbca6]
Tor[22]: Bug: /usr/bin/tor(do_main_loop+0x195) [0x7f5f75e85815]
Tor[22]: Bug: /usr/bin/tor(tor_main+0x16e5) [0x7f5f75e88625]
Tor[22]: Bug: /usr/lib/libc.so.6(libc_start_main+0xf0) [0x7f5f74834800]

Child Tickets

Change History (13)

comment:1 Changed 2 years ago by cypherpunks

Tor[22]: Bug: /usr/bin/tor(command_process_cell+0x499) [0x7f5f75f03019]

According to offset, more likely it was command_process_destroy_cell().
Could be wrong though.

Then, every process function call of circuit_get_by_circid_channel(cell->circ_id, chan), and assert_circuit_ok check fails after:

circuit_t *c2 = circuit_get_by_circid_channel_impl(c->n_circ_id,
                                                   c->n_chan, NULL);

Two way to trigger this assert:

circ->n_circ_id != cell->circ_id

or

circ->n_chan != chan

comment:2 in reply to:  1 Changed 2 years ago by cypherpunks

Replying to cypherpunks:

Tor[22]: Bug: /usr/bin/tor(command_process_cell+0x499) [0x7f5f75f03019]

According to offset, more likely it was command_process_destroy_cell().
Could be wrong though.

For "the destroy came from behind" .

comment:3 in reply to:  1 Changed 2 years ago by cypherpunks

Replying to cypherpunks:

Two way to trigger this assert:

circ->n_circ_id != cell->circ_id

or

circ->n_chan != chan

Ignore this part, it's wrong for destroy or relay cells.

comment:4 Changed 2 years ago by Spider.007

4 hours after restarting I got:

*** Error in `/usr/bin/tor': free(): invalid next size (fast): 0x00007f341e6d5d60 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x7198e)[0x7f3411f4198e]
/usr/lib/libc.so.6(+0x76dee)[0x7f3411f46dee]
/usr/lib/libc.so.6(+0x775cb)[0x7f3411f475cb]
/usr/bin/tor(circuitmux_detach_circuit+0x100)[0x7f34135b40b0]
/usr/bin/tor(circuit_mark_for_close_+0x229)[0x7f34135b0e79]
/usr/bin/tor(command_process_cell+0x499)[0x7f34135bf019]
/usr/bin/tor(channel_tls_handle_cell+0x20b)[0x7f34135a29db]
/usr/bin/tor(+0xd1184)[0x7f34135e0184]
/usr/bin/tor(+0xc8175)[0x7f34135d7175]
/usr/bin/tor(+0x31031)[0x7f3413540031]
/usr/lib/libevent-2.0.so.5(event_base_loop+0x806)[0x7f3412b97ca6]
/usr/bin/tor(do_main_loop+0x195)[0x7f3413541815]
/usr/bin/tor(tor_main+0x16e5)[0x7f3413544625]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f3411ef0800]
/usr/bin/tor(_start+0x29)[0x7f341353ddb9]
======= Memory map: ========

Not sure if this is related to the original assertion, could this somehow be related to http://www.openwall.com/lists/oss-security/2015/04/21/4 ?

comment:5 Changed 2 years ago by nickm

Keywords: 026-backport 025-backport crash added
Milestone: Tor: 0.2.7.x-final

Probably not; we shouldn't be calling nss_anything from that code.

comment:6 Changed 2 years ago by cypherpunks

Probably hardware problems, if not then wrong free for circuit_t somewhere in code (assert happens if memory was reused already, and 2nd failure is sign of double free)

comment:7 Changed 2 years ago by nickm

If this is repeatable, could you try running Tor under valgrind (see instructions in doc/HACKING) to find out if it produces any warnings?

comment:8 Changed 2 years ago by cypherpunks

/usr/bin/tor(circuitmux_detach_circuit+0x100)[0x7f34135b40b0]

According to offset, it was about to call of cmux->policy->free_circ_data()
Could be wrong though.

comment:9 Changed 2 years ago by nickm

Keywords: PostFreeze027 added

I'd merge patches for these for 0.2.7 if they come in on time. In some cases, that will require figuring out an as-yet-unsolved bugs.

comment:10 Changed 23 months ago by nickm

Keywords: 027-backport added
Milestone: Tor: 0.2.7.x-finalTor: 0.2.8.x-final

comment:11 Changed 21 months ago by nickm

Severity: Normal
Status: newneeds_information

We'll need insight to figure this one out.

comment:12 Changed 21 months ago by Spider.007

Resolution: worksforme
Status: needs_informationclosed

I no longer experience this issue, and am unable to reproduce it. If the backtrace doesn't provide enough information I think we should consider this fixed until.

comment:13 Changed 18 months ago by nickm

Keywords: 2016-bug-retrospective added

Mark more tickets for severe bug retrospective, based on Priority and date and hand-inspection.

Note: See TracTickets for help on using tickets.