Opened 2 months ago

Closed 4 weeks ago

#23159 closed defect (fixed)

Bug: Non-fatal assertion ei failed in launch_intro_point_circuits at src/or/hs_service.c:1784

Reported by: asn Owned by: dgoulet
Priority: Medium Milestone: Tor: 0.3.2.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-hs, prop224
Cc: Actual Points:
Parent ID: Points: 0.2
Reviewer: Sponsor:

Description (last modified by asn)

I had the following non-fatal assert trigger for me when I SIGTERMed my service in chutney:

Aug 08 10:34:25.000 [warn] Bug: Non-fatal assertion ei failed in launch_intro_point_circuits at src/or/hs_service.c:1784. Stack trace: (bon Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(log_backtrace+0x42) [0x2802a2] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(tor_bug_occurred_+0xb7) [0x299127] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(hs_service_run_scheduled_events+0x189d) [0x27339d] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(+0x4b198) [0x153198] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(+0x69e10) [0x171e10] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5(event_base_loop+0x7fc) [0x53613dc] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(do_main_loop+0x244) [0x156674] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(tor_main+0x1c35) [0x159ec5] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(main+0x19) [0x151de9] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x623db45] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:25.000 [warn] Bug:     ./src/or/tor(+0x49e39) [0x151e39] (on Tor 0.3.2.0-alpha-dev eafb26f30db8c809)
Aug 08 10:34:26.000 [notice] Catching signal TERM, exiting cleanly.

I have not reproed it since then.

Child Tickets

Change History (11)

comment:1 Changed 2 months ago by asn

Description: modified (diff)

comment:2 Changed 2 months ago by dgoulet

My intuition is that extend_info_from_node() returns NULL for the intro point node_t. We might want to not BUG() on that since it can happen.

Furthermore, we might also want to try to get the extend info object once we pick the intro point and if it fails, pick another one. It will assure us that even in 18 hours after being picked, we can safely have the extend information without being worried that it could fail.

comment:3 Changed 7 weeks ago by dgoulet

Parent ID: #20657

FYI, I'm not seeing that bug anymore...

comment:4 Changed 7 weeks ago by asn

I just got this again on my prop224_unity hidden service. Here is a hint from the info logs:

Sep 02 05:02:11.000 [info] hs_circ_launch_intro_point(): Launching a circuit to intro point $6D61AF550F3C13799FD43C5ACE7474EA7723C3FF~hansdahlss
on at 178.62.18.161 for service 4acth47i6kxnvkewtm6q7ib2s3ufpo5sqbsnzjpbi7utijcltosqemad.
Sep 02 05:02:11.000 [info] circuit_launch_by_extend_info(): Cannibalizing circ 'Tor4wohjipahmi' for purpose 14 (Hidden service: Establishing int
roduction point)
Sep 02 05:02:11.000 [info] circuit_send_intermediate_onion_skin(): Sending extend relay cell.
Sep 02 05:02:11.000 [warn] tor_bug_occurred_(): Bug: src/or/hs_service.c:1903: launch_intro_point_circuits: Non-fatal assertion ei failed. (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug: Non-fatal assertion ei failed in launch_intro_point_circuits at src/or/hs_service.c:1903. Stack trace: (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(log_backtrace+0x44) [0x55ee845d2ce4] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(tor_bug_occurred_+0xb9) [0x55ee845ebca9] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(hs_service_run_scheduled_events+0x1ab8) [0x55ee845c4fb8] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(+0x4e781) [0x55ee8449f781] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(+0x6d990) [0x55ee844be990] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5(event_base_loop+0x6a0) [0x7ff1cd4b15a0] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(do_main_loop+0x23d) [0x55ee844a2c1d] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(tor_main+0x1c25) [0x55ee844a6605] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(main+0x19) [0x55ee8449e459] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7ff1cc19a2e1] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)
Sep 02 05:02:11.000 [warn] Bug:     ./src/or/tor(_start+0x2a) [0x55ee8449e4aa] (on Tor 0.3.2.0-alpha-dev 63e26b216a0c463b)

comment:5 Changed 5 weeks ago by dgoulet

Owner: set to dgoulet
Status: newaccepted

comment:6 Changed 5 weeks ago by dgoulet

Status: acceptedneeds_information

This issue is a bit peculiar if it does still exists so we are currently running a test patch on the real network with some non fatal assert to catch the issue.

You can see the test code in bug23159_032_01

comment:7 Changed 4 weeks ago by nickm

Is it appropriate to merge that in 0.3.2 if it doesn't trigger on your testnet?

comment:8 Changed 4 weeks ago by dgoulet

We aren't running it on the testnet and it was actually triggered on the real network.

We've been running a custom patch with many more asserts() to catch this and since 12 days ago, still nothing. I'm optimistic.

comment:9 Changed 4 weeks ago by dgoulet

Status: needs_informationaccepted

All in all, even though we haven't hit it yet, having a NULL extend info object from a node_t is possible so we should handle that gracefully.

comment:10 Changed 4 weeks ago by dgoulet

Status: acceptedneeds_review

See branch: bug23159_032_02.

comment:11 Changed 4 weeks ago by nickm

Resolution: fixed
Status: needs_reviewclosed

Sure; merging. But next time please make sure "make check" passes?

Note: See TracTickets for help on using tickets.