Opened 6 years ago

Closed 5 months ago

#6767 closed defect (worksforme)

tor crashes with Assertion smartlist_get(rl->old_routers, idx) == sd failed

Reported by: darkk Owned by: nickm
Priority: Medium Milestone: Tor: 0.3.3.x-final
Component: Core Tor/Tor Version: Tor: 0.2.2.37
Severity: Major Keywords: tor-relay, crash, assertion, 029-backport, 031-backport, needs-insight, 033-triage-20180320, 033-included-20180320
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I started my gateway with stderr redirected to syslog and got following crash:

Sep  2 12:30:07 OpenWrt cron.info crond[708]: crond: USER root pid 1794 cmd /root/tor-watchdog.sh
Sep  2 12:43:19 OpenWrt daemon.err Tor[1280]: routerlist_remove_old(): Bug: routerlist.c:3004: routerlist_remove_old: Assertion idx == sd->routerlist_index failed; aborting.
Sep  2 12:43:21 OpenWrt user.notice tor.stdout: routerlist.c:3004 routerlist_remove_old: Assertion idx == sd->routerlist_index failed; aborting.
Sep  2 12:45:01 OpenWrt cron.info crond[708]: crond: USER root pid 1797 cmd /root/tor-watchdog.sh
Sep  2 12:45:01 OpenWrt user.notice Tor.watchdog: pidfile exists and process is missing - start
Sep  2 12:45:02 OpenWrt user.notice Tor.init: Sep 02 12:45:02.684 [notice] Tor v0.2.2.37 (git-fce6eb1c44e87bc2). This is experimental software. Do not rely on it for strong anonymity. (Running on Linux mips)

OpenWrt package versions: libopenssl - 1.0.1c-1, tor - 0.2.2.37-1

openssl build information:

root@OpenWrt:~# strings /usr/lib/libcrypto.so.1.0.0 | grep gcc
libgcc_s.so.1
mipsel-openwrt-linux-uclibc-gcc -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DDSO_DLFCN -DHAVE_DLFCN_H -I/home/fnord/slave/brcm47xx/build/staging_dir/target-mipsel_uClibc-0.9.33.2/usr/include -I/home/fnord/slave/brcm47xx/build/staging_dir/target-mipsel_uClibc-0.9.33.2/include -I/home/fnord/slave/brcm47xx/build/staging_dir/toolchain-mipsel_gcc-4.6-linaro_uClibc-0.9.33.2/usr/include -I/home/fnord/slave/brcm47xx/build/staging_dir/toolchain-mipsel_gcc-4.6-linaro_uClibc-0.9.33.2/include -DOPENSSL_SMALL_FOOTPRINT -DHAVE_CRYPTODEV -DOPENSSL_NO_ERR -DTERMIO -Os -pipe -mips32 -mtune=mips32 -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -msoft-float -fpic -DOPENSSL_THREADS -pthread -D_REENTRANT -D_THREAD_SAFE -D_THREADSAFE -fomit-frame-pointer -Wall

Child Tickets

Change History (30)

comment:1 Changed 6 years ago by rransom

Component: - Select a componentTor Client

See also #6697 (and #1194).

comment:2 Changed 6 years ago by darkk

FYI: I don't use tor as a client, only as relay (if that matters).
https://atlas.torproject.org/#details/F44DA505AD91CFC8D5745BB070909F20F21E06D9

comment:3 Changed 6 years ago by nickm

Component: Tor ClientTor Relay
Milestone: Tor: 0.2.2.x-final
Priority: normalmajor

comment:4 Changed 6 years ago by nickm

Milestone: Tor: 0.2.2.x-finalTor: 0.2.4.x-final

comment:5 Changed 6 years ago by nickm

Keywords: tor-relay added

comment:6 Changed 6 years ago by nickm

Component: Tor RelayTor

comment:7 Changed 6 years ago by nickm

Status: newneeds_information

Did checking out rransom's advice about #1194 help at all? This could be a compiler bug. If not, a backtrace from gdb would be extremely helpful.

comment:8 Changed 5 years ago by darkk

I'll try to upgrade that router to latest OpenWRT and tor versions and will update the issue when I have more information.

comment:9 Changed 5 years ago by nickm

Milestone: Tor: 0.2.4.x-finalTor: unspecified

Did anything come of this?

comment:10 Changed 5 years ago by Tritonio

My bridge crashed with this message:

Oct xx xx:xx:xx.xxx [Error] routerlist_remove_old(): Bug: routerlist.c:3146: routerlist_remove_old: Assertion smartlist_get(rl->old_routers, idx) == sd failed; aborting.

I suppose it's related to this bug? Or should I create a new bug report?

No idea how to reproduce or why it happened. I just noticed it after a few hours.

comment:11 Changed 5 years ago by nickm

What version of Tor were you running?

comment:12 Changed 5 years ago by Tritonio

Sorry, I forgot to mention that: 0.2.4.17-rc on windows.

comment:13 Changed 5 years ago by darkk

Did anything come of this?

No. That router died. :-(

comment:14 Changed 4 years ago by nickm

See also #12068

comment:15 Changed 4 years ago by nickm

Milestone: Tor: unspecifiedTor: 0.2.???

Hm. I wonder if there is any common element among the configurations that have run into this bug. It's been reported with two 0.2.4.x versions, on openwrt, Windows, and some unknown Linux (#12068). It's only been reported on relays and bridges so far.

Are there any unusual options set in your torrc?

If this persists in 0.2.5, we should get a stack trace for when it happens. Getting any stack trace for this bug would probably help.

comment:16 Changed 3 years ago by Ch0nG

I just had this occur on a relay under Raspbian, 0.2.4.27 and DD-WRT.

comment:17 in reply to:  15 Changed 2 years ago by cypherpunks

Severity: Normal

Replying to nickm:

If this persists in 0.2.5, we should get a stack trace for when it happens. Getting any stack trace for this bug would probably help.

Stack trace reported by someone to IRC:

Sep 07 12:25:31.000 [err] tor_assertion_failed_(): Bug: ../src/or/routerlist.c:3293: routerlist_remove_old: Assertion smartlist_get(rl->old_routers, idx) == sd failed; aborting. (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug: Assertion smartlist_get(rl->old_routers, idx) == sd failed in routerlist_remove_old at ../src/or/routerlist.c:3293. Stack trace: (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(log_backtrace+0x33) [0x54c3b8d0] (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(tor_assertion_failed_+0x61) [0x54c467e6] (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(+0x63382) [0x54bb6382] (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(update_consensus_router_descriptor_downloads+0x2d1) [0x54bba5a2] (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(update_all_descriptor_downloads+0x2d) [0x54bba7aa] (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(+0x2c316) [0x54b7f316] (on Tor 0.2.8.6 )
Sep 07 12:25:31.000 [err] Bug:     /usr/bin/tor(+0x3ed4a) [0x54b91d4a] (on Tor 0.2.8.6 )

comment:18 Changed 2 years ago by cypherpunks

Summary: tor crashes with Assertion `idx == sd->routerlist_index`tor crashes with Assertion smartlist_get(rl->old_routers, idx) == sd failed

comment:19 Changed 21 months ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:20 Changed 20 months ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:21 Changed 15 months ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:22 Changed 15 months ago by nickm

Keywords: crash assertion 029-backport 030-backport 031-backport needs-insight added
Milestone: Tor: unspecifiedTor: 0.3.2.x-final
Severity: NormalMajor
Status: needs_informationnew

comment:23 Changed 9 months ago by nickm

Owner: set to nickm
Status: newaccepted

comment:24 Changed 9 months ago by nickm

Milestone: Tor: 0.3.2.x-finalTor: 0.3.3.x-final

comment:25 Changed 6 months ago by nickm

Keywords: 030-backport removed

Remove 030-backport from all open tickets that have it: 0.3.0 is now deprecated.

comment:26 Changed 6 months ago by nickm

Keywords: 033-maybe-must added

Mark some tickets as possibly belonging in 033-must.

comment:27 Changed 6 months ago by nickm

Keywords: 033-must added; 033-maybe-must removed

move 033-maybe-must into 033-must

comment:28 Changed 5 months ago by nickm

Keywords: 033-triage-20180320 added

Marking all tickets reached by current round of 033 triage.

comment:29 Changed 5 months ago by nickm

Keywords: 033-included-20180320 added

Mark 033-must tickets as triaged-in for 0.3.3

comment:30 Changed 5 months ago by nickm

Keywords: 033-must removed
Priority: HighMedium
Resolution: worksforme
Status: acceptedclosed

Hm. We have no reports of this since 0.2.8.x, and looking through the code, I'm not sure that it can. I don't think this belongs in 0.3.3.x. Please reopen if this is actually happening with any supported version.

Note: See TracTickets for help on using tickets.