Opened 11 months ago

Last modified 8 months ago

#32718 needs_information defect

Crash: Consensus diff src/lib/memarea/memarea.c:147: memarea_chunk_free_unchecked

Reported by: teor Owned by: nickm
Priority: High Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.4.1.6
Severity: Normal Keywords: crash, tor-dir, openbsd
Cc: Actual Points:
Parent ID: Points: 1
Reviewer: Sponsor:

Description

Here's the Tor log from OpenBSD:

Tor[96521]: Could not apply consensus diff because an ed command was
missing a line number.
Tor[96521]: consdiff_gen_diff: Refusing to generate consensus diff
because the generated ed diff could not be tested to successfully
generate the target consensus.
Tor[96521]: tor_assertion_failed_: Bug: src/lib/memarea/memarea.c:147:
memarea_chunk_free_unchecked: Assertion sent_val == 0x90806622u failed;
aborting. (on Tor 0.4.1.6 )
Tor[96521]: Bug: Assertion sent_val == 0x90806622u failed in
memarea_chunk_free_unchecked at src/lib/memarea/memarea.c:147: . (Stack
trace not available) (on Tor 0.4.1.6 )

Here's the original email:
https://lists.torproject.org/pipermail/tor-relays/2019-December/017950.html

Child Tickets

Change History (9)

comment:1 Changed 11 months ago by random

thanks @teor for creating the ticket. I'm the one with the problem.
This morning tor crashed again, this time without any log messages. Atm I have
"Log notice syslog" set, what would be a good logging config to circle this problem? thanks!

comment:2 Changed 11 months ago by teor

Tor has an interrupt-level logging mode, where it writes error-level logs to stderr, and any open file logs.

Try adding Log notice /var/log/tor/log (or whatever the default tor log path is on your platform).

comment:3 Changed 10 months ago by ahf

Keywords: 043-must added; 043-should removed

comment:4 Changed 9 months ago by nickm

Priority: MediumHigh

Mark 043-must tickets as high priority

comment:5 Changed 9 months ago by nickm

Owner: set to nickm
Status: newaccepted

So these two lines are the bug:

Tor[96521]: Could not apply consensus diff because an ed command was
missing a line number.
Tor[96521]: consdiff_gen_diff: Refusing to generate consensus diff
because the generated ed diff could not be tested to successfully
generate the target consensus.

Together they mean that this is a relay, and it generated a diff from one consensus to another, but that the diff in question was invalid and couldn't be applied.

comment:6 Changed 9 months ago by nickm

I almost want to suspect some kind of hardware problem here because fuzz_diff.c already fuzzes for this kind of bug: it makes a consensus then gives an assertion failure if it can't be applied. I'm going to look harder at this code.

comment:7 Changed 9 months ago by nickm

So here are a couple of issues that might be confusing this: First, our fuzzing code is not usually built with memarea.c turned on, since arena allocators can sometimes suppress memory bugs. Second, our fuzzing code doesn't consider it an error if we can't generate a working diff, since that can be caused by bad inputs as well as bad code. More investigation is needed, though.

I still suspect data corruption somewhere along the line, but for now I'm going to try fuzzing with memareas turned on, and seeing what that does.

comment:8 Changed 8 months ago by nickm

I've been fuzzing all day and I can't find any way to trigger this. Maybe this is bsd-only? I'll see if I can find somebody to run libfuzzer on openbsd, if it works there.

comment:9 Changed 8 months ago by nickm

Keywords: 043-must BugSmashFund removed
Milestone: Tor: 0.4.3.x-finalTor: unspecified
Status: acceptedneeds_information
Note: See TracTickets for help on using tickets.