Opened 3 years ago

Closed 2 years ago

#19166 closed defect (user disappeared)

HS connections randomly fail

Reported by: TvdW Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.2.7.6
Severity: Normal Keywords: tor-hs
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor: SponsorR-can

Description

For the sake of checking whether my bridges are running properly, a regular nagios job tries to establish a tcp connection to a HS running on each bridge, every 5 minutes. This is done through 'torsocks check_ssh -H abcdef.onion -p 22' but the issue in this ticket is reproducible with 'torsocks nc abcdef.onion 22' as well.

Connections randomly fail, causing a lot of monitoring noise, though I bet that for other users this manifests as hosts being unavailable completely.

As a test, I restarted tor and waited for a nagios check to fail. I then stopped nagios completely, causing tor to be idle, and manually ran a single check. The tor debug log, trimmed to the rough timeframe in which I ran the check, can be found at https://tvdw.eu/tor-debug-20160524220000.log

torsocks says: [May 24 22:00:33] ERROR torsocks[28974]: General SOCKS server failure (in socks5_recv_connect_reply() at socks5.c:516)
(note the timestamp, this failure is visible in tor's logs)

In my test the client ran Tor 0.2.7.6, and the HS ran 0.2.8.2, though I have been able to reproduce this with various tor versions on the HS side.

Child Tickets

Change History (10)

comment:1 Changed 3 years ago by TvdW

Potentially relevant:

  • restarting the tor daemon on the client consistently fixes this issue
  • it takes a few minutes after restarting for the issue to appear again, not hours
  • occasionally these connections will fail many (10+) times in a row
  • some 60 hosts are monitored, which results in a nice 720 checks per hour. Every hour several of these fail, sometimes up to 40

comment:2 Changed 3 years ago by arma

btw, when i make debug logs, I set

LogTimeGranularity 1

so it's easier to tell which debug statements are grouped with each other.

comment:3 Changed 3 years ago by arma

Can you reproduce this bug on quiet hidden services and quiet clients?

Or do they need to be active?

comment:4 Changed 3 years ago by arma

Milestone: Tor: 0.2.???
Sponsor: SponsorR-can

Ok. The summary of the client-side log appears to be that it receives the socks request, finds an already-established rendezvous circuit, sends the begin cell, and I didn't see for sure if it got any answer because right about then it decided to fetch a new consensus and once it got the consensus it decided to fetch the new microdescriptors it learned about.

If you would set up boring clients and onion services that don't do anything else, and set safelogging 0 and LogTimeGranularity 1 on both sides, we'll be in a better position to debug next time.

Thanks!

comment:5 Changed 3 years ago by arma

tvdw: any news?

See also #3733 for a ticket that looks related.

comment:6 Changed 3 years ago by dgoulet

Keywords: tor-hs added

comment:7 Changed 3 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:8 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:9 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:10 Changed 2 years ago by dgoulet

Resolution: user disappeared
Status: newclosed
Note: See TracTickets for help on using tickets.