Opened 6 months ago

Closed 6 months ago

Last modified 6 months ago

#29919 closed defect (duplicate)

Tor DOS attack help, can a dev take this seriously please?

Reported by: HelpDOS Owned by:
Priority: Very High Milestone:
Component: Core Tor/Tor Version:
Severity: Normal Keywords:
Cc: asn, dgoulet Actual Points:
Parent ID: Points:
Reviewer: Sponsor: Sponsor27-can

Description

Multiple times in the past, I and other hidden service operators have published tickets for help with extreme DoS attacks that are preventing our hidden services from being accessed. This seems like a complete flaw in how Tor works and there is no logical solution in sight.

One reference from an operator I have spoken with:
https://trac.torproject.org/projects/tor/ticket/29607

Exact same attack as myself and we were both initially able to prevent it until the attacker changed their method.

At first the attack was overloading the Tor process CPU of 100% and so new connections could not be processed. After running Tor-vanguards add-on I found a recurring fingerprint for an authority relay was being used for the attack. I excluded the fingerprint in my Torrc and immediately the site was back online. He then changed to ANOTHER authority relay, which seems strange to be a coincidence considering they are low bandwidth relays so wouldn't make sense to me to use in this type of attack.

Again, blocked this fingerprint and then the attack completely changed and is seemingly undiscoverable other than a few warnings that stick out in the logs, but they aren't much help.

Any suggestions we have been given we have tried to no avail. The difference with this ticket, if a Tor dev is actually willing to put some time into investigating, I can provide full server access to an under attack hidden service so it can be actively monitored and hopefully resolved. I beg that someone helps with this, it has been going on for years with no real solutions to similar attacks.

Child Tickets

Change History (8)

comment:1 Changed 6 months ago by HelpDOS

To add to this, Onion Balance does not resolve the issue.

comment:2 Changed 6 months ago by HelpDOS

Some recurrences in the logs:

Your network connection speed appears to have changed. Resetting timeout to 60s after 18 timeouts and 1000 buildtimes.

[warn] Insanely large circuit build timeout value. (timeout = 1966080000.000000msec, close = 1966080000.000000msec)

comment:3 Changed 6 months ago by HelpDOS

Hidden service * exceeded launch limit with 11 intro points in the last 12 seconds. Intro circuit launches are limited to 10 per 300 seconds.

comment:4 Changed 6 months ago by gk

Cc: asn dgoulet added
Component: - Select a componentCore Tor/Tor

comment:5 Changed 6 months ago by asn

Keywords: tor-dos tor-hs added
Sponsor: Sponsor27-can

comment:6 Changed 6 months ago by HelpDOS

Keywords: tor-dos tor-hs removed
Sponsor: Sponsor27-can

I don't know enough about how Tor works to diagnose any further than I already have, with the lack of relevant errors/warnings in any of the logs could it be that he is attacking the introduction points? I read some possible issues for DoS attacks listed in the OnionBalance docs and their explanation of a "Complex Mode", which seems like it could resolve this, however no other references to how to enable that, I am assuming it hasn't yet been implemented.

https://onionbalance.readthedocs.io/en/latest/design.html#complex-mode

If it is currently available, that could be worth us trying but I don't know how to enable it.

This stuck out to me "An onion service instance may rapidly rotate its introduction point circuits when subjected to a Denial of Service attack. An introduction point circuit is closed by the onion service when it has received max_introductions for that circuit. During DoS this circuit rotating may occur faster than the management server polls the HSDir system for new descriptors. As a result clients may retrieve master descriptors which contain no currently valid introduction points." under the Basic mode limitations listed here:
https://onionbalance.readthedocs.io/en/latest/design.html#limitations

Could that relate to the "Server not found" error shown within Tor Browser? When researching that, it is often related to DNS issues in Firefox.

comment:7 in reply to:  6 Changed 6 months ago by asn

Resolution: duplicate
Status: newclosed

Replying to HelpDOS:

I don't know enough about how Tor works to diagnose any further than I already have, with the lack of relevant errors/warnings in any of the logs could it be that he is attacking the introduction points? I read some possible issues for DoS attacks listed in the OnionBalance docs and their explanation of a "Complex Mode", which seems like it could resolve this, however no other references to how to enable that, I am assuming it hasn't yet been implemented.

https://onionbalance.readthedocs.io/en/latest/design.html#complex-mode

If it is currently available, that could be worth us trying but I don't know how to enable it.

This stuck out to me "An onion service instance may rapidly rotate its introduction point circuits when subjected to a Denial of Service attack. An introduction point circuit is closed by the onion service when it has received max_introductions for that circuit. During DoS this circuit rotating may occur faster than the management server polls the HSDir system for new descriptors. As a result clients may retrieve master descriptors which contain no currently valid introduction points." under the Basic mode limitations listed here:
https://onionbalance.readthedocs.io/en/latest/design.html#limitations

Could that relate to the "Server not found" error shown within Tor Browser? When researching that, it is often related to DNS issues in Firefox.

Greetings HelpDOS and sorry for the troubles you are facing.

I'm gonna close this ticket since it's a duplicate of #29607. It's not because we don't care about the issue, but more about keeping things organized and not splitting info over multiple tickets. Please aggregate all info in that ticket.

Sorry for not being swift and responsive on this, but we are very low on resources right now since we are closing one grant (#28634) and we are about to jump to the next one. The good news is that the next grant is about onion services and we will have lots of time to look at issues like this. It's just that now we are completely out of time. We are a small org and we simply dont have enough resources to have incident response teams for incidents like that.

Anyhow, I've been inspecting logs in my spare time and will try to write anything useful I find in #29607.

PS: I don't think that onionbalance complex mode will help here, but I still have not deduced the root of the issue. If you want to experiment with it, let me know how it went.

Last edited 6 months ago by asn (previous) (diff)

comment:8 Changed 6 months ago by asn

Sponsor: Sponsor27-can
Note: See TracTickets for help on using tickets.