Opened 3 years ago

Closed 2 years ago

Last modified 2 years ago

#12844 closed task (fixed)

Allow Tor2Web mode to specify its own Rendezvous Point

Reported by: asn Owned by:
Priority: Medium Milestone: Tor: 0.2.6.x-final
Component: Core Tor/Tor Version:
Severity: Keywords: tor-hs, nickm-review
Cc: virgilgriffith Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Virgil from tor2web is interested in implementing a feature of tor2web mode that will allow Tor to specify a specific node as the rendezvous point of an HS circuit.

The idea here is that if tor2web specifies itself as the RP, the latency of RP establishment during HS circuit building will decrease. (I don't know if this is true or whether the speedup will be measurable, but it's worth trying).

My idea for implementing this feature is to introduce a torrc option (temp name Tor2webRendezvousPoints) that acts similar to ExitNodes etc., and forces Tor to select one of the specified RPs as the RP of all HS circuits.

I'm calling this a tor2web feature since I don't see much use for non-tor2web clients.

Child Tickets

Attachments (1)

tor2web_rp_instructions.txt (2.4 KB) - added by asn 3 years ago.

Download all attachments as: .zip

Change History (38)

comment:1 Changed 3 years ago by nickm

  • Milestone set to Tor: 0.2.???

comment:2 Changed 3 years ago by asn

I did a bit of work on this. Virgil, some questions:

a)

Tor has an optimization when building circuits, where if there
already is an unused circuit lying around, it cannibalizes it and
uses that instead. This can also happen in HS circuits, and in that
case the exit node of the cannibalized circuit becomes the RP of
the HS circuit.

It's unclear what Tor2Web wants to do with regards to this. I have
currently disabled the canibalization completely, so that all HS
circuits use the provided RP node.

Another approach would be to always prefer canibalizing circuits if
we can find any, with the reasoning that circuit establishment is
the most costly procedure of the HS circuit (not sure if it's
true). Of course in this case, you won't be the RP anymore, so you
will have more latency after circuit establishment.

You can imagine optimizations here, where Tor keeps creating many
circuits to our RP, so that those get canibalized later. I don't
think I have time to properly implement this approach, and I'm not
sure if it can get merged upstream.

FWIW, I'm not sure how frequent canibalization will be in a use
case like Tor2Web. I will look a bit into this.

b)

What should happen if the white-listed RP is not available in the
network?

Normally this should not happen in your use case (since you run the
RP), so I'm thinking of just refusing to establish any RP
circuits. However, since this is tailored for tor2web, would you
prefer it if Tor picked a random other node when the whitelisted RP
is not available?

Last edited 3 years ago by asn (previous) (diff)

comment:3 follow-up: Changed 3 years ago by virgilgriffith

(a) I support the simplicity of "always use the designated RP" instead of cannibalizing circuits. Multi-layered funny exceptions confuse my meager brain.

(b) I support sending an error to the logfile (high severity) and then going with a random node as the RP.

comment:4 Changed 3 years ago by virgilgriffith

An aside:

My idea for implementing this feature is to introduce a torrc option (temp name Tor2webRendezvousPoints) that acts similar to ExitNodes etc., and forces Tor to select one of the specified RPs as the RP of all HS circuits.

Is there a reason to make this a specific feature of Tor2web mode? Why not just call it "RendezvousPoints" which works just like ExitNodes ? Then if a user wants to set herself as the sole entry in RendezvousPoints that her liability.

Last edited 3 years ago by virgilgriffith (previous) (diff)

Changed 3 years ago by asn

comment:5 Changed 3 years ago by asn

  • Status changed from new to needs_review

OK. I think this is ready. You can find the code in branch bug12844 in https://git.torproject.org/user/asn/tor.git. Here is the gitweb repo: https://gitweb.torproject.org/user/asn/tor.git/shortlog/refs/heads/bug12844

I also wrote some unittests and man page documentation.
I also attached some use intructions at attachment:tor2web_rp_instructions.txt.
Hope it works for you. Let me know if it doesn't.

BTW, while working on this ticket, I found a weird HS bug (#13151). Till the bug is fixed in tor, it's likely that your setup is not going to work if you don't apply Roger's client-side patch to your tor2web tor. You can find Roger's patch at comment:4:ticket:13151.

comment:6 in reply to: ↑ 3 Changed 3 years ago by asn

Replying to virgilgriffith:

(a) I support the simplicity of "always use the designated RP" instead of cannibalizing circuits. Multi-layered funny exceptions confuse my meager brain.

Done.

(b) I support sending an error to the logfile (high severity) and then going with a random node as the RP.

Done.

Is there a reason to make this a specific feature of Tor2web mode? Why not just call it "Tor2webRendezvousPoints" which works just like ExitNodes ? Then if a user wants to set herself as the sole entry in RendezvousPoints that her liability.

I think I prefer it this way since the feature can actually harm client anonymity, and I don't see an actual reason for non-tor2web clients to do this. Furthermore, the behavior of "always use the designated RP, but if it's down just pick a random node as RP" is not secure, and it's likely not what normal users expect. If we later decide to make this feature available to non-tor2web clients it's quite easy to do.

comment:7 Changed 3 years ago by nickm

  • Milestone changed from Tor: 0.2.??? to Tor: 0.2.6.x-final

comment:8 Changed 3 years ago by naif

This is a duplicate of #9685 (that should be closed) .

Aside from this RP improvement, it would be nice also to:

  • Make Tor2web to be released with the default version of Tor, by adding some difficult extra layer like commandline arguments --yes-i-know-that-i-am-taking-risks-by-running-tor2web-mode

OR

  • Make Tor Project to release a Tor2web-enabled version of Tor as part of it's release cycle

That would be very important to enable easier installation/deployment of Tor2web software having Tor software as it's core dependency.

comment:9 follow-up: Changed 3 years ago by naif

Question: To became an RP, you must be a Relay?

Because currently a Tor2web node is yet not a Relay itself, if it must became, we shall adjust the way it create the default configuration file and have some specific default setting while acting as a Relay too.

comment:10 in reply to: ↑ 9 ; follow-up: Changed 3 years ago by asn

Replying to naif:

Question: To became an RP, you must be a Relay?

Correct. There is no other way.
Please see attachment:tor2web_rp_instructions.txt​ .

comment:11 follow-up: Changed 3 years ago by nickm

  1. Why disable cannibalization when we're using this feature? I'm not sure what the benefit is supposed to be, and I don't see it explained above.
  1. I'd be more comfortable if the code here were mostly #if'd out when we aren't building with tor2web mode. (e.g, if the pick_tor2web_rendezvous node didn't exist for most builds.)
  1. Should we safe_str_client() our choice of rendezvous point?

Other than that, does it look okay to me. Does it work?

comment:12 Changed 3 years ago by nickm

  • Status changed from needs_review to needs_revision

comment:13 in reply to: ↑ 11 Changed 3 years ago by asn

Replying to nickm:

  1. Why disable cannibalization when we're using this feature? I'm not sure what the benefit is supposed to be, and I don't see it explained above.

I tried to explain in comment:2.

Cannibalization does not enforce a last hop in the case of RP circuits:
https://gitweb.torproject.org/tor.git/blob/00fafe9ab43a6ee8ca5d1338995f9c64b338bdc1:/src/or/circuituse.c#l1704

If I wanted to allow canibalization, I would either have to extend to a 4th hop (the wanted RP) or only accept canibalization when the last hop is the wanted RP. So I decided to not tinker with that part of the code, and disable canibalization for RP circuits completely.

BTW, note that cannibalization is only disabled for client RP circuits (purpose == CIRCUIT_PURPOSE_C_ESTABLISH_REND) if Tor2webRendezvousPoints is enabled.

Is there a better behavior?

  1. I'd be more comfortable if the code here were mostly #if'd out when we aren't building with tor2web mode. (e.g, if the pick_tor2web_rendezvous node didn't exist for most builds.)

Me too. But I'm calling these functions from the unittests. Should I #if out the unittests too?

Unfortunately, I just checked and the test binary does not work when Tor2web-mode is enabled...

$ ./src/test/test
Sep 22 21:22:55.408 [err] This copy of Tor was compiled to run in 'tor2web mode'. It can only be run with the Tor2webMode torrc option enabled.
Sep 22 21:22:55.408 [err] set_options(): Bug: Acting on config options left us in a broken state. Dying.
  1. Should we safe_str_client() our choice of rendezvous point?

Done and pushed!

Other than that, does it look okay to me. Does it work?

comment:14 Changed 2 years ago by virgilgriffith

This is officially ready for running performance tests on to see if it makes a difference?

comment:15 follow-up: Changed 2 years ago by virgilgriffith

Is an acceptable test for this simply pinging the top 100 most popular .onion services and getting the median response time with and without setting Tor2webRendezvousPoints to the second Tor instance?

I just want to verify that I'm performing the appropriate test.

comment:16 in reply to: ↑ 15 Changed 2 years ago by asn

Replying to virgilgriffith:

Is an acceptable test for this simply pinging the top 100 most popular .onion services and getting the median response time with and without setting Tor2webRendezvousPoints to the second Tor instance?

I just want to verify that I'm performing the appropriate test.

Yes, this is as ready as it will ever be.

Your test seems appropriate, but I would suggest to look at what karsten has been doing over at #1944.
He is using torperf which is more robust, and he also has a Tor patch for cleaning the rendezvous caches of Tor that impact performance.

comment:17 Changed 2 years ago by nickm

  • Milestone changed from Tor: 0.2.6.x-final to Tor: 0.2.7.x-final

comment:18 Changed 2 years ago by virgilgriffith

I offer $300 for this patch to be included in the 0.2.6.x release.

comment:19 Changed 2 years ago by nickm

  • Milestone changed from Tor: 0.2.7.x-final to Tor: 0.2.6.x-final

We can take another look at the patch; no payment is necessary. (If it's ready, we should merge it for free. If it isn't ready, we really shouldn't merge it.)

comment:20 Changed 2 years ago by nickm

  • Keywords andrea-review added

comment:21 Changed 2 years ago by nickm

  • Status changed from needs_revision to needs_review

comment:22 Changed 2 years ago by nickm

  • Keywords nickm-review added; andrea-review removed

comment:23 Changed 2 years ago by nickm

  • Resolution set to implemented
  • Status changed from needs_review to closed

I added some extra ifdef guards as discussed above. The trick was to look at defined(ENABLE_TOR2WEB_MODE) || defined(TOR_UNIT_TESTS) . Then I merged to master and fixed up some conflicts. Review of merge commit would be appreciated. (caf28519d9a4f7f0a5fa36a0cd94fe9c0ee292dc)

comment:24 Changed 2 years ago by asn

Merge commit looks good to me.

comment:25 Changed 2 years ago by arma

For posterity, there was once (long ago) a RendNodes torrc option that did this feature. (It was removed in 0.2.1.3-alpha.)

Last edited 2 years ago by arma (previous) (diff)

comment:26 in reply to: ↑ 10 ; follow-up: Changed 2 years ago by arma

Replying to asn:

Replying to naif:

Question: To became an RP, you must be a Relay?

Correct. There is no other way.

Hrm. But it doesn't have to be that way. In fact, I think #9685 was about opening an ORPort on the tor2web client, and setting the rendezvous point to that ip:port:fingerprint, even though it is not a known relay. Should that idea get rescued somewhere in a ticket? It seems a shame to build something different and then close #9685 as a duplicate.

comment:27 Changed 2 years ago by arma

For code cleanliness, I think you don't want to check #if defined(ENABLE_TOR2WEB_MODE) in circuitbuild.c, but instead rely on the value of options->Tor2webMode, which is enforced in options_act(). That's what we do elsewhere in the code, e.g. connection_ap_handshake_rewrite_and_attach().

I actually got snookered thinking that grepping for ENABLE_TOR2WEB_MODE was the way to find all the changes that are affected by tor2web mode, because that's how this change was enabled.

comment:28 Changed 2 years ago by arma

  • Resolution implemented deleted
  • Status changed from closed to reopened

Ok, I feel enough about the code cleanliness comment above that I'm going to reopen the ticket so other people notice it too.

comment:29 follow-up: Changed 2 years ago by nickm

I think that the right fix is not to remove the #ifdefs, but to add more #ifdefs. Tor2web code should not be compiled for average users; otherwise is begging for trouble.

comment:30 in reply to: ↑ 29 Changed 2 years ago by arma

Replying to nickm:

I think that the right fix is not to remove the #ifdefs, but to add more #ifdefs. Tor2web code should not be compiled for average users; otherwise is begging for trouble.

I could get behind that. Consistency is what I want.

comment:31 Changed 2 years ago by naif

I would really love if Tor would build a Tor version with Tor2web enabled, sitting in it's own repository, but being build explicitly for Tor2web uses. It would over-simplify the Tor2web setup process.
If i open a ticket about it, any chance to get a Tor release explicitly build for Tor2web mode?

comment:32 Changed 2 years ago by nickm

  • Status changed from reopened to needs_review

The change I think is desirable is in branch "bug12844_macros" in my public repository.

comment:33 Changed 2 years ago by yawning

The "bug12844_macros" branch lgtm.

comment:34 Changed 2 years ago by nickm

  • Resolution set to fixed
  • Status changed from needs_review to closed

Merged; thanks!

comment:35 in reply to: ↑ 26 Changed 2 years ago by asn

Replying to arma:

Replying to asn:

Replying to naif:

Question: To became an RP, you must be a Relay?

Correct. There is no other way.

Hrm. But it doesn't have to be that way. In fact, I think #9685 was about opening an ORPort on the tor2web client, and setting the rendezvous point to that ip:port:fingerprint, even though it is not a known relay. Should that idea get rescued somewhere in a ticket? It seems a shame to build something different and then close #9685 as a duplicate.

Yeah that would be an alternative approach yes. I reopened #9685.

comment:36 follow-up: Changed 2 years ago by virgilgriffith

@asn: I attempted this on some EC2 machines. Can you paste me a log of one your tor2web instances correctly setting the RP to:

"25199EBF02D6F9C93DEEC0CC15DC6F690ED928CA~OnionCityRP"

I got the relay setup but so far no dice in getting the client tor2web machine to use it.

If it helps the version of Tor I'm running on both the tor2web-node and the RP is "version 0.2.6.7 (git-c1a17aafcc854762)".

comment:37 in reply to: ↑ 36 Changed 2 years ago by asn

Replying to virgilgriffith:

@asn: I attempted this on some EC2 machines. Can you paste me a log of one your tor2web instances correctly setting the RP to:

"25199EBF02D6F9C93DEEC0CC15DC6F690ED928CA~OnionCityRP"

I got the relay setup but so far no dice in getting the client tor2web machine to use it.

If it helps the version of Tor I'm running on both the tor2web-node and the RP is "version 0.2.6.7 (git-c1a17aafcc854762)".

The feature still seems to work for me, even with tor-0.2.6.7.
Unfortunately, the OnionCityRP relay does not seem to be up.

For my testing, I used the 'beer' relay and I set:

Tor2webRendezvousPoints B120B6310D626FA54B6667DCAFEBD17A1EF47890

in my torrc: https://globe.torproject.org/#/relay/B120B6310D626FA54B6667DCAFEBD17A1EF47890

Then in my info logs I got:

Apr 20 23:33:58.000 [info] choose_good_exit_server(): Picked new RP: $B120B6310D626FA54B6667DCAFEBD17A1EF47890~beer at 193.166.167.68

FWIW, I will be less active the next weeks.

Note: See TracTickets for help on using tickets.