Opened 8 years ago

Closed 4 years ago

#3521 closed enhancement (fixed)

Allow controllers to retrieve HS descriptors from Tor

Reported by: rransom Owned by: dgoulet
Priority: Medium Milestone: Tor: 0.2.8.x-final
Component: Core Tor/Tor Version: Tor: 0.2.7
Severity: Keywords: tor-hs, controller, needs-proposal, SponsorS
Cc: kostas@…, i@… Actual Points:
Parent ID: #8993 Points: medium
Reviewer: Sponsor: SponsorR

Description


Child Tickets

TicketTypeStatusOwnerSummary
#14845enhancementcloseddgouletController: retrieve an HS descriptor from the client's cache
#14846enhancementcloseddonnchaController: retrieve an HS descriptor of a service run by a user
#14847enhancementcloseddgouletController: add a command to fetch HS descriptor from HSdir(s)

Change History (25)

comment:1 Changed 8 years ago by rransom

I want a client's controller to be able to:

  • retrieve the current descriptor for a hidden service run by the client,
  • retrieve hidden service descriptors from the client's client-side HS descriptor cache, and
  • cause Tor to fetch a hidden service's descriptor from the HSDir system.

We currently have no way to retrieve a hidden service's current set of introduction points for debugging purposes, and we could have used one today.

comment:2 Changed 8 years ago by nickm

Milestone: Tor: 0.2.3.x-finalTor: unspecified

comment:3 Changed 7 years ago by nickm

Keywords: needs-proposal added

comment:4 Changed 7 years ago by nickm

Keywords: tor-hs added

comment:5 Changed 7 years ago by nickm

Component: Tor Hidden ServicesTor

comment:6 Changed 6 years ago by asn

Me and hellais wrote a small patch for this issue. You can find it in branch bug8891 in https://git.torproject.org/user/asn/tor.git.

It's a bit of a PoC and it was done the easiest way possible: You send GETINFO hs/desc/id/idnxcnkne4qt76tg to the control port, and Tor will search its cache of hidden service descriptors and if it finds a descriptor for idnxcnkne4qt76tg.onion it spits it out.

In the future we will also want a command that actually fetches the descriptor even if it's not cached.

comment:7 Changed 6 years ago by asn

Note to self: write changes file and update torspec.

comment:8 Changed 6 years ago by nickm

Milestone: Tor: unspecifiedTor: 0.2.5.x-final

Needs a spec patch IMO before the implementation is reviewable; please put the ticket in needs_review once it has one and you think it's reviewable.

Things to specify: does it look at client-side cache, server-side cache, or both? Does it return the descriptors for your own hidden services? And is there a way to enumerate the cache?

comment:9 Changed 6 years ago by atagar

In the future we will also want a command that actually fetches the descriptor even if it's not cached.

Shouldn't this be done before merging? It seems pretty clunky to require controllers to figure out how to populate the cache before it's useable.

comment:10 Changed 6 years ago by arma

Parent ID: #8993

comment:11 Changed 6 years ago by grarpamp

Parts of this thread contain a possible command operation spec for this feature, including first/cache retrievals, sources, descriptor content printing, etc...

Subject: regarding control spec for hidden service descriptor
https://lists.torproject.org/pipermail/tor-dev/2013-September/005349.html

comment:12 Changed 5 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.???

comment:13 Changed 5 years ago by wfn

Cc: kostas@… added

comment:14 in reply to:  6 Changed 5 years ago by wfn

Replying to asn:

Me and hellais wrote a small patch for this issue. You can find it in branch bug8891 in https://git.torproject.org/user/asn/tor.git.

It's a bit of a PoC and it was done the easiest way possible: You send GETINFO hs/desc/id/idnxcnkne4qt76tg to the control port, and Tor will search its cache of hidden service descriptors and if it finds a descriptor for idnxcnkne4qt76tg.onion it spits it out.

In the future we will also want a command that actually fetches the descriptor even if it's not cached.

I've been tinkering with or/control.c to allow for [1]-type of descriptor fetching (cache, network (i.e. "force network refetch" kind of logic.)) But, the "fetch descriptors from directories" logic is asynchronous in nature (because that's the the way it all works (libevent)) (rend_client_refetch_v2_renddesc() -> directory_get_from_hs_dir() which chooses the dirauth to query, etc., and which in turn calls directory_initiate_command_routerstatus_rend() -> connection_connect() -> send things, and then register via connection_watch_events(), or connection_start_reading()).

Therefore, implementing a "try to remotely fetch an HS descriptor and return that exact descriptor" type of thing turned out not to be that easy. I suppose one could start from the bottom-up and not use things like directory_get_from_hs_dir(), but I'm not sure. The problem is that the "look up HS descriptor cache entry" and "fetch and return HS descriptor" logics are different.

Hence it would probably make sense to have two different GETINFO commands: for cache lookup, and for network fetch (the latter would return asynchronously, in a way that control-spec allows us to do.) Or, to make it more (e.g. stem-)user-friendly, the GETINFO command could allow for asynchronous response, even for cache-based reply. So if cache was hit, it would return pretty much immediately, but the controlling client should allow for async controller responses.

This is just in case someone else is considering implementing the same thing. It would be useful for sure.

[1]: https://lists.torproject.org/pipermail/tor-dev/2013-September/005366.html

Last edited 5 years ago by wfn (previous) (diff)

comment:15 in reply to:  11 Changed 5 years ago by grarpamp

Replying to 11:
Michael of briarproject posted a patch to flush HS descriptor cache:
# Patches to improve mobile hidden service performance
https://lists.torproject.org/pipermail/tor-dev/2014-October/007590.html

comment:16 Changed 5 years ago by nickm

Status: newneeds_review

comment:17 Changed 5 years ago by dgoulet

Keywords: SponsorR controller added
Owner: changed from rransom to dgoulet
Status: needs_reviewassigned

Taking ownership of this one, I'm working on a patch on the control-spec document for this feature. Will flag it back in need_review when the patch is ready.

comment:18 Changed 4 years ago by dgoulet

Milestone: Tor: 0.2.???Tor: 0.2.7.x-final

comment:19 Changed 4 years ago by nickm

Keywords: 027-triaged-1-in added

Marking some tickets as triaged-in for 0.2.7 based on early triage

comment:20 Changed 4 years ago by isabela

Keywords: SponsorS added
Points: medium
Version: Tor: 0.2.7

comment:21 Changed 4 years ago by nickm

Milestone: Tor: 0.2.7.x-finalTor: 0.2.8.x-final

comment:22 Changed 4 years ago by virgilgriffith

Cc: i@… added

comment:23 Changed 4 years ago by nickm

Keywords: SponsorR removed
Sponsor: SponsorR

Bulk-replace SponsorR keyword with SponsorR sponsor field in Tor component.

comment:24 Changed 4 years ago by dgoulet

Keywords: 027-triaged-1-in removed

comment:25 Changed 4 years ago by dgoulet

Resolution: fixed
Status: assignedclosed

All child tickets are now closed! Wow closing a 3000-ish ticket as completed, epic! :)

Note: See TracTickets for help on using tickets.