Opened 4 years ago

Last modified 20 months ago

#15008 new defect

Design an opt-in Hidden Service Public Directory Submission system

Reported by: asn Owned by:
Priority: Low Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-hs needs-design research-program
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Hidden services are not as visible as the normal Internet. There are interesting hidden services out there that no one knows about, not even search engines like ahmia. Also, unlike the real internet, the onion space is not very well connected so search engine crawlers have a hard time harvesting more addresses.

We've been discussing adding some kind of system that Hidden Services can use and it will automatically publish their onion address for the whole world to see. This fits to the threat model of Facebook or of blog sites, and we should make sure that it never happens to our private SSH hidden services etc.

This ticket is about designing such a system. I2P has been doing something very similar, apparently with good success and with no drama. See section Incoming Subscriptions and Merging of https://geti2p.net/en/docs/naming. Furthermore, their system is basically a FIFO petname system, where people can register their public keys and match them with a human memorable name.

There are many questions here:

  • Should this be a little-t-tor-feature? Sebastian feels that there is too much complexity in Tor already and adding features like this only intensifies it. He suggested to make this a controller feature, and just have the controller submit our descriptor to a directory. Unfortunately, there is no real controller for hidden services, since TBB is not in the picture, and arm does not work with hidden services. Maybe we could make this a feature of txtorcon or stem?

special pointed out that this feature is not much better than hidden service operators manually submitting their URL to ahmia through the web UI. The problem is that operators might not know of ahmia. A suggestion here was to improve hidden service operator education, and just suggest ahmia in the hidden service documentation. Maybe in the future, if ahmia or onioncity become a big thing, this will be an obvious step to do if you are a hidden service operator.

  • Another big question here is where the onions should go? Would we have an OnionAddressAuth that accepts these announcements, and then spits out a list of onions and their descriptors? But who runs this authority? Is it Tor? Or a friend, like tor2web or ahmia, like the other authorities? Or we decentralize the URL list through the existing directory system (which increases disk space and traffic for directories...). There might be certain legal issues here that we should examine carefully before proceeding.

Also, we are still on the loookout for better names for this system. Especially names that make it obvious that you should only enable this if you belong to the Public Hidden Service threat model.

Child Tickets

Change History (15)

comment:1 Changed 4 years ago by johnakabean

yes, we could register and own our own .onion tld's that, by passphrase, would resolve to the hidden service/relay address. Say, I type mydomain.com.onion where I own mydomain.com in the real root dns servers. It would have the clients connect to the base64? descriptor of my relay hosting the hidden services for port 80 and port 443. instead of the tld resolving to ip addresses, it would resolv to relay hidden service descriptor

Before I was able to have my relay "login" to the mydomain.com.onion "dns zone", to associate my relay with the tld, I registered it on the tor registrar and chose my key to associate my relay to that .onion domain.

Meanwhile, the tor search engines are able to use api to see new registrations to crawl :) I could also add "mydomain.com.tld" as cname or w/e in the REAL dns of "mydomain.com" to have people on the tor network connect by tor FIRST and then have the other A records as secondary.

This way, companies and entities on the real internet that support tor could automatically have people connect to their tor hidden service instead of them having to share and us edit our torrc (for programs that require ip address).

And, as far as the programs that require resolving ip addresses, when we resolv mydomain.com.tld, tor, and only that tor instance, would automatically choose a PRIVATE ip address, to temporarily allocate to the real address.

This means if I go to "google.com.onion", my tor (and only mine) will pick a random private ip to resolve to that address (just like we do manually with "mapaddress") so that client programs requiring an ip address for it's connections can use them.

After a certain SPECIFIABLE time of no activity on that ip address, in torrc, that tor instance will put that ip back on the shelf to be reused again for association with .onion tld resolves. It could even use private ipv6 addresses for no fear of running out of temporary addresses, for a tor that heavily resolves the new tld's. (again, if we choose by setting in torrc)

afaik, this could even make google or any search engine on the public net that is tor-enabled be able to crawl your hidden sites.

Last edited 4 years ago by johnakabean (previous) (diff)

comment:2 Changed 4 years ago by asn

FWIW, we decided as the first step of this ticket to mention ahmia on:
https://www.torproject.org/docs/tor-hidden-service.html.en
so that new HS operators get informed of ahmia's index.

I will try to do this soon.

comment:3 Changed 4 years ago by Jesse V.

I am attempting to solve this with the Onion Name System (OnioNS), a Tor-powered distributed DNS. See https://blog.torproject.org/blog/tor-summer-privacy-projects

comment:4 in reply to:  2 Changed 4 years ago by asn

Replying to asn:

FWIW, we decided as the first step of this ticket to mention ahmia on:
https://www.torproject.org/docs/tor-hidden-service.html.en
so that new HS operators get informed of ahmia's index.

I will try to do this soon.

OK, I wrote some text on that. You can find it in branch bug15008 at https://git.torproject.org/user/asn/hax.git . Or here: https://gitweb.torproject.org/user/asn/hax.git/commit/?h=bug15008&id=2eb0ea0d6553d796314910e4166f6d20b7269737

It basically adds this text:

    It's worth noting that your hidden service's address will not be
    announced to anyone when you set it up. This means that if you
    want people to visit it, you should advertise its onion address
    yourself on social networks, hidden service search engines and
    whatnot. As an example, you can <a
    href="https://ahmia.fi/add/">register your hidden service
    address</a> to the <a href="https://ahmia.fi/">Ahmia hidden
    service index</a>

to the end of
https://www.torproject.org/docs/tor-hidden-service.html.en#three

comment:5 Changed 4 years ago by asn

If people think this is acceptable and useful, I'll find someone to commit upstream.

comment:6 Changed 4 years ago by dgoulet

I think it would be better to mention "Ahmia hidden service search engine" instead of "index" because ahmia crawls thus operator should expect that.

The rest is great. Short and concise. Go for it! :)

comment:7 in reply to:  6 Changed 4 years ago by asn

Replying to dgoulet:

I think it would be better to mention "Ahmia hidden service search engine" instead of "index" because ahmia crawls thus operator should expect that.

The rest is great. Short and concise. Go for it! :)

addressed in fixup commit. thanks!
will wait for some more feedback before pushing.

comment:8 Changed 3 years ago by nickm

Keywords: SponsorR removed
Sponsor: SponsorR

Bulk-replace SponsorR keyword with SponsorR sponsor field in Tor component.

comment:9 Changed 3 years ago by dgoulet

Sponsor: SponsorRSponsorR-can

Move those from SponsorR to SponsorR-can.

comment:10 Changed 2 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:11 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:12 Changed 21 months ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:13 Changed 21 months ago by dgoulet

Severity: Normal
Sponsor: SponsorR-can

comment:14 Changed 21 months ago by nickm

Keywords: needs-design added
Priority: MediumLow

comment:15 Changed 20 months ago by nickm

Keywords: research-program added
Note: See TracTickets for help on using tickets.