Opened 2 years ago

Closed 4 months ago

#27502 closed defect (wontfix)

Prioritize .onion hosts in AltSvc?

Reported by: arthuredelstein Owned by: sysrqb
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: TorBrowserTeam202003
Cc: tbb-team Actual Points:
Parent ID: #30024 Points:
Reviewer: Sponsor: Sponsor27-must

Description

We now support AltSvc and Mahrud suggested we might want to prioritize .onion AltSvc hosts in Tor Browser. It's an interesting idea and I would need to think about it more to know if it's desirable. Mahrud also pointed out a web page that demonstrates AltSvc over .onion:

https://perfectoid.space/test.php

Refresh several times and it will turn green when .onion is being used.

Child Tickets

Change History (19)

comment:1 Changed 2 years ago by tom

Is this saying that if a website says Alt-Svc: a.com, b.onion we should use b.onion before a.com? Doesn't Alt-Svc specify the priority to connect to be in the order provided?

Why do we need to do that? Can't the website put the onion at the beginning and UAs (should) know not to connect to it because it's a reserved domain?

Or is the problem that UAs (Chrome? Edge? Safari?) are dumb and will spin trying to connect to the .onion so a website is forced to put them last?

comment:2 Changed 2 years ago by mahrud

Not quite. Even if there is only one .onion alt-svc presented, sometimes after a few requests the browser seems to ignore the alt-svc and connect directly. I haven't read the code, but my guess is that this happens because the .onion takes a bit longer to load the first time. If that's the case the solution would be to force the browser to prioritize the onion route even if the first request took longer.

On a tangential note: should the Circuit Display show the onion address used when connecting through an alt-svc? Currently it doesn't.

comment:3 Changed 2 years ago by traumschule

Is this about making a redirect to the onion a default? This could break functionality in some cases (for example to be logged in at the clearnet and the onion address with different profiles).
Users can already decide for themselves to use #26581.
Having an option in preferences with a link explaining the implications could be nice though.
comment:51:ticket:21952 points out that domains could use this feature to distinguish TB users from other browsers.

comment:4 in reply to:  2 Changed 2 years ago by tom

Not quite. Even if there is only one .onion alt-svc presented, sometimes after a few requests the browser seems to ignore the alt-svc and connect directly. I haven't read the code, but my guess is that this happens because the .onion takes a bit longer to load the first time. If that's the case the solution would be to force the browser to prioritize the onion route even if the first request took longer.

Ah that makes sense.

Is this about making a redirect to the onion a default?

No. There has been discussion about that in various forms, but this is not that.

comment:5 Changed 22 months ago by btasker

Is this saying that if a website says Alt-Svc: a.com, b.onion we should use b.onion before a.com? Doesn't Alt-Svc specify the priority to connect to be in the order provided?

No, the RFC (7838) says that it's down to the user-agent to decide how to prioritise them - so TBB prioritising .onion would totally be valid.

I support the idea of having TBB prioritise .onion domains:

It lowers the barrier to entry, otherwise a site/server operator is going to need to try and identify exit nodes so that the can decide whether to _only_ include the .onion. That's not particularly hard to do, but is still more work than should actually be required.

Chrome and Firefox already disregard any alternates that they cannot resolve/reach, so it's safe to just include the .onion in all responses.

But if TBB doesn't prioritise it, the browser might instead connect out to one of the other clearnet origins, using exit b/w and undermining the entire point of having the .onion in the header in the first place.

As noted above, where things stand currently is that even if it is selected the onion may initially be slower to respond, leading to it getting de-prioritised.

Or is the problem that UAs (Chrome? Edge? Safari?) are dumb and will spin trying to connect to the .onion so a website is forced to put them last?

Purely for info as you've asked and I've skimmed the patch recently for other purposes:

At time of writing, only Firefox has implemented support for Alt-Svc and HTTP/2.0 alternates. Chrome supports QUIC, but doesn't appear to have implemented anything further.

As far as handling goes, when FF receives the header it triggers an asynchronous null request out to the specified alternates and assesses them (i.e. can they present the right cert etc). Any already queued requests continue to go to the original origin, and new requests will go to the selected alternate.

comment:7 Changed 17 months ago by pili

Sponsor: Sponsor27

comment:8 Changed 16 months ago by gk

Sponsor: Sponsor27Sponsor27-must

Add Sponsor27-must items for Objective 2

comment:9 Changed 16 months ago by pili

Parent ID: #30024

comment:10 Changed 15 months ago by cypherpunks

I think this should be delayed until #30599 is dealt with.

comment:11 Changed 11 months ago by redshiftzero

Similarly, it would also be nice to prioritize v3 onions over v2 onions. For context, for SecureDrop we want to add Alt-Srv headers on our v2 onions to direct traffic to v3 onions.

comment:12 Changed 10 months ago by naif

GlobaLeaks project is definitively interesting in such implementation and willing to implement it in it's own minimalistic Twisted based HTTP1/2/3 support (rif. https://github.com/globaleaks/GlobaLeaks/issues/2687)

comment:13 Changed 7 months ago by sysrqb

Owner: changed from tbb-team to sysrqb
Status: newassigned

This is on my plate now.

comment:14 in reply to:  5 Changed 7 months ago by sysrqb

Cc: tbb-team added
Keywords: TorBrowserTeam202001 added

Replying to btasker:

Is this saying that if a website says Alt-Svc: a.com, b.onion we should use b.onion before a.com? Doesn't Alt-Svc specify the priority to connect to be in the order provided?

No, the RFC (7838) says that it's down to the user-agent to decide how to prioritise them - so TBB prioritising .onion would totally be valid.

To be precise it says:

   When multiple values are present, the order of the values reflects
   the server's preference (with the first value being the most
   preferred alternative).

   The value(s) advertised by Alt-Svc can be used by clients to open a
   new connection to an alternative service.

https://tools.ietf.org/html/rfc7838#page-8

I support the idea of having TBB prioritise .onion domains:

It lowers the barrier to entry, otherwise a site/server operator is going to need to try and identify exit nodes so that the can decide whether to _only_ include the .onion. That's not particularly hard to do, but is still more work than should actually be required.

#32256

Chrome and Firefox already disregard any alternates that they cannot resolve/reach, so it's safe to just include the .onion in all responses.

When/if Chrome supports h2 alt-svc, I assume Chrome will leak the onion address by DNS and then fail, because Chrome didn't respect RFC 7686 the last time I checked.

But if TBB doesn't prioritise it, the browser might instead connect out to one of the other clearnet origins, using exit b/w and undermining the entire point of having the .onion in the header in the first place.

As noted above, where things stand currently is that even if it is selected the onion may initially be slower to respond, leading to it getting de-prioritised.

Or is the problem that UAs (Chrome? Edge? Safari?) are dumb and will spin trying to connect to the .onion so a website is forced to put them last?

Purely for info as you've asked and I've skimmed the patch recently for other purposes:

At time of writing, only Firefox has implemented support for Alt-Svc and HTTP/2.0 alternates. Chrome supports QUIC, but doesn't appear to have implemented anything further.

As far as handling goes, when FF receives the header it triggers an asynchronous null request out to the specified alternates and assesses them (i.e. can they present the right cert etc). Any already queued requests continue to go to the original origin, and new requests will go to the selected alternate.

As I understand it, Firefox only tests one alternate at a time. Firefox maintains a hashmap where the key is based on the origin, and it doesn't include any attributes of the alternative(s). Therefore, Firefox prefers the first alternate in the alt-svc list, but it validates subsequent alt-svc entries after the first entry is validated. This only happens on responses that arrive after the alt-svc validation completes (?).

comment:15 Changed 7 months ago by sysrqb

To some extent it seems Cloudflare is accidentally exacerbating this breakage. On each onion service alternative service it seems they are advertising a different onion service.

2020-01-16 00:07:20.383705 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> perfectoid.space:443
2020-01-16 00:07:22.297908 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443"; ma=86400; persist=1
2020-01-16 00:07:22.312586 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443
2020-01-16 00:09:05.517688 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflarexljc3rw355ysrkrzwapozws6nre6xsy3n4yrj7taye3uiby3ad.onion:443"; ma=86400; persist=1
2020-01-16 00:09:05.537993 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> cflarexljc3rw355ysrkrzwapozws6nre6xsy3n4yrj7taye3uiby3ad.onion:443
2020-01-16 00:10:32.763704 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443"; ma=86400; persist=1
2020-01-16 00:10:53.159927 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> perfectoid.space:443
2020-01-16 00:10:54.882338 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflareub6dtu7nvs3kqmoigcjdwap2azrkx5zohb2yk7gqjkwoyotwqd.onion:443"; ma=86400; persist=1
2020-01-16 00:10:54.904940 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> cflareub6dtu7nvs3kqmoigcjdwap2azrkx5zohb2yk7gqjkwoyotwqd.onion:443
2020-01-16 00:10:55.938627 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflareer7qekzp3zeyqvcfktxfrmncse4ilc7trbf6bp6yzdabxuload.onion:443"; ma=86400; persist=1

I assume this is for load balancing purposes. However, because each new alt service must be validated before it is used, the browser is continuously chasing the next onion service and validating it. Sometimes the validation fails, for whatever reason.

2020-01-16 00:07:20.383705 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> perfectoid.space:443
2020-01-16 00:07:22.297908 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443"; ma=86400; persist=1
2020-01-16 00:07:22.312586 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443
2020-01-16 00:07:25.541287 UTC - [Parent 2414: Socket Thread]: D/nsHttp AltSvcTransaction dtor 0x7f50a89119d0 map 0x7f50a93e64a0 validated 1 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:09:05.517688 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflarexljc3rw355ysrkrzwapozws6nre6xsy3n4yrj7taye3uiby3ad.onion:443"; ma=86400; persist=1
2020-01-16 00:09:05.537993 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> cflarexljc3rw355ysrkrzwapozws6nre6xsy3n4yrj7taye3uiby3ad.onion:443
2020-01-16 00:09:10.233862 UTC - [Parent 2414: Socket Thread]: D/nsHttp AltSvcTransaction dtor 0x7f50a95bed40 map 0x7f50a972d050 validated 1 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:10:32.763704 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443"; ma=86400; persist=1
2020-01-16 00:10:32.776046 UTC - [Parent 2414: Socket Thread]: D/nsHttp AltSvcTransaction dtor 0x7f50a94bda80 map 0x7f50a892f230 validated 0 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:10:53.159927 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> perfectoid.space:443
2020-01-16 00:10:54.882338 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflareub6dtu7nvs3kqmoigcjdwap2azrkx5zohb2yk7gqjkwoyotwqd.onion:443"; ma=86400; persist=1
2020-01-16 00:10:54.904940 UTC - [Parent 2414: Socket Thread]: D/nsSocketTransport SOCKS 0 Host/Route override: perfectoid.space:443 -> cflareub6dtu7nvs3kqmoigcjdwap2azrkx5zohb2yk7gqjkwoyotwqd.onion:443
2020-01-16 00:10:55.938627 UTC - [Parent 2414: Socket Thread]: V/nsHttp Http2Decompressor::OutputHeader alt-svc h2="cflareer7qekzp3zeyqvcfktxfrmncse4ilc7trbf6bp6yzdabxuload.onion:443"; ma=86400; persist=1
2020-01-16 00:10:57.336038 UTC - [Parent 2414: Socket Thread]: D/nsHttp AltSvcTransaction dtor 0x7f50a93bac90 map 0x7f50aa251e20 validated 1 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:07:20.342876 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/test.php
2020-01-16 00:07:20.342936 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a8877000 Using default connection info
2020-01-16 00:07:22.330752 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/favicon.ico
2020-01-16 00:07:22.330811 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a9440000 Using default connection info
2020-01-16 00:09:04.881885 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/test.php
2020-01-16 00:09:04.882120 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a9443000 Alt Service Mapping Found https://cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:09:04.882144 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a9443000 Using connection info from altsvc mapping
2020-01-16 00:09:05.558213 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/favicon.ico
2020-01-16 00:09:05.558259 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a94ee000 Alt Service Mapping Found https://cflares35lvdlczhy3r6qbza5jjxbcplzvdveabhf7bsp7y4nzmn67yd.onion:443 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:09:05.558266 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a94ee000 Using connection info from altsvc mapping
2020-01-16 00:10:32.080806 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/test.php
2020-01-16 00:10:32.080862 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a896b000 Alt Service Mapping Found https://cflarexljc3rw355ysrkrzwapozws6nre6xsy3n4yrj7taye3uiby3ad.onion:443 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:10:32.080867 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a896b000 Using connection info from altsvc mapping
2020-01-16 00:10:32.787419 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/favicon.ico
2020-01-16 00:10:32.787463 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a95f4000 Alt Service Mapping Found https://cflarexljc3rw355ysrkrzwapozws6nre6xsy3n4yrj7taye3uiby3ad.onion:443 [https:perfectoid.space:443:P:^privateBrowsingId=1&firstPartyDomain=perfectoid.space]
2020-01-16 00:10:32.787469 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a95f4000 Using connection info from altsvc mapping
2020-01-16 00:10:53.149106 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/test.php
2020-01-16 00:10:53.149207 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a8929000 Using default connection info
2020-01-16 00:10:54.938840 UTC - [Parent 2414: Main Thread]: D/nsHttp uri=https://perfectoid.space/favicon.ico
2020-01-16 00:10:54.938894 UTC - [Parent 2414: Main Thread]: D/nsHttp nsHttpChannel 0x7f50a97e0000 Using default connection info

Firefox *should* continue using the previously validated alt service when the new address fails (as long as the previous alt svc hasn't expired). This seems to be a (or the) bug (see #30599).

With regard to prioritizing .onion alt services, I'm leaning toward not - but we can discuss it with Mozilla as to whether they'd uplift it.

comment:16 Changed 7 months ago by sysrqb

Status: assignedneeds_information

While #30599 is in needs_info, I'm curious if this is actually needed. Are there any examples of sites where they advertise a .onion and non-.onion address as alt services? Cloudflare only advertise a .onion when they detect a connection from the Tor network (and they don't advertise alt-svc otherwise).

comment:17 Changed 6 months ago by pili

Keywords: TorBrowserTeam202002 added; TorBrowserTeam202001 removed

Moving tickets to February

comment:18 Changed 5 months ago by pili

Keywords: TorBrowserTeam202003 added; TorBrowserTeam202002 removed

We are no longer in February, moving tickets

comment:19 Changed 4 months ago by pili

Resolution: wontfix
Status: needs_informationclosed

We decided to close this one out in the end for the following reasons:

  1. firefox's current implementation makes a change like this a little tricky
  2. tor browser shouldn't make arbitrary prioritization decisions about alt-svc

If website operators want their .onion to be preferred, then they can tell the browser that by only including a .onion or putting the .onion address first in the list

More details on the discussion: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-24-14.59.log.html

Note: See TracTickets for help on using tickets.