Support onionbalance in HSv3

We are implementing onionbalance in v3! This is the master ticket.

[Description changed to not confuse people with the old design.]

changed milestone to %Tor: unspecified

added component::core tor/tor milestone::Tor: unspecified network-team-roadmap-2020Q1 onionbalance owner::asn parent::29998 points::8 priority::medium resolution::fixed scaling severity::normal sponsor::27-must status::closed tor-hs tor-spec type::defect labels

Trac:
Description: HSv3 need some mods to support the onionbalance design.

That's because of the cross-certs of the intro key in the descriptor (with the desc signing key). Which means that the onionbalance node would need to know the intro privkey to be able to complete the cross-cert with its own descriptor signing key.

Here is an approach to route around this with v3 coming from the Seattle hackfest:

The descriptor-signing private key for each day is generated based on a hash of a shared secret that is shared between the onion service controller and the onion service instances.  This way, the instances know what the signing key for each day will be.  [Because this is a signing key, forward secrecy is not endangered.]

When uploading descriptors, the instances generate "bogus" descriptors (associated with different identity keys) containing intro points and keys generated in a way suitable for including in the combined service's onion descriptor.  They cross-certify the master signing key, not their own descriptors' signing keys.  They upload these descriptors to the hash ring.  They look normal to the hash ring directory servers, since only the encrypted parts are weird.

To generate the combined descriptors, the service controller periodically downloads all the "bogus" descriptors above, and stuffs their contents into a combined descriptor.

Since the shared secret produces the descriptor keys, the instances can  also produce descriptors with the valid descriptor signing key generated from the shared secret.

Instances can optionally use client authentication.

This is quite a bit of mods to support onionbalance, but it does seem like a plausible approach.

We should investigate more and move forward here!

to

HSv3 need some mods to support the onionbalance design.

That's because of the cross-certs of the intro key in the descriptor (with the desc signing key). Which means that the onionbalance node would need to know the intro privkey to be able to complete the cross-cert with its own descriptor signing key.

Here is an approach to route around this with v3 coming from the Seattle hackfest:

The descriptor-signing private key for each day is generated based on a hash of a shared secret that is shared between the onion service controller and the onion service instances. This way, the instances know what the signing key for each day will be. [Because this is a signing key, forward secrecy is not endangered.]

When uploading descriptors, the instances generate "bogus" descriptors (associated with different identity keys) containing intro points and keys generated in a way suitable for including in the combined service's onion descriptor. They cross-certify the master signing key, not their own descriptors' signing keys. They upload these descriptors to the hash ring. They look normal to the hash ring directory servers, since only the encrypted parts are weird.

To generate the combined descriptors, the service controller periodically downloads all the "bogus" descriptors above, and stuffs their contents into a combined descriptor.

Since the shared secret produces the descriptor keys, the instances can also produce descriptors with the valid descriptor signing key generated from the shared secret.

Instances can optionally use client authentication.

This is quite a bit of mods to support onionbalance, but it does seem like a plausible approach.

We should investigate more and move forward here!

Trac:
Cc: N/A to s7r

Just some notes from a recent discussion about onionbalance vs the current poor man's onionbalance where every node races each other for uploading descriptors:

With the poor man's solution, there are issues when you start removing/rebooting nodes, since if the offline node currently has the active descriptor there will be reachability issues until another node wins the race.

We could fix this by making all nodes upload more frequently, and be able to pause publishes from the rebooting node, and also by ensuring that all clients will re-fetch descriptors smoothly if they can't connect to the intro points.

Trac:
Cc: s7r to s7r, gk

Trac:
Sponsor: N/A to Sponsor27-must

Trac:
Points: N/A to 20

Trac:
Parent: N/A to #29998 (moved)

A big choice to make here will be whether to fix #29583 (moved) or not. I think we should fix #29583 (moved), but to do so will create some compatibility issues that we'll need to navigate.

If we don't fix #29583 (moved), this ticket is easier. If we do fix it, we'll need additional machinery to make onionbalance possible on v3 descriptors. I'm attaching a draft proposal I wrote a while ago about how to make that work; we should turn it into a real proposal if we decide to fix #29583 (moved).

Trac:
xxx-onionbalance-v3.txt

Trac:
Keywords: tor-hs scaling onionbalance deleted, tor-hs scaling onionbalance network-team-roadmap-2019-Q1Q2 added

Trac:
Keywords: tor-hs scaling onionbalance network-team-roadmap-2019-Q1Q2 deleted, tor-hs scaling onionbalance network-team-roadmap-september added

I've taken nickm's draft and cleaned it up as an official draft: prop306.

https://lists.torproject.org/pipermail/tor-dev/2019-July/013942.html

For merge, see my torspec.git branch: ticket26768_01

At this point, we'll proceed with the easy approach for OnionBalance v3 that is not fixing #29583 (moved) just now but still having prop306 in the backlog.

Trac:
Keywords: tor-hs scaling onionbalance network-team-roadmap-september deleted, tor-hs scaling onionbalance network-team-roadmap-september tor-spec added

Trac:
Owner: N/A to asn
Status: new to assigned

Points changed at the Stockholm meeting.

Trac:
Points: 20 to 15

Opened ticket about v3 descriptor support for stem: https://trac.torproject.org/projects/tor/ticket/31369#ticket

Still need to figure out how the blinded key generation is gonna work in Python since that's needed for HSPOST.

Replying to asn:

Opened ticket about v3 descriptor support for stem: https://trac.torproject.org/projects/tor/ticket/31369#ticket

Still need to figure out how the blinded key generation is gonna work in Python since that's needed for HSPOST.

We could start with the reference implementation from the tests? https://gitweb.torproject.org/tor.git/tree/src/test/ed25519_exts_ref.py#n34

FWIW, I learned that Joe Landers started working on onionbalance v3 a few months ago and have some stem code and OB code that could be useful to us: ​https://github.com/joelanders/stem/commit/e8455584cf50d7a398f994a7ea761baf3c7d6c00 ​https://github.com/joelanders/onionbalance/commit/1d30e6c5076ec2ee17e4b7a2a63ed72d0c32a670

Reducing the amount of points, since I also assigned points to child ticket #31369 (closed).

Trac:
Points: 15 to 8