New design for broker -- proxy protocol for snowflakes

This is related to the Snowflake protocol design tickets #29206 (moved) and #29293 (moved).

We want to write these protocols in a way that is not Snowflake-specific but allows any type of bridge to connect to or poll our broker/BridgeDB bridge distribution service.

The idea is that in the beginning we will start with very reliable "bridge" (which could be snowflakes) that perhaps rotate IP addresses every month or so. After that we can collect measurements and move towards more ephemeral "bridges".

Some things to keep in mind are the types of information that the snowflakes give to the broker (such as proxy version/type) to allow for metrics. This information might change so we'll want a flexible and extensible protocol.

Trac:
Child Ticket(s): #29734 (moved)

added actualpoints::2 anti-censorship-roadmap component::circumvention/snowflake design ex-sponsor-19 owner::cohosh points::5 priority::high resolution::fixed severity::normal snowflake sponsor::28-must status::closed type::enhancement labels

Trac:
Cc: dcf, arlolra to dcf, arlolra, cohosh

Trac:
Sponsor: N/A to Sponsor19

Trac:
Keywords: N/A deleted, snowflake, design added
Description: Add information that the snowflakes give to the broker (such as proxy version/type) to allow for metrics.

to

This is related to the Snowflake protocol design tickets #29206 (moved) and #29293 (moved).

We want to write these protocols in a way that is not Snowflake-specific but allows any type of bridge to connect to or poll our broker/BridgeDB bridge distribution service.

The idea is that in the beginning we will start with very reliable "bridge" (which could be snowflakes) that perhaps rotate IP addresses every month or so. After that we can collect measurements and move towards more ephemeral "bridges".

Some things to keep in mind are the types of information that the snowflakes give to the broker (such as proxy version/type) to allow for metrics. This information might change so we'll want a flexible and extensible protocol.

Trac:
Points: N/A to 5

As referenced in #29426 (moved), the broker currently gives proxies a 504 message if no client is available which is a questionable design:

2019/02/07 12:11:51 broker returns: 504 INFO: peerconnection.go:468: fired OnIceCandidateError: 143 2019/02/07 12:12:01 broker returns: 504

At the very least it makes logs confusing.

Trac:
Keywords: N/A deleted, network-team-roadmap-2019-Q1Q2 added

We have some preliminary notes about what the protocols between each part of the Snowflake system should accomplish: https://github.com/ahf/snowflake-notes/blob/master/Protocol.markdown

Replying to cohosh:

We have some preliminary notes about what the protocols between each part of the Snowflake system should accomplish: https://github.com/ahf/snowflake-notes/blob/master/Protocol.markdown

Regarding the question

Does the "token bucket" algorithm in Snowflake right now do what we think? I suspect the answer is no. I'm pretty sure BucketRateLimit didn't work before ab34f8e889 (part of #28732 (moved)), and I won't swear that the commit fixed it :) I was tempted, then, to rip out the rate-limit code for the sake of starting from a simpler basis, but decided not to.

I'm pretty sure BucketRateLimit and DummyRateLimit are based off objects of the same name in the flash proxy code. As I recall, they were working in flash proxy.

Trac:
Keywords: network-team-roadmap-2019-Q1Q2 deleted, N/A added

Adding the keyword to mark everything that didn't fit into the time for sponsor 19.

Trac:
Keywords: N/A deleted, ex-sponsor-19 added

Moving from Sponsor 19 to Sponsor 28.

Trac:
Sponsor: Sponsor19 to Sponsor28-must

Trac:
Owner: N/A to ahf
Status: new to assigned
Keywords: N/A deleted, anti-censorship-roadmap added

Replying to cohosh:

As referenced in #29426 (moved), the broker currently gives proxies a 504 message if no client is available which is a questionable design

I suggest changing the status to 204 No Content, it would represent that there's no error but it also no one to connect to.

Trac:
Username: serna

Thanks serna! I like the idea of providing a 2XX response. This will also help with the error/warning messages produced in the browser-based proxy logs.

Replying to serna:

Replying to cohosh:

As referenced in #29426 (moved), the broker currently gives proxies a 504 message if no client is available which is a questionable design

I suggest changing the status to 204 No Content, it would represent that there's no error but it also no one to connect to.

I disagree here. It's better not to hide any necessary information in the HTTP layer, because not every way of interacting with the broker will have those HTTP features. See for example comment:11:ticket:25985. Even assuming HTTP, codes like 204 are probably less likely to pass untouched by proxies. I propose to just always use status code 200, unless there is a genuine internal server error, and encode all necessary information in the HTTP body. We're not designing a REST interface here.

Replying to dcf:

Replying to serna:

Replying to cohosh:

As referenced in #29426 (moved), the broker currently gives proxies a 504 message if no client is available which is a questionable design

I suggest changing the status to 204 No Content, it would represent that there's no error but it also no one to connect to.

I disagree here. It's better not to hide any necessary information in the HTTP layer, because not every way of interacting with the broker will have those HTTP features. See for example comment:11:ticket:25985. Even assuming HTTP, codes like 204 are probably less likely to pass untouched by proxies. I propose to just always use status code 200, unless there is a genuine internal server error, and encode all necessary information in the HTTP body. We're not designing a REST interface here.

Okay, this makes sense to me. So the idea would be to change all codes to 200 and then do a redesign as suggested in #30704 (moved).

Trac:
Owner: ahf to cohosh

Trac:
Priority: Very High to High
Type: task to enhancement