Investigate Snowflake blocking in China
We've been receiving updates from amiableclarity2011 about snowflake issues in China for a while now (they first reported the blocking of our default STUN servers a while back).
Recently they have not been able to bootstrap a connection at all. I wrote a script for monitoring snowflake health (#32545 (moved)) to check to see if this is due to the faulty snowflake problem we've been having, and ran these tests from a VPS in China and a VPS in North America. I was expecting to see about half of the snowflake connection attempts to succeed and set a 3 minute time out on circuit construction. However, what I saw was a 90% success rate from North America and a 0% success rate in China.
This almost certainly due to blocking, and as ambleclarity recently confirmed in #32597 (moved), not due to the blocking of STUN servers.
Activity
Even though all the file downloads failed, there is still some data getting through (at least for a while). I took a look at the logs and noticed some Tor connections through snowflake are fully bootstrapping from the VPS:
[1] "Number of failed downloads: 100" [1] "Average throughput (KB/s): NaN" [1] "Standard deviation: NA" [1] "Number of failed snowflakes: 42" [1] "Number of full bootstraps: 49" [1] "Average bootstrap progress: 56.95"If a snowflake doesn't bootstrap past 10% I counted it as failed. It looks like about half of boostraps succeeded. This is much lower than the > 90% success rate we get from a VPS in North America, and the fact that all file downloads fail is also concerning. These connections are over UDP, so they can't be blocked by sending a RST packet. It's possible that the GFW is more actively monitoring and blocking snowflake IP addresses, or that there is some forced packet loss happening that snowflake can't handle well.
Replying to cohosh:
This almost certainly due to blocking, and as ambleclarity recently confirmed in #32597 (moved), not due to the blocking of STUN servers.
Could it also be exacerbated by #32129 (moved), which has the proxy-go's polling every 5 s and the web-based proxies polling every 300 s? Blocking the single proxy-go IP address would therefore block a large fraction of effective proxies.
But even so, that should only cause a failure rate of about 10%. https://snowflake-broker.torproject.net/debug says:
current snowflakes available: 41 standalone proxies: 4 browser proxies: 0 webext proxies: 37 unknown proxies: 0-
I've updated the bridgetest scripts used in #30368 (moved) to consolidate the different kinds of tests we've been running and hopefully collect enough information to figure out what's going on here: https://github.com/cohosh/bridgetest
I have a test running in north america and in china on a VPS at the moment.
-
-
-
-
Here are the results from some throughput tests I ran from the VPS in China on December 6th:
To summarize the tests, I attempted to bootstrap a Tor connection using Snowflake as the pluggable transport 100 times. If the bootstrap was successful, I then proceeded to download a 1MB file using
torsocks. I measured the bootstrap progress (i.e., how far the Tor connection bootstrap proceeded before the 90 second timeout), and also looked at packet captures of UDP traffic to determine where, if at all, the Snowflake WebRTC connection failed.
Surprisingly, almost all Snowflake connections succeeded this time around, with only 4 connections that failed to bootstrap to 100%.
Looking at the snowflake connection data, only three connections to the snowflake provided by the snowflake-broker failed at the phase where the client tries to open a data channel to the snowflake.
The throughput was also better than expected:
The mean throughput was 110 KBps, with a standard deviation of 80 KBps. This isn't that much slower than the throughput of 190 KBps reported from a VPS in Canada in #32545 (moved):comment3. Some slowness is to be expected given the geographical distance of this probe site.
This data contradicts the previous tests above where half of all snowflakes are failing. We used the same basic probe test to collect this data, which suggests that if censorship is happening, it is not consistent yet. I'm going to set up some recurring probes of this type to see if the reachability of snowflake changes over the next few days.
Trac:
Status: assigned to needs_information mentioned in issue #32833 (moved)
