Opened 3 months ago

Last modified 10 hours ago

#29279 assigned task

Reach out to NGOs to test obfs4 reachability

Reported by: cohosh Owned by: cohosh
Priority: Very High Milestone:
Component: Obfuscation/Obfsproxy Version:
Severity: Normal Keywords: NGO, community
Cc: cohosh, phw Actual Points:
Parent ID: Points:
Reviewer: Sponsor: Sponsor19

Description

Check to see if obfs4 is actually being blocked/throttled in China and other areas. If it is, we should figure out how and see if we can make changes to obfs4 to make it viable in these regions again.

Child Tickets

TicketStatusOwnerSummaryComponent
#29297closedcohoshWrite reachability tests to verify if obfs4 is working or notObfuscation/Obfsproxy

Attachments (8)

obfs4-reachability-2019-04-04.pdf (9.1 KB) - added by cohosh 3 weeks ago.
obfs4-reachability-2019-04-08.pdf (13.2 KB) - added by cohosh 2 weeks ago.
obfs4 reachability results in China as of 2019-04-08
obfs4-reachability-2019-04-08.png (26.7 KB) - added by cohosh 2 weeks ago.
obfs4 reachability results in China as of 2019-04-08
infer-throughput.py (2.6 KB) - added by phw 2 weeks ago.
Python script to infer a download's throughput over time.
plot-throughput.R (343 bytes) - added by phw 2 weeks ago.
R script to visualise throughput over time.
throughput_ca.pdf (19.7 KB) - added by cohosh 2 weeks ago.
Throughput measurements from North American probe site as of 2019-04-09
throughput_cn.pdf (36.8 KB) - added by cohosh 2 weeks ago.
Throughput measurements from Chinese probe site as of 2019-04-09
obfs4-reachability-2019-04-09.pdf (14.5 KB) - added by cohosh 2 weeks ago.
obfs4 reachability results in China as of 2019-04-08

Download all attachments as: .zip

Change History (25)

comment:1 Changed 3 months ago by teor

Owner: asn deleted
Status: newassigned

asn does not need to own any obfuscation tickets any more. Default owners are trouble.

comment:2 Changed 3 months ago by cohosh

Owner: set to cohosh

Noting some previous possible mentions of blocking:

China: #19408 (2016)
Kazakhstan: #20348 (2016-?)
Poland: #27435 (2018)
UAE: #27723, #25966 (2017/2018)

Some things that might be the cause of blocking:
#7349, #28655, #26083, #26122

Tickets related to reachability tests:
#17159, #29275, #29277

comment:3 Changed 7 weeks ago by cohosh

The current plan is to do the following:

1) Set up several diverse, new, private obfs4 bridges for these tests

2) Write a client-side script that will test for obfs4 reachability, making sure to measure bandwidth for throttling (#29297)

3) Get set up on a VPS to perform these tests ourselves initially

4) Contact users to run the scripts and send us collected data

comment:4 Changed 7 weeks ago by gaba

Keywords: network-team-roadmap-2019-Q1Q2 added

comment:5 Changed 5 weeks ago by cohosh

Summary: Reach out to NGOs about obfs4Reach out to NGOs to test obfs4 reachability

comment:6 Changed 3 weeks ago by phw

Cc: phw added

Changed 3 weeks ago by cohosh

comment:7 Changed 3 weeks ago by cohosh

Here's some preliminary results after 3 days of testing.

The bridges ndnop3 and ndnop4 were used in previous testing so it's likely they became blocked long before this test for other reasons.

So far the private bridges we set up for these tests have not been blocked and the throughput is not being throttled.


Changed 2 weeks ago by cohosh

obfs4 reachability results in China as of 2019-04-08

Changed 2 weeks ago by cohosh

obfs4 reachability results in China as of 2019-04-08

comment:8 Changed 2 weeks ago by cohosh

So far the private bridges still look reachable (despite the fact that we're trying to reach known censored bridges from the VPS. There was a weird blip on saturday for one of the bridges.

obfs4 reachability results in China as of 2019-04-08

Bandwidth results also look good.

comment:9 Changed 2 weeks ago by phw

Interesting results! It would be great if we could add one of the two unblocked bridges to BridgeDB at some point, and see how long it takes for it to be blocked. Ideally, we would have several unblocked bridges and add them to different BridgeDB distribution sets, which would give us an idea of what distribution mechanisms are crawled successfully, and how quickly.

comment:10 Changed 2 weeks ago by cohosh

That's a great idea.

I'm not super familiar with the distribution mechanisms used by BridgeDB (is that documented somewhere)?

Roger mentioned in a meeting a while ago that we have a reasonable number of university contacts we can ping to set up bridges. So when we are ready to move to this step, we should get into contact with them :)

comment:11 in reply to:  10 ; Changed 2 weeks ago by phw

Replying to cohosh:

I'm not super familiar with the distribution mechanisms used by BridgeDB (is that documented somewhere)?

I believe that three mechanisms are currently in place. You can get bridges over HTTPS (by visiting bridges.torproject.org), over email (by writing to bridges@…) and directly in Tor Browser (the system is called moat).

Roger mentioned in a meeting a while ago that we have a reasonable number of university contacts we can ping to set up bridges. So when we are ready to move to this step, we should get into contact with them :)

Sounds good!

comment:12 in reply to:  11 Changed 2 weeks ago by cohosh

Replying to phw:

Replying to cohosh:

I'm not super familiar with the distribution mechanisms used by BridgeDB (is that documented somewhere)?

I believe that three mechanisms are currently in place. You can get bridges over HTTPS (by visiting bridges.torproject.org), over email (by writing to bridges@…) and directly in Tor Browser (the system is called moat).

Ah, yep. It might also be a good idea to test any finer-grained sort of distribution we're doing (e.g., I think we're partitioning bridges and handing out some only to email addresses from certain providers). But this step can be even further down the road.

Changed 2 weeks ago by phw

Attachment: infer-throughput.py added

Python script to infer a download's throughput over time.

Changed 2 weeks ago by phw

Attachment: plot-throughput.R added

R script to visualise throughput over time.

comment:13 Changed 2 weeks ago by phw

Here's how the scripts I uploaded work:

  1. Adapt the variables CLIENT_TUPLE and SERVER_TUPLE in infer-throughput.py.
  2. Run the script: python infer-throughput.py download.pcap > download.csv
  3. Plot the results: Rscript plot-throughput.R download.csv

Changed 2 weeks ago by cohosh

Attachment: throughput_ca.pdf added

Throughput measurements from North American probe site as of 2019-04-09

Changed 2 weeks ago by cohosh

Attachment: throughput_cn.pdf added

Throughput measurements from Chinese probe site as of 2019-04-09

Changed 2 weeks ago by cohosh

obfs4 reachability results in China as of 2019-04-08

comment:14 Changed 2 weeks ago by cohosh

I've attached the throughput results from the probe site in China and the control probe site.

Unfortunately some of my packet captures were corrupted so I lost some data in the middle but this shows the throughput for the first and most recent day of testing.

It's worth noting that the file downloaded is located in Canada, so it will naturally take longer for it to download in China.

comment:15 Changed 2 weeks ago by cohosh

Ah, I also modified the scripts a bit and put them here

comment:16 Changed 13 days ago by cohosh

I think it's a good idea to move forward with expanding the obfs4 tests to try to pin down the bridge enumeration problem. Especially now that we're pretty sure probing alone doesn't get them blocked. These are the steps as I see them:

1) Set up the study design. Get more bridges (reach out to university contacts or perhaps probe all new bridges).

2) Have good documentation on how bridgeDB currently partitions bridges (e.g., are we giving different bridges to different IP addresses?)

3) Measure if/when bridges handed out by each method (website, email) and in each partition within the method get blocked.

comment:17 Changed 10 hours ago by gaba

Keywords: network-team-roadmap-2019-Q1Q2 removed
Note: See TracTickets for help on using tickets.