Opened 16 months ago

Closed 5 months ago

#30794 closed project (implemented)

Create lightweight censorship analyser for users

Reported by: phw Owned by: phw
Priority: Medium Milestone:
Component: Circumvention/Censorship analysis Version:
Severity: Normal Keywords: tbb-bridges, s30-o22a2
Cc: cohosh, gaba, dcf Actual Points: 0.8
Parent ID: #31279 Points: 5
Reviewer: cohosh Sponsor: Sponsor30-can

Description

Users occasionally show up on #tor and wonder why they are unable to connect to the network. We sometimes suspect censorship but it's often difficult to confirm this hypothesis. It would be useful to have a lightweight censorship analysis tool for users to run. Think of it as a small, specialised OONI: It should be a self-contained executable that tests if the user's computer can do the following:

  • Connect to the TCP port of our directory authorities.
  • Connect to the TCP port of a handful of relays.
  • Connect to the TCP port of our default bridges.
  • Resolve critical domains (e.g., bridges.tp.o) correctly.
  • Fetch the index page of critical websites (e.g., bridges.tp.o) over HTTPS.
  • Establish a TLS connection with a bridge authority and a relay.
  • ...

The output of the tool can be a simple text file that the user can then email to us, or paste in a chat window. We originally had this idea several years ago and documented it in a research paper but nobody every followed up. Such a tool could also be useful as part of an anti-censorship rapid response process.

If this sounds like a good idea, then I suggest that we build the tool in Go because 1) we have several talented Go hackers, 2) Go binaries are self-contained, and 3) since Go 1.5, cross-compiling for Windows seems relatively simple.

Child Tickets

Change History (10)

comment:1 Changed 15 months ago by ahf

Sponsor: Sponsor28-can

Removing sponsor.

comment:2 Changed 15 months ago by phw

I created a simple proof of concept: https://dip.torproject.org/anti-censorship/emma. I cross-compiled a .exe file by running:

GOOS=windows GOARCH=386 go build -o emma.exe

comment:3 Changed 14 months ago by dcf

I tried it at commit c8924dce26a28193b8478f72941d3b857a24c343 and got the following output:

Testing TCP port of default bridges.
        reachable: 109.105.109.163:38980
        reachable: 109.105.109.163:47779
        reachable: 109.105.109.165:10527
        reachable: 109.105.109.147:13764
        unreachable: 85.17.30.79:443 (dial tcp 85.17.30.79:443: connect: connection refused)
        reachable: 38.229.1.78:80
        reachable: 38.229.33.83:80
        unreachable: [2001:470:b381:bfff:216:3eff:fe23:d6c3]:443 (dial tcp [2001:470:b381:bfff:216:3eff:fe23:d6c3]:443: i/o timeout)
        reachable: 192.95.36.142:443
        reachable: 37.218.240.34:40035
        reachable: 37.218.245.14:38224
        reachable: 85.31.186.98:443
        reachable: 85.31.186.26:443
        reachable: 216.252.162.21:46089
        reachable: 144.217.20.138:80
Testing TCP port of directory authorities.
        reachable: 128.31.0.39:9131
        reachable: [2001:858:2:2:aabb:0:563b:1526]:443
        reachable: 86.59.21.38:80
        reachable: 194.109.206.212:80
        reachable: 66.111.2.131:9030
        reachable: [2001:638:a000:4140::ffff:189]:443
        reachable: 131.188.40.189:80
        reachable: [2001:678:558:1000::244]:443
        reachable: 193.23.244.244:80
        reachable: [2001:67c:289c::9]:80
        reachable: 171.25.193.9:443
        reachable: 154.35.175.225:80
        reachable: 199.58.81.140:80
        reachable: [2620:13:4000:6000::1000:118]:443
        reachable: 204.13.164.118:80
Testing *.torproject.org domains.
        Everything as expected.

Ideas:

  • If emma.exe is double-clicked on Windows, it may open a terminal window, display the output, then close the window again. Perhaps in the default configuration, it should append its output (timestamped) to a file.
  • For reachability tests, it would be good to show an elapsed time, in both the reachable and unreachable case.
  • For the DNS lookups, it would be nice to see what names were resolved. This is in case there are multiple versions of the program distributed, so we can tell from the output whether the domains included a particular one that we care about.

comment:4 Changed 5 months ago by phw

Keywords: s30-o22a2 added
Parent ID: #31279
Sponsor: Sponsor30-can

Marking this as Sponsor30-can because our tool helps with creating the censorship snapshot we're working on over at #28531. In fact, a volunteer in Iran already ran emma, revealing that 1) default bridges are reachable, 2) directory authorities are reachable, 3) *.torproject.org is blocked by DNS.

comment:5 in reply to:  3 Changed 5 months ago by phw

Reviewer: cohosh
Status: assignedneeds_review

Replying to dcf:

  • If emma.exe is double-clicked on Windows, it may open a terminal window, display the output, then close the window again. Perhaps in the default configuration, it should append its output (timestamped) to a file.
  • For reachability tests, it would be good to show an elapsed time, in both the reachable and unreachable case.
  • For the DNS lookups, it would be nice to see what names were resolved. This is in case there are multiple versions of the program distributed, so we can tell from the output whether the domains included a particular one that we care about.


These are great ideas, thanks! I addressed your feedback as follows:

  • By default, the tool now writes its report to a file unless given the -stdout flag.
  • Each test now shows its execution time.
  • I replaced DNS lookups with HTTPS requests.

Here's a run from my laptop:

$ ./bin/emma -stdout
2020/04/24 11:52:47 Starting analysis.
Time: 2020-04-24 11:52:47.933101623 -0700 PDT m=+0.000827916
Testing default bridges:
  ✓ 38.229.1.78:80                        71ms
  ✓ 38.229.33.83:80                       49ms
  ✓ 192.95.36.142:443                     74ms
  ✓ 37.218.240.34:40035                  166ms
  ✓ 37.218.245.14:38224                  147ms
  ✓ 85.31.186.98:443                     165ms
  ✓ 85.31.186.26:443                     166ms
  ✓ 144.217.20.138:80                     71ms
  ✓ 193.11.166.194:27015                 165ms
  ✓ 193.11.166.194:27020                 163ms
  ✓ 193.11.166.194:27025                 166ms
  ✓ 209.148.46.65:443                    107ms
  ✗ 146.57.248.225:22                       3s  error: dial tcp 146.57.248.225:22: i/o timeout
  ✓ 45.145.95.6:27015                    158ms
  ✓ [2a0c:4d80:42:702::1]:27015          154ms
Testing websites:
  ✓ https://gettor.torproject.org        896ms
  ✓ https://ajax.aspnetcdn.com           489ms
  ✓ https://bridges.torproject.org       521ms
  ✓ https://torproject.org              1.074s
Testing directory authorities.
  ✓ 128.31.0.39:9131                      75ms
  ✓ [2001:858:2:2:aabb:0:563b:1526]:443  183ms
  ✓ 86.59.21.38:80                       179ms
  ✓ 45.66.33.45:80                       160ms
  ✓ 66.111.2.131:9030                     67ms
  ✓ [2001:638:a000:4140::ffff:189]:443   158ms
  ✓ 131.188.40.189:80                    162ms
  ✓ [2001:678:558:1000::244]:443         157ms
  ✓ 193.23.244.244:80                    157ms
  ✓ [2001:67c:289c::9]:80                161ms
  ✓ 171.25.193.9:443                     173ms
  ✓ 154.35.175.225:80                     49ms
  ✓ 199.58.81.140:80                      80ms
  ✓ [2620:13:4000:6000::1000:118]:443     39ms
  ✓ 204.13.164.118:80                     37ms
Testing relays.
  ✓ 193.11.166.196:443                   167ms
  ✓ 81.7.18.7:9001                       164ms
  ✓ 91.143.80.147:995                    163ms
  ✗ 162.247.74.7:443                        3s  error: dial tcp 162.247.74.7:443: i/o timeout
  ✓ 62.102.148.68:443                    163ms
  ✓ 185.220.100.253:9000                 166ms
2020/04/24 11:53:01 Wrote output to: /dev/stdout

I'm setting this ticket to needs_review because I would like to hear cohosh's thoughts on this.

comment:6 Changed 5 months ago by cohosh

This looks good!

  • As stated on #28531, it would be also nice to test our GetTor providers and distributors.

comment:7 Changed 5 months ago by cohosh

Status: needs_reviewneeds_information

comment:8 in reply to:  6 Changed 5 months ago by phw

Status: needs_informationneeds_review

Replying to cohosh:

  • As stated on #28531, it would be also nice to test our GetTor providers and distributors.


Yes, that's a great idea. Addressed in ca7508dc. (I also made the tool truncate URLs.)


Addressed in 2c5a799f.

comment:9 Changed 5 months ago by cohosh

Status: needs_reviewmerge_ready

These changes look good to me :)

comment:10 Changed 5 months ago by phw

Actual Points: 0.8
Resolution: implemented
Status: merge_readyclosed

Thanks for the review! We now have a minimum viable product, so I'm closing this ticket.

Note: See TracTickets for help on using tickets.