Opened 8 years ago

Closed 8 years ago

Last modified 7 years ago

#5028 closed project (implemented)

Tor bridge scanning

Reported by: hellais Owned by: runa
Priority: Medium Milestone:
Component: Archived/Ooni Version:
Severity: Keywords: SponsorF20120315
Cc: Runa, Karsten, Arma, gsathya.ceg@…, aagbsn Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We need to scan the reachability of Tor Bridges from parts of the world. This is geared at detecting what bridges are working correctly and which are censored from where.

This is what the ooni-probe test BridgeT does.

The main question this tool wants to answer is:
"Amongst this set of bridges which ones are working from where?"

the sub-question is:
"The not working ones are not working because they are censored?"

The data that should come out of BridgeT are:

  • Timestamp of the scan
  • IP address of the scanner
  • IP address of the scanned bridge
  • Status: working | not working | censored
  • Some debug information on what went wrong

The debug information should be in a first stage the output of Tors log for loglevel
notice, info or debug. In future this may be expanded to include some other information
(what? packet captures?).

This will be deployed in a first stage on Planet labs servers. In future it will be one of
the default ooni-probe tests and will run from wherever ooni-probe is running from.

Another worth considering question is how BridgeT learns about the bridges to test. If we want this to scale well we should figure out a mechanism for making BridgeT learn about new bridges to scan.

In the first phase the list is just a static text file.

Somebody should test out the current BridgeT code to make sure that it is working correctly.

Child Tickets

TicketStatusOwnerSummaryComponent
#5207closedhellaisooni-probe should only set UseMicrodescriptors 0 if Tor understands that optionArchived/Ooni
#5209closedhellaisooni-probe enters some endless loop after scanning a few hundred bridgesArchived/Ooni
#5229closedhellaisooni-probe/bridget should make sure that random port numbers are not already takenArchived/Ooni

Attachments (2)

sanitized-logs-2012-03-15.csv (4.6 KB) - added by karsten 8 years ago.
bridge-scan-eval-2012-03-15.png (21.9 KB) - added by karsten 8 years ago.

Download all attachments as: .zip

Change History (55)

comment:1 Changed 8 years ago by runa

Owner: set to runa
Status: newassigned

comment:2 Changed 8 years ago by karsten

#4075 was a duplicate of this ticket.

comment:3 Changed 8 years ago by runa

Component: Tor BridgeOoni

comment:4 in reply to:  description ; Changed 8 years ago by runa

Replying to hellais:

The debug information should be in a first stage the output of Tors log for loglevel
notice, info or debug. In future this may be expanded to include some other information
(what? packet captures?).

Packet captures would be fun to have. I believe this is on the ooni-probe road map already.

This will be deployed in a first stage on Planet labs servers. In future it will be one of
the default ooni-probe tests and will run from wherever ooni-probe is running from.

So they are happy with us using their servers for this kind of project? What kind of information do you need (either from me or someone else) to get this started?

Somebody should test out the current BridgeT code to make sure that it is working correctly.

Tested and found to be working. I wonder if it gives a false positive every now and then, though. Maybe gsathya can comment on that?

comment:5 in reply to:  4 ; Changed 8 years ago by gsathya

Cc: gsathya.ceg@… added

Tested and found to be working. I wonder if it gives a false positive every now and then, though. Maybe gsathya can comment on that?

AFAIK It gives false negatives not positives. We could correct that by reducing the number of concurrent tor processes.

comment:6 in reply to:  5 ; Changed 8 years ago by runa

Replying to gsathya:

Tested and found to be working. I wonder if it gives a false positive every now and then, though. Maybe gsathya can comment on that?

AFAIK It gives false negatives not positives. We could correct that by reducing the number of concurrent tor processes.

Do you know why it gives false negatives? Would you mind opening another ticket about that issue and setting the component to Ooni?

comment:7 in reply to:  6 Changed 8 years ago by gsathya

Do you know why it gives false negatives? Would you mind opening another ticket about that issue and setting the component to Ooni?

#5167

comment:8 Changed 8 years ago by karsten

I completed a test run on a VM located in Germany with the bridges from 2012-02-20 16:37 that ran from 2012-02-25 16:06 to 19:01:

    477 Working: false
    643 Working: true

A large part of 477 non-working bridges probably comes from the fact that input bridges were already a few days old when starting the scan. #5209 has more details about the problems encountered when running scans.

comment:9 Changed 8 years ago by hellais

In commit 0bfffb1952869b02076fe21641246834ab1b2976 I added support for debug information on what went wrong.

It basically stores the output of Tor info log when a bridge is not working.

Is there anything more that needs to be done for this ticket? Or for this deliverable in general?

comment:10 in reply to:  9 ; Changed 8 years ago by runa

Replying to hellais:

Is there anything more that needs to be done for this ticket? Or for this deliverable in general?

I believe we need to run the scan on the same set of bridge addresses from different countries, including China. How should we organize that? Do we want to run the scans at roughly the same time as well?

comment:11 in reply to:  10 Changed 8 years ago by hellais

Replying to runa:

I believe we need to run the scan on the same set of bridge addresses from different countries, including China. How should we organize that? Do we want to run the scans at roughly the same time as well?

Are we sure this is a good idea?
Wouldn't we basically be giving all the bridge addresses to China in this way?

comment:12 Changed 8 years ago by runa

Seems like we need to come up with a plan for: (1) what kind of bridge addresses to scan (e.g. a subset of the bridges we give out on bridges.tpo), (2) where to scan them from, (3) what to do with the results. Karsten and/or Roger, do you have any suggestions?

comment:13 Changed 8 years ago by karsten

I don't have suggestions here, mostly because I don't know enough about not giving away bridge addresses while scanning bridges. I just remember that there were plans to "ping" bridges via TCP from China to see if they're listening on their OR port. But I don't know if that's still a good idea.

Runa, you're in charge of this deliverable, so please decide something. Feel free to poke Roger to see if he has any input.

Just keep in mind that this deliverable is due in 10 days. We have completed a scan of all bridges from Germany to confirm that ooni-probe/bridget works correctly. But we don't have "some sort of automated ground truth of bridge reachability from some countries" as we promised in the deliverable.

I'm currently not doing anything for this deliverable, so if you want me to do something, please let me know.

comment:14 Changed 8 years ago by arma

Doing a full Tor connection to a bridge from inside China is a poor idea. Doing a TCP scan to the bridge address is probably fine, and I think it would tell us everything we need to know at this point.

If we wanted to be extra cautious we could do the scan on all the bridges in the https bucket, since they're more likely to be known already. I think we don't have a script to pull out just those bridges from the bridgedb database; but that's 'a simple matter of programming' as they say.

But yes, Karsten's point about "automated" is an important one: we should script a periodic export of a list of bridges to scan, cron a periodic scan, and add each set of scan results to some long-term storage in a way that is easy to do analysis on.

comment:15 in reply to:  14 ; Changed 8 years ago by runa

Cc: aagbsn added

Replying to arma:

Doing a full Tor connection to a bridge from inside China is a poor idea. Doing a TCP scan to the bridge address is probably fine, and I think it would tell us everything we need to know at this point.

Ok, we'll do that.

If we wanted to be extra cautious we could do the scan on all the bridges in the https bucket, since they're more likely to be known already. I think we don't have a script to pull out just those bridges from the bridgedb database; but that's 'a simple matter of programming' as they say.

Who can do this? Aaron? Would be great if Arturo, Karsten, and I could have an email with the bridge addresses in it.

But yes, Karsten's point about "automated" is an important one: we should script a periodic export of a list of bridges to scan, cron a periodic scan, and add each set of scan results to some long-term storage in a way that is easy to do analysis on.

Yes, we should, but that's something we can focus on after March 15th.

comment:16 in reply to:  15 ; Changed 8 years ago by arma

Replying to runa:

If we wanted to be extra cautious we could do the scan on all the bridges in the https bucket, since they're more likely to be known already. I think we don't have a script to pull out just those bridges from the bridgedb database; but that's 'a simple matter of programming' as they say.

Who can do this? Aaron? Would be great if Arturo, Karsten, and I could have an email with the bridge addresses in it.

Aaron: I had in mind some variant of the incants in kaner's filterBridges() and related functions in bin/bridge2bucket.py. Speaking of which, that file should probably find its way into a git repository rsn.

Runa: no need to block on any of the other steps while waiting for an automated bridge list export. You can for example make up some text file and pretend it's the one that is emailed to you.

comment:17 in reply to:  16 Changed 8 years ago by runa

Replying to arma:

Runa: no need to block on any of the other steps while waiting for an automated bridge list export. You can for example make up some text file and pretend it's the one that is emailed to you.

Not blocking on anything, I just want to make sure we get this can done for Sponsor F before the deadline - before focusing on other things.

comment:18 in reply to:  16 Changed 8 years ago by karsten

Replying to arma:

Replying to runa:

If we wanted to be extra cautious we could do the scan on all the bridges in the https bucket, since they're more likely to be known already. I think we don't have a script to pull out just those bridges from the bridgedb database; but that's 'a simple matter of programming' as they say.

Who can do this? Aaron? Would be great if Arturo, Karsten, and I could have an email with the bridge addresses in it.

Aaron: I had in mind some variant of the incants in kaner's filterBridges() and related functions in bin/bridge2bucket.py. Speaking of which, that file should probably find its way into a git repository rsn.

I can do this quite easily, so that we're not blocking on Aaron to mess with BridgeDB. I just sent the list of HTTPS bridges to Runa. I can update that list easily.

comment:19 in reply to:  15 ; Changed 8 years ago by karsten

Replying to runa:

Replying to arma:

Doing a full Tor connection to a bridge from inside China is a poor idea. Doing a TCP scan to the bridge address is probably fine, and I think it would tell us everything we need to know at this point.

Ok, we'll do that.

The last time I asked this type of scan wasn't implemented. Has this changed?

comment:20 in reply to:  19 ; Changed 8 years ago by runa

Replying to karsten:

Replying to runa:

Replying to arma:

Doing a full Tor connection to a bridge from inside China is a poor idea. Doing a TCP scan to the bridge address is probably fine, and I think it would tell us everything we need to know at this point.

Ok, we'll do that.

The last time I asked this type of scan wasn't implemented. Has this changed?

Yes, I created a plugin called tcpcon in November/December last year.

comment:21 Changed 8 years ago by asn

A TCP scan on a bridge from .kz would return a false positive, since the bridge actually gets blocked real-time on the SSL layer.

An SSL scan on a bridge from .cn would return a false positive, since the bridge would get active-scanned and blocked afterwards.

From discussing this with hellais the past few days, I think this ticket needs some more thought poured into it. Specifically, I think that the purpose of this project, the scanning method, and the subset of bridges to-be scanned could be specified better, before scanning all the bridges from a single computer in a censored country.

Also, scanning of unpublished bridges should be of interest too.
Also, I think that one of the main questions of this project should be "What common characteristics do the blocked bridges share, and why are the non-blocked bridges non-blocked?"

PS: sorry for raiding the ticket

comment:22 in reply to:  21 Changed 8 years ago by karsten

Replying to asn:

A TCP scan on a bridge from .kz would return a false positive, since the bridge actually gets blocked real-time on the SSL layer.

Let's not scan from .kz then.

An SSL scan on a bridge from .cn would return a false positive, since the bridge would get active-scanned and blocked afterwards.

But a TCP scan from .cn should work fine, right?

From discussing this with hellais the past few days, I think this ticket needs some more thought poured into it. Specifically, I think that the purpose of this project, the scanning method, and the subset of bridges to-be scanned could be specified better, before scanning all the bridges from a single computer in a censored country.

More thinking sounds fine. We have 6 days---for more thinking and for getting actual results. My intuition is that we should be done with thinking about this project by now. Heck, we should be done doing stuff and only writing down results by now.

Also, scanning of unpublished bridges should be of interest too.

We could scan bridges in the reserved pool, which are sorta unpublished. Let's save that for after March 15 though.

Also, I think that one of the main questions of this project should be "What common characteristics do the blocked bridges share, and why are the non-blocked bridges non-blocked?"

Sure. That's part of the analysis that we should do once we have actual results.

PS: sorry for raiding the ticket

No worries. It was a feeble attempt anyway. ;)

comment:23 in reply to:  20 ; Changed 8 years ago by karsten

Replying to runa:

Yes, I created a plugin called tcpcon in November/December last year.

Is this tested? Bridget was also written and said to be working two weeks ago, and then we took almost a week to debug it.

When are we going to start scanning with tcpcon? Who's doing that?

comment:24 in reply to:  23 ; Changed 8 years ago by runa

Replying to karsten:

Replying to runa:

Yes, I created a plugin called tcpcon in November/December last year.

Is this tested? Bridget was also written and said to be working two weeks ago, and then we took almost a week to debug it.

I tested it with a small set of bridges, but can't promise that you won't run into problems with a larger set.

When are we going to start scanning with tcpcon? Who's doing that?

Arturo is the one with access to PlanetLab, so he will have to run the scan.

comment:25 in reply to:  24 ; Changed 8 years ago by hellais

Replying to runa:

Arturo is the one with access to PlanetLab, so he will have to run the scan.

I don't feel like taking the responsibility of running these scans, collecting the results and producing the deliverable for this task. Especially since I was assigned to it now at the last minute without any preemptive warning.

If you email Ian he will be able to set you up with a PlanetLab account.

comment:26 Changed 8 years ago by ioerror

I suggest that you print out a copy of every ip and simply call the Chinese embassy to find out if these are blocked.

comment:27 Changed 8 years ago by phw

I could provide a host inside China for the chinese bridge scan. A TCP scan from inside China should be enough to detect blocked bridges. I don't know the details about ooni but if it's just about testing the availability of bridges, a simple shell script with netcat should be enough?

comment:28 in reply to:  26 Changed 8 years ago by rransom

Replying to ioerror:

I suggest that you print out a copy of every ip and simply call the Chinese embassy to find out if these are blocked.

That's actually a good idea! That way, They won't be able to record the IP addresses as easily for later blocking.

comment:29 in reply to:  26 ; Changed 8 years ago by karsten

Replying to ioerror:

I suggest that you print out a copy of every ip and simply call the Chinese embassy to find out if these are blocked.

Was that only supposed to be funny, or is there an actual suggestion for doing things differently that you want to tell us?

comment:30 in reply to:  25 ; Changed 8 years ago by karsten

Replying to hellais:

Replying to runa:

Arturo is the one with access to PlanetLab, so he will have to run the scan.

I don't feel like taking the responsibility of running these scans, collecting the results and producing the deliverable for this task. Especially since I was assigned to it now at the last minute without any preemptive warning.

If you email Ian he will be able to set you up with a PlanetLab account.

It seems nobody is happy with the current situation. I can understand that. I know it's hard, but can we postpone the discussion who's responsible for all this for a few more days?

How can we solve the problem that we need to run these scans today or tomorrow, ideally from China and Germany?

Arturo, can you help me set up tcpcon on my VM and prepare the same setup on a PlanetLab host in China? Maybe today? I'm around on IRC.

I'll analyze the results and write a tech report, ideally with Arturo's help.

comment:31 in reply to:  30 ; Changed 8 years ago by hellais

Replying to karsten:

It seems nobody is happy with the current situation. I can understand that. I know it's hard, but can we postpone the discussion who's responsible for all this for a few more days?

How can we solve the problem that we need to run these scans today or tomorrow, ideally from China and Germany?

Arturo, can you help me set up tcpcon on my VM and prepare the same setup on a PlanetLab host in China? Maybe today? I'm around on IRC.

I'll analyze the results and write a tech report, ideally with Arturo's help.

Although I don't believe that scanning these bridges from inside China is a good idea, at least without putting some more thought to it I wrote a TCP Connect scanner.
The tcpcon written by runa was for the old version of ooni-probe and I thought it would be faster to rewrite it rather than port the old code.

You can find it here:
https://gitweb.torproject.org/ooni-probe.git/commit/81b5de65d178d08ac1a01c7e84f8bde397e60c03

To run it you need to edit assets/tcpscan.txt to include the list of bridges in <IP>:<PORT> format and then run

$ ./ooniprobe.py -r tcpscan

comment:32 in reply to:  29 ; Changed 8 years ago by ioerror

Replying to karsten:

Replying to ioerror:

I suggest that you print out a copy of every ip and simply call the Chinese embassy to find out if these are blocked.

Was that only supposed to be funny, or is there an actual suggestion for doing things differently that you want to tell us?

The deliverable is being driven by a sponsorship item. However, the circumstances have since changed - active probing in China means that blocking happens in a different, additive, set of ways. Some IPs are on a blocklist, some are added by their behavior; thus any scan to a real bridge with a tcp connection will merely tell us that the bridge is not on the block list but any attempt to use it will almost certainly result in an active probe that in turn will probably block the bridge. Any result from the TCP connect scan will be either 0) possibly confirmation that the IP is blocked 1) a false negative where we believe the bridge is unblocked or 2) we will cause the bridge to be discovered and then actually blocked.

So why risk it? Because a funder has a line item? It seems like we should be smarter than that and not be so hung up on line items that we created before the environment changed.

Thus, my point was to be both humorous and also to be blunt - doing a scan of bridges may simply result in those bridges being instantly blocked or just as likely, I think the data will be inconclusive.

comment:33 in reply to:  32 ; Changed 8 years ago by karsten

Replying to ioerror:

The deliverable is being driven by a sponsorship item. However, the circumstances have since changed - active probing in China means that blocking happens in a different, additive, set of ways. Some IPs are on a blocklist, some are added by their behavior; thus any scan to a real bridge with a tcp connection will merely tell us that the bridge is not on the block list but any attempt to use it will almost certainly result in an active probe that in turn will probably block the bridge. Any result from the TCP connect scan will be either 0) possibly confirmation that the IP is blocked 1) a false negative where we believe the bridge is unblocked or 2) we will cause the bridge to be discovered and then actually blocked.

So why risk it? Because a funder has a line item? It seems like we should be smarter than that and not be so hung up on line items that we created before the environment changed.

Thus, my point was to be both humorous and also to be blunt - doing a scan of bridges may simply result in those bridges being instantly blocked or just as likely, I think the data will be inconclusive.

I think I understand your concerns. But that doesn't mean it's impossible to obtain "some sort of automated ground truth of bridge reachability from some countries" which is what we promised in the deliverable. The TCP scan of HTTPS bridges may not be the best approach, but it's the best we have right now. At least so far I only heard "oh noes, don't do it," not "here's a better way to deliver what we promised, and we can do it within 3 days." Until I hear the latter I'll stick with the approach we have. I don't know if the results will be conclusive, but I sure want to find out.

comment:34 in reply to:  31 Changed 8 years ago by karsten

Replying to hellais:

Although I don't believe that scanning these bridges from inside China is a good idea, at least without putting some more thought to it [...]

Understood.

[...] I wrote a TCP Connect scanner.

Thanks for writing the scanner. I ran it on my VM, and it worked just fine.

comment:35 Changed 8 years ago by karsten

I ran a scan on my host in .de this morning. I scanned 313 bridges in the HTTPS pool that were contained in the bridge pool assignments file from March 13, 06:30:18 UTC and in the network status published at 06:37:04 UTC. The scan ran from 07:19:30 to 07:23:44 UTC and resulted in 300 bridges being reachable and 13 bridges being unreachable:

  • 299 bridges that were found as reachable were contained in both network statuses published at 06:37:04 and 07:37:04 UTC. Scan results are correct.
  • 1 bridge that was found as reachable was only contained in the network status published at 06:37:04 UTC and not in the one published at 07:37:04 UTC. That means that Tonga could not reach this bridge, but the scanner could. This is an actual false positive.
  • 9 bridges that were found as unreachable had the Running flag in the assignments file from 06:30:18 UTC and in the network status from 06:37:04 UTC, but lost it in the network status published at 07:37:04 UTC. These negative scan results are correct.
  • 2 bridges that were found as unreachable were contained in the assignments file from 06:30:18 UTC but did not have the Running flag in the network statuses published at 06:37:04 and 07:37:04 UTC. These bridges went offline between 6:30 and 6:37 and should not have been scanned at all. Can't blame the scanner for that.
  • 2 bridges that were found as unreachable were contained in the assignments file and all both network statuses, and they were reachable in a subsequent scan at 09:00:55 UTC. These 2 are actual false negatives.

So, 1 false positive and 2 false negatives. Should be fine.

The next step will be to run a similar scan from .cn.

comment:36 in reply to:  33 ; Changed 8 years ago by ioerror

Replying to karsten:

Replying to ioerror:

The deliverable is being driven by a sponsorship item. However, the circumstances have since changed - active probing in China means that blocking happens in a different, additive, set of ways. Some IPs are on a blocklist, some are added by their behavior; thus any scan to a real bridge with a tcp connection will merely tell us that the bridge is not on the block list but any attempt to use it will almost certainly result in an active probe that in turn will probably block the bridge. Any result from the TCP connect scan will be either 0) possibly confirmation that the IP is blocked 1) a false negative where we believe the bridge is unblocked or 2) we will cause the bridge to be discovered and then actually blocked.

So why risk it? Because a funder has a line item? It seems like we should be smarter than that and not be so hung up on line items that we created before the environment changed.

Thus, my point was to be both humorous and also to be blunt - doing a scan of bridges may simply result in those bridges being instantly blocked or just as likely, I think the data will be inconclusive.

I think I understand your concerns. But that doesn't mean it's impossible to obtain "some sort of automated ground truth of bridge reachability from some countries" which is what we promised in the deliverable.

We already have that ground truth, don't we?

Obfu bridges are generally reachable, tls bridges are generally blocked either before we test or by confirmation with a follow up probe. Has that changed? Are we doing a scan of the obfu bridges? Or just the normal HTTPS/TLS bridges?

The TCP scan of HTTPS bridges may not be the best approach, but it's the best we have right now. At least so far I only heard "oh noes, don't do it," not "here's a better way to deliver what we promised, and we can do it within 3 days." Until I hear the latter I'll stick with the approach we have. I don't know if the results will be conclusive, but I sure want to find out.

My suggestion is to deliver the news that we know without impacting the resources which are scarce. The fact that the ground truth is now "active probing" is really quite a thing! If that is indeed still happening, of course.

comment:37 in reply to:  36 ; Changed 8 years ago by hellais

Replying to ioerror:

Replying to karsten:

I think I understand your concerns. But that doesn't mean it's impossible to obtain "some sort of automated ground truth of bridge reachability from some countries" which is what we promised in the deliverable.

We already have that ground truth, don't we?

Obfu bridges are generally reachable, tls bridges are generally blocked either before we test or by confirmation with a follow up probe. Has that changed? Are we doing a scan of the obfu bridges? Or just the normal HTTPS/TLS bridges?

I agree with what Jake is saying. To put it in another way, what do we plan on discovering by doing this scan that we don't already know?
Is this discovery worth possibly risking the disclosure of Tor bridge addresses to the adversary?

The TCP scan of HTTPS bridges may not be the best approach, but it's the best we have right now. At least so far I only heard "oh noes, don't do it," not "here's a better way to deliver what we promised, and we can do it within 3 days." Until I hear the latter I'll stick with the approach we have. I don't know if the results will be conclusive, but I sure want to find out.

My suggestion is to deliver the news that we know without impacting the resources which are scarce. The fact that the ground truth is now "active probing" is really quite a thing! If that is indeed still happening, of course.

Maybe we can do some extra research on the active probing being done by GFW. There is already a lot of information that we have
collected with respect to this in the past and we could get more.

comment:38 in reply to:  36 ; Changed 8 years ago by karsten

Replying to ioerror:

Replying to karsten:

I think I understand your concerns. But that doesn't mean it's impossible to obtain "some sort of automated ground truth of bridge reachability from some countries" which is what we promised in the deliverable.

We already have that ground truth, don't we?

We seem to have some knowledge about how .cn blocks Tor. But we're looking for an automated way to detect whether bridges are reachable from any given country. If some other country starts blocking bridges tomorrow we'll want to have an automated way to detect that.

I ran a scan from .de yesterday and can say that we have very few false positives and false negatives in finding which bridges are reachable. No big surprise, but useful to confirm that the scanner is working correctly.

Runa is going to run the same scan from .cn today, twice with a delay of one hour. Will all bridges be unreachable? Just the ones that are new and were not used from .cn yet? Will the ones which were working in the first scan be blocked in the second scan? How will the reported bridge statistics tomorrow differ from the statistics today? That's stuff we hope to learn, and we can't just guess what will happen.

Maybe we should run a third scan from another country. How about .ru? Do we know if bridges are blocked from .ru? And do we know whether a simple TCP scan with a timeout of 20 seconds will give us reliable information which bridges are reachable?

Obfu bridges are generally reachable, tls bridges are generally blocked either before we test or by confirmation with a follow up probe. Has that changed?

I wouldn't know.

Are we doing a scan of the obfu bridges? Or just the normal HTTPS/TLS bridges?

We're scanning normal HTTPS/TLS bridges, no obfsproxy bridges.

The TCP scan of HTTPS bridges may not be the best approach, but it's the best we have right now. At least so far I only heard "oh noes, don't do it," not "here's a better way to deliver what we promised, and we can do it within 3 days." Until I hear the latter I'll stick with the approach we have. I don't know if the results will be conclusive, but I sure want to find out.

My suggestion is to deliver the news that we know without impacting the resources which are scarce. The fact that the ground truth is now "active probing" is really quite a thing! If that is indeed still happening, of course.

I think that's great to know, but it's not what we hope to learn in this deliverable.

comment:39 in reply to:  37 Changed 8 years ago by karsten

Replying to hellais:

I agree with what Jake is saying. To put it in another way, what do we plan on discovering by doing this scan that we don't already know?

I think I addresses that question in my reply to Jake's comment above.

Is this discovery worth possibly risking the disclosure of Tor bridge addresses to the adversary?

There are not just risks, but also potential benefits. If we have good results from active scans we might be able to infer reachability information from passive data, e.g., by looking at bridge statistics. And once we have a reliable way to say which bridges are blocked in a country we can stop giving out blocked bridges to users in that country.

Maybe we can do some extra research on the active probing being done by GFW. There is already a lot of information that we have collected with respect to this in the past and we could get more.

That's not what the deliverable is about. But this is already addresses in my response to Jake, too.

comment:40 in reply to:  38 ; Changed 8 years ago by ioerror

Replying to karsten:

Replying to ioerror:

Replying to karsten:

I think I understand your concerns. But that doesn't mean it's impossible to obtain "some sort of automated ground truth of bridge reachability from some countries" which is what we promised in the deliverable.

We already have that ground truth, don't we?

We seem to have some knowledge about how .cn blocks Tor. But we're looking for an automated way to detect whether bridges are reachable from any given country. If some other country starts blocking bridges tomorrow we'll want to have an automated way to detect that.

It seems like such an automated scan should contain special cases for countries with network censorship systems that are as advanced as China.

I ran a scan from .de yesterday and can say that we have very few false positives and false negatives in finding which bridges are reachable. No big surprise, but useful to confirm that the scanner is working correctly.

I'd expect that things would run rather smoothly from Germany, yes.

Runa is going to run the same scan from .cn today, twice with a delay of one hour. Will all bridges be unreachable? Just the ones that are new and were not used from .cn yet? Will the ones which were working in the first scan be blocked in the second scan? How will the reported bridge statistics tomorrow differ from the statistics today? That's stuff we hope to learn, and we can't just guess what will happen.

For a TCP connection scan or an actual bridge handshake, I predict we'll have totally different results for those above questions. If it's just a TCP connection, I'm not convinced the data will be the full picture, if it's the full handshake, I suspect it will result in blocking. We should be aware that such a scan may result in blocking and only use a pool of addresses we're OK with losing, probably where there is no intersection with obfu bridges.

Maybe we should run a third scan from another country. How about .ru? Do we know if bridges are blocked from .ru?

I think the Perfect Privacy VPN service has some servers in Russia and many others. I don't think Russia has blocked any bridges though.

And do we know whether a simple TCP scan with a timeout of 20 seconds will give us reliable information which bridges are reachable?

I'm not convinced that this is actually a useful test because of the different kinds of blocking at play. In Syria we found Bluecoat devices that would tear down a TCP connection only if a Tor client and a Tor bridge were speaking TLS; normal OpenSSL s_client did not have the same the same distinguishers with the same Tor bridge. If we're not doing a real TLS handshake, I think we only learn about IP blocking but not about actual Tor bridge reachability. That's useful info too but not actually giving us real data on bridges being reachable when a network is dynamically censoring by protocol, rather than IP.

Obviously as arma points out, a full Tor handshake in China might not be the best idea, still, I bet we learn more ground truth with a real test and not a simulated test, given what we know about China these days.

Obfu bridges are generally reachable, tls bridges are generally blocked either before we test or by confirmation with a follow up probe. Has that changed?

I wouldn't know.

This is a test that seems extremely relevant and if we're going to do scans from China, I bet it's one of the safest and most useful.

Are we doing a scan of the obfu bridges? Or just the normal HTTPS/TLS bridges?

We're scanning normal HTTPS/TLS bridges, no obfsproxy bridges.

Ok. Are we scanning from all buckets or just a single bucket?

The TCP scan of HTTPS bridges may not be the best approach, but it's the best we have right now. At least so far I only heard "oh noes, don't do it," not "here's a better way to deliver what we promised, and we can do it within 3 days." Until I hear the latter I'll stick with the approach we have. I don't know if the results will be conclusive, but I sure want to find out.

My suggestion is to deliver the news that we know without impacting the resources which are scarce. The fact that the ground truth is now "active probing" is really quite a thing! If that is indeed still happening, of course.

I think that's great to know, but it's not what we hope to learn in this deliverable.

Ok.

The ground truth for China is different from other countries these days. Any automated tool to do this kind of scanning should not have an assumption that IP or TCP connect scans and a full handshake with data will produce the same results.

comment:41 in reply to:  40 ; Changed 8 years ago by karsten

Replying to ioerror:

It seems like such an automated scan should contain special cases for countries with network censorship systems that are as advanced as China.

Sure. In the long run we could do full Tor handshakes for most countries and just the TCP scan for China. But we'll have to start somewhere, and the TCP scan seems to be the safest approach.

Obfu bridges are generally reachable, tls bridges are generally blocked either before we test or by confirmation with a follow up probe. Has that changed?

I wouldn't know.

This is a test that seems extremely relevant and if we're going to do scans from China, I bet it's one of the safest and most useful.

If we only scan obfsproxy bridges, we'll only learn if those are reachable. But we want to have a mechanism for telling if normal bridges are reachable as well. Excluding obfsproxy bridges for the moment.

Are we doing a scan of the obfu bridges? Or just the normal HTTPS/TLS bridges?

We're scanning normal HTTPS/TLS bridges, no obfsproxy bridges.

Ok. Are we scanning from all buckets or just a single bucket?

The plan was to scan all HTTPS bridges, but I guess we can reduce that to 2 out of 5 rings. 125 bridges in total.

The ground truth for China is different from other countries these days. Any automated tool to do this kind of scanning should not have an assumption that IP or TCP connect scans and a full handshake with data will produce the same results.

Right, that's an assumption that needs to be made explicit. Want to help write the report, so that these things come over correctly?

comment:42 in reply to:  41 ; Changed 8 years ago by ioerror

Replying to karsten:

Replying to ioerror:

It seems like such an automated scan should contain special cases for countries with network censorship systems that are as advanced as China.

Sure. In the long run we could do full Tor handshakes for most countries and just the TCP scan for China. But we'll have to start somewhere, and the TCP scan seems to be the safest approach.

Hrm. I think that full Tor handshakes are best all around - the question is more about how we transport the handshake - that is to say that a full obfu bridge test is the useful test for China today, not TCP or TLS - we already know the latter two fail almost all of the time.

I do agree that a TCP scan is safer for China than a full handshake but I'm not convinced it is terribly useful for China for that very same reason. I do think one useful outcome of the TCP scan is that we might learn where other people have triggered an active probe that resulted in an IP block. I wonder though, do we know for sure that active scans put those IP addresses into a firewall forever? Or have active scans replaced that static block list?

Obfu bridges are generally reachable, tls bridges are generally blocked either before we test or by confirmation with a follow up probe. Has that changed?

I wouldn't know.

This is a test that seems extremely relevant and if we're going to do scans from China, I bet it's one of the safest and most useful.

If we only scan obfsproxy bridges, we'll only learn if those are reachable. But we want to have a mechanism for telling if normal bridges are reachable as well. Excluding obfsproxy bridges for the moment.

Ok. I think that if we exclude those, we'll have a set of bridges that have likely been harvested and blocked or just blocked by active probing. It's not clear how we'll know the difference with a TCP connection scan.

Are we doing a scan of the obfu bridges? Or just the normal HTTPS/TLS bridges?

We're scanning normal HTTPS/TLS bridges, no obfsproxy bridges.

Ok. Are we scanning from all buckets or just a single bucket?

The plan was to scan all HTTPS bridges, but I guess we can reduce that to 2 out of 5 rings. 125 bridges in total.

I think the lower the number, the better a chance we'll get at refining these tests over a few hours - if we lose them all in on shot, we're sorta out of luck until they churn.

The ground truth for China is different from other countries these days. Any automated tool to do this kind of scanning should not have an assumption that IP or TCP connect scans and a full handshake with data will produce the same results.

Right, that's an assumption that needs to be made explicit. Want to help write the report, so that these things come over correctly?

I'm happy to look at the data but I don't know what such a report entails, I assume that's a different deliverable?

comment:43 in reply to:  42 ; Changed 8 years ago by karsten

Replying to ioerror:

Hrm. I think that full Tor handshakes are best all around - the question is more about how we transport the handshake - that is to say that a full obfu bridge test is the useful test for China today, not TCP or TLS - we already know the latter two fail almost all of the time.

Testing a normal bridge and testing an obfsproxy bridge are two different things. And we don't know if the TCP test will fail for normal bridges almost all of the time.

I do agree that a TCP scan is safer for China than a full handshake but I'm not convinced it is terribly useful for China for that very same reason. I do think one useful outcome of the TCP scan is that we might learn where other people have triggered an active probe that resulted in an IP block. I wonder though, do we know for sure that active scans put those IP addresses into a firewall forever? Or have active scans replaced that static block list?

I don't know.

Ok. I think that if we exclude those, we'll have a set of bridges that have likely been harvested and blocked or just blocked by active probing. It's not clear how we'll know the difference with a TCP connection scan.

It's a start. It's not supposed to be perfect.

The plan was to scan all HTTPS bridges, but I guess we can reduce that to 2 out of 5 rings. 125 bridges in total.

I think the lower the number, the better a chance we'll get at refining these tests over a few hours - if we lose them all in on shot, we're sorta out of luck until they churn.

I don't expect us to have time for another test after this one, so these 125 bridges will be it.

Right, that's an assumption that needs to be made explicit. Want to help write the report, so that these things come over correctly?

I'm happy to look at the data but I don't know what such a report entails, I assume that's a different deliverable?

Ah, the "report" would simply summarize what we found in this deliverable, so that the sponsor doesn't have to read lengthy Trac tickets. I'd simply start a LaTeX article document, but it could also be a blog post. Something we can finish by tomorrow.

comment:44 in reply to:  43 Changed 8 years ago by ioerror

Replying to karsten:

Replying to ioerror:

Hrm. I think that full Tor handshakes are best all around - the question is more about how we transport the handshake - that is to say that a full obfu bridge test is the useful test for China today, not TCP or TLS - we already know the latter two fail almost all of the time.

Testing a normal bridge and testing an obfsproxy bridge are two different things.

Sure and they're both bridges.

And we don't know if the TCP test will fail for normal bridges almost all of the time.

Ah, to clarify, I mean _if_ the IP is known/blocked or an active probe is confirms it in real time, we know those two issues well enough. The problem scope is pretty well defined for those and we think currently that obfu bridges are not yet in the protocol detection bucket but rather only in the IP harvesting/known/blocked bucket.

I do agree that a TCP scan is safer for China than a full handshake but I'm not convinced it is terribly useful for China for that very same reason. I do think one useful outcome of the TCP scan is that we might learn where other people have triggered an active probe that resulted in an IP block. I wonder though, do we know for sure that active scans put those IP addresses into a firewall forever? Or have active scans replaced that static block list?

I don't know.

Me either.

Ok. I think that if we exclude those, we'll have a set of bridges that have likely been harvested and blocked or just blocked by active probing. It's not clear how we'll know the difference with a TCP connection scan.

It's a start. It's not supposed to be perfect.

Sure, I think small batches are probably best at first.

The plan was to scan all HTTPS bridges, but I guess we can reduce that to 2 out of 5 rings. 125 bridges in total.

I think the lower the number, the better a chance we'll get at refining these tests over a few hours - if we lose them all in on shot, we're sorta out of luck until they churn.

I don't expect us to have time for another test after this one, so these 125 bridges will be it.

I'd suggest that for such a scan - the first test should be something like a dozen bridges, the second the same dozen and so on. If you use all 125 at once, I wouldn't be surprised to learn that the next hour results in all of them being blocked with no more bridges to experiment with or use for the day.

Right, that's an assumption that needs to be made explicit. Want to help write the report, so that these things come over correctly?

I'm happy to look at the data but I don't know what such a report entails, I assume that's a different deliverable?

Ah, the "report" would simply summarize what we found in this deliverable, so that the sponsor doesn't have to read lengthy Trac tickets. I'd simply start a LaTeX article document, but it could also be a blog post. Something we can finish by tomorrow.

Well, it's 3am PST now on the 15th, so I think we'll need some data before anyone can commit to writing up an analysis of that data. I'm happy to try to understand the data and to contribute to whatever write up happens. I'm going to sleep for a bit and then check back when I wake up.

Good luck with the data collection!

comment:45 Changed 8 years ago by karsten

We ran the first scans from a node in .cn today. The scan had two phases: first, we scanned 20 bridges twice with a delay of one hour to see whether scanning affects bridge reachability in any way; second, we scanned 100 bridges (10 of which were already contained in the previous phase) to have a larger sample. All bridges were taken from the HTTPS bucket.

In the first phase, only 2 out of 20 bridges were found as reachable. One hour later, the same 2 bridges and another bridge were found as reachable. We concluded that the scan didn't lead to bridges being blocked and continued with the second phase.

In the second phase, 14 out of 100 bridges were found as reachable. From the 86 unreachable bridges we removed 6 bridges that were not found as reachable by Tonga, either. That leaves us with 80 bridges that Tonga found to be running and the scan found to be unreachable. Reasons for bridges being unreachable were: 77 x connection timed out, 1 x connection refused, and 2 x no route to host.

A manual analysis of the bridge usage statistics reported by the 14 reachable bridges confirms that these bridges are actually reachable from .cn: 1 bridge reported 24 connections from .cn, 5 bridges reported 16 connections, 6 bridges reported 8 connections, and 2 bridges didn't report country statistics. The bridge usage statistics reported by the 80 presumably blocked bridges have not been analyzed.

What else should we look at in the results?

And what are the next steps towards "some sort of automated ground truth of bridge reachability from some countries" that we can take in, say, the next week?

comment:46 in reply to:  45 ; Changed 8 years ago by ioerror

Replying to karsten:

We ran the first scans from a node in .cn today. The scan had two phases: first, we scanned 20 bridges twice with a delay of one hour to see whether scanning affects bridge reachability in any way; second, we scanned 100 bridges (10 of which were already contained in the previous phase) to have a larger sample. All bridges were taken from the HTTPS bucket.

In the first phase, only 2 out of 20 bridges were found as reachable. One hour later, the same 2 bridges and another bridge were found as reachable. We concluded that the scan didn't lead to bridges being blocked and continued with the second phase.

In the second phase, 14 out of 100 bridges were found as reachable. From the 86 unreachable bridges we removed 6 bridges that were not found as reachable by Tonga, either. That leaves us with 80 bridges that Tonga found to be running and the scan found to be unreachable. Reasons for bridges being unreachable were: 77 x connection timed out, 1 x connection refused, and 2 x no route to host.

A manual analysis of the bridge usage statistics reported by the 14 reachable bridges confirms that these bridges are actually reachable from .cn: 1 bridge reported 24 connections from .cn, 5 bridges reported 16 connections, 6 bridges reported 8 connections, and 2 bridges didn't report country statistics. The bridge usage statistics reported by the 80 presumably blocked bridges have not been analyzed.

What else should we look at in the results?

We should know the network from where the scans were performed - it is thought that a few large telecoms do DPI, did we try from one of those? Or if it makes sense and is accurate, we should simply say "we did not try it from the known networks that filter" or something similar.

And what are the next steps towards "some sort of automated ground truth of bridge reachability from some countries" that we can take in, say, the next week?

I think we'd need to automate these scans - can one simply toss them in a cron job and send the results somewhere to be processed? It seems like the process is pretty manual at this point, isn't it?

comment:47 in reply to:  46 Changed 8 years ago by runa

Replying to ioerror:

Replying to karsten:

We ran the first scans from a node in .cn today. The scan had two phases: first, we scanned 20 bridges twice with a delay of one hour to see whether scanning affects bridge reachability in any way; second, we scanned 100 bridges (10 of which were already contained in the previous phase) to have a larger sample. All bridges were taken from the HTTPS bucket.

In the first phase, only 2 out of 20 bridges were found as reachable. One hour later, the same 2 bridges and another bridge were found as reachable. We concluded that the scan didn't lead to bridges being blocked and continued with the second phase.

In the second phase, 14 out of 100 bridges were found as reachable. From the 86 unreachable bridges we removed 6 bridges that were not found as reachable by Tonga, either. That leaves us with 80 bridges that Tonga found to be running and the scan found to be unreachable. Reasons for bridges being unreachable were: 77 x connection timed out, 1 x connection refused, and 2 x no route to host.

A manual analysis of the bridge usage statistics reported by the 14 reachable bridges confirms that these bridges are actually reachable from .cn: 1 bridge reported 24 connections from .cn, 5 bridges reported 16 connections, 6 bridges reported 8 connections, and 2 bridges didn't report country statistics. The bridge usage statistics reported by the 80 presumably blocked bridges have not been analyzed.

What else should we look at in the results?

We should know the network from where the scans were performed - it is thought that a few large telecoms do DPI, did we try from one of those? Or if it makes sense and is accurate, we should simply say "we did not try it from the known networks that filter" or something similar.

I scanned from one of the hosts I had access to in PlanetLab. Would have to dig around if you want to know anything about the network the server is on.

And what are the next steps towards "some sort of automated ground truth of bridge reachability from some countries" that we can take in, say, the next week?

I think we'd need to automate these scans - can one simply toss them in a cron job and send the results somewhere to be processed? It seems like the process is pretty manual at this point, isn't it?

Yes, that's easy to do.

Changed 8 years ago by karsten

Changed 8 years ago by karsten

comment:48 Changed 8 years ago by karsten

Replying to karsten:

A manual analysis of the bridge usage statistics reported by the 14 reachable bridges confirms that these bridges are actually reachable from .cn: 1 bridge reported 24 connections from .cn, 5 bridges reported 16 connections, 6 bridges reported 8 connections, and 2 bridges didn't report country statistics. The bridge usage statistics reported by the 80 presumably blocked bridges have not been analyzed.

I looked closer at the scan results and compared bridge statistics of reachable and unreachable bridges. Here's a graph of the number of bridges reporting a given number of Tor connections within a 24-hour interval. Bridges in the first bin reported 1 to 8 connections, bridges in the second bin 9 to 16 connections, and so on. Bridges in the red parts were found unreachable by our scans, those in the green parts were found reachable.


It seems that our scan results aren't entirely random. The vast majority of bridges we couldn't reach reported 1 to 8 connections, whereas a fair fraction of reachable bridges reported 9 to 24 or even 57 to 64 connections. On average, unreachable bridges reported 5 connections, reachable bridges reported 17 connections.

I also attached the scan results containing fingerprint hashes matching the ones in sanitized bridge descriptors for future analysis.

comment:49 Changed 8 years ago by karsten

Resolution: implemented
Status: assignedclosed

Summarizing this sponsor F deliverable:

We developed a tool for scans of bridge lists which attempts to either open a simple TCP connection to the bridge's OR port or that will perform a full Tor handshake to the bridge. We tested the scanner by running a scan from .de and comparing results to bridge reachability information obtained from the bridge authority in .nl. We also performed a scan from .cn and found that over 80% of bridges were unreachable. We compared active scan results to passive usage statistics and confirmed that bridges that were found as unreachable in the scan also reported significantly fewer connections from .cn than other bridges.

I'm closing this project ticket. Any further discussion or analysis should go in a new ticket.

comment:50 Changed 8 years ago by runa

Is active bridge scanning something we want to continue doing? If yes, then I'll open a new ticket and we can discuss there.

comment:51 in reply to:  50 Changed 7 years ago by karsten

Replying to runa:

Is active bridge scanning something we want to continue doing? If yes, then I'll open a new ticket and we can discuss there.

It is! AFAICS, there's no specific sponsor deliverable for running more scans, but there are deliverables which depend on good bridge reachability information once we have it (sponsor F, year 2, items 15 and 17). For the moment we could do it for sponsor Z or without assigning it to a sponsor. And of course, if we have a better answer to item 10 (which this ticket was about), we can write a better summary there, even though we already consider it done. So, yes, please open a new ticket for this discussion.

comment:52 Changed 7 years ago by asn

I would say that we are also still interested in having access to machines in various censored countries, so that we can easily test if specific bridges and pluggable transports are blocked, and whether they are doing DPI or Active Scanning.

We currently have some sort of access in some such countries, but we don't have any good and fast backup plans if they get blocked.

comment:53 Changed 7 years ago by karsten

Keywords: SponsorF20120315 added
Milestone: Sponsor F: March 15, 2012

Switching from using milestones to keywords for sponsor deliverables. See #6365 for details.

Note: See TracTickets for help on using tickets.