"Censorship-timeline" for Tor
It was shortly discussed on #tor-dev that some sort of "censorship-timeline" for Tor would be helpful. In particular, this should provide:
- Detailed technical analyses of the censorship mechanisms in place (DPI fingerprints and manufacturers, traceroutes, ...)
- Code and data to reproduce all experiments
- Tor patches and standalone tools to evade the censorship devices
After all, this timeline should serve as a comprehensive archive for all people interested in how Tor is getting blocked. It should make it easy to answer questions such as "What happened to Tor in country X back in Y?".
There are also some open questions:
- How should the data be structured? In form of a timeline? Or country based? Something else?
- What data should be published and when? Full disclosure too early in the process helps the censors.
- How should it be presented? In a wiki page or a standalone web site?
Activity
Replying to runa:
Packet captures can be sensitive and we probably don't want to publish them online for everyone to see. Maybe we should put them in a private git.tpo repo for now?
It depends what the packet captures contain. If they are the packet captures of what a censorship event looks like as long as you strip the src IP they should be fine.
-
I recommend the tests that get written are made as an OONI test and abstracted in a way that they can be reused in other circumstances. I recommend to structure the Tests using the template on the OONI wiki: https://trac.torproject.org/projects/tor/wiki/doc/OONI/Tests/TestTemplate
Replying to hellais:
Replying to runa:
Packet captures can be sensitive and we probably don't want to publish them online for everyone to see. Maybe we should put them in a private git.tpo repo for now?
It depends what the packet captures contain. If they are the packet captures of what a censorship event looks like as long as you strip the src IP they should be fine.
I'd say that the source IP address is pretty useful to have. I don't know if there is a way to sanitize client and bridge pcap files without removing data that is useful to the person analyzing the files.
-
I recommend this data gets collected in the to be formed censorship wiki. This is a project started with other researchers in the field of censorship at rightscon: https://trac.torproject.org/projects/tor/wiki/doc/OONI/censorshipwiki.
We can then, if it becomes non practical to have it in a wiki, move it to a standalone website.
I think having both timeline and per country indexes would be of great use. I don't see why one should exclude the other. They will end up anyways being event specific so there is no reason to go for one over the other.
-
Replying to runa:
Replying to hellais:
Replying to runa:
Packet captures can be sensitive and we probably don't want to publish them online for everyone to see. Maybe we should put them in a private git.tpo repo for now?
It depends what the packet captures contain. If they are the packet captures of what a censorship event looks like as long as you strip the src IP they should be fine.
I'd say that the source IP address is pretty useful to have. I don't know if there is a way to sanitize client and bridge pcap files without removing data that is useful to the person analyzing the files.
Just put the ASN in place of the source ip. I don't think that makes the data at all less useful.
Sebastian created a git repository for this project; https://gitweb.torproject.org/censorship-timeline.git
Replying to hellais:
Just put the ASN in place of the source ip. I don't think that makes the data at all less useful.
Be very careful when thinking you've anonymized data. For example, if you take out the IP address, but you leave in a checksum of the previous thing that included the IP address, it is not hard to recompute the IP address.
-
Some good suggestions WRT sanitizing the pcap logs appeared on IRC:
< rransom> Runa, hellais: Keep in mind that country + IP header checksum is probably sufficient to recover redacted packet IP addresses.
< radii> hellais: then, it's important that in anonymized.pcap, all the frames for 192.168.1.100 map to a random key, say 3.4.5.6; while the frames for 192.168.1.101 map to a different random key, 8.7.6.5
< radii> if you just rand() for every packet, you lose way too much information and can't reconstruct TCP streams anymore (among many other problems)
Replying to arma:
Replying to hellais:
Just put the ASN in place of the source ip. I don't think that makes the data at all less useful.
Be very careful when thinking you've anonymized data. For example, if you take out the IP address, but you leave in a checksum of the previous thing that included the IP address, it is not hard to recompute the IP address.
Depending on how sensitive the data is, even port numbers can be a problem since we have to assume that data might be captured and stored by the censor for later analysis. Anonymizing traffic traces is a hard problem and in most cases it might be better to just provide the tools to quickly reproduce traffic traces.
We should also probably consider moving to a database design in the future, so that people can search by-country, or by-year, or by-DPI-box-manufacturer. But I guess that with the current amount of data, the wiki is a fine start.
BTW, I think failsafe pcap sanitization is pretty much a lost cause, except if someone audits all packets by hand to make sure that no application-layer leaks exist (assuming that we plugged all the network/transport-layer leaks). I agree with 'phw' that providing the tools to quickly reproduce traffic traces is a good idea.
-
I added some stuff to censorshipwiki (https://trac.torproject.org/projects/tor/wiki/doc/OONI/censorshipwiki). I tried to make up a general template for censorship incidents; it needs structure improvement, more data and polishing.
Trac:
Cc: asn, runa, arma to asn, runa, arma, g.koppen@jondos.de-
Replying to asn:
I added some stuff to censorshipwiki (https://trac.torproject.org/projects/tor/wiki/doc/OONI/censorshipwiki). I tried to make up a general template for censorship incidents; it needs structure improvement, more data and polishing.
I gave it a little bit more structure and data. However, just one wiki page might not be the best way to organize all the data since it becomes confusing rather quickly.
I experimented a little bit with timeline software. You can see an example here: http://www.7c0.org/tldemo/index2.html It's free, written in Javascript and relatively easy to extend using an XML file: http://www.7c0.org/tldemo/example1.xml
One possibility would be to use this timeline software for visualization and link to single trac pages which then cover all the censorship incidents in detail.
Any thoughts?
-
Replying to phw:
Replying to asn:
I added some stuff to censorshipwiki (https://trac.torproject.org/projects/tor/wiki/doc/OONI/censorshipwiki). I tried to make up a general template for censorship incidents; it needs structure improvement, more data and polishing.
I gave it a little bit more structure and data. However, just one wiki page might not be the best way to organize all the data since it becomes confusing rather quickly.
You can use as many wiki pages as you want. I restructured the data to be on a country by country basis. If we end up having too much information for country we can create sub pages for the countries.
I experimented a little bit with timeline software. You can see an example here: http://www.7c0.org/tldemo/index2.html It's free, written in Javascript and relatively easy to extend using an XML file: http://www.7c0.org/tldemo/example1.xml
One possibility would be to use this timeline software for visualization and link to single trac pages which then cover all the censorship incidents in detail.
I think we can achieve something similar with just a master trac page that has this information. If we want to do it the right way we may want to find a good trac plugin that does it, but I would try not to depend too much on external infrastructure.
Switching from using milestones to keywords for sponsor deliverables. See #6365 (moved) for details.
Trac:
Milestone: Sponsor Z: November 1, 2013 to N/A
Keywords: archive, dpi, block, censorship deleted, dpi archive censorship block SponsorZ addedThrowing in this blurb relevant for SponsorZ-stuff: As of right now, we have logs and network captures from six or seven different blocking events. What I would like to do is to analyze the data we have and see if there are any similarities between them, heuristics on spoofed packets, number of TCP resets, and so on. This could help answer questions such as "Is Ethiopia using the same type of device as the Philippines?", "Does Kazakhstan have a filter similar to the one used in the UAE?", and will hopefully make future packet analysis projects a bit easier.
-
Note( to self):
Another thing I would like to see in the censorshipwiki are the changes that Tor has done to its source code to dodge censorship. Also, the tor versions where the changes were introduced.
This is interesting both from a history perspective and for understanding how a specific Tor version can be blocked.
-
Replying to asn:
Note( to self):
Another thing I would like to see in the censorshipwiki are the changes that Tor has done to its source code to dodge censorship. Also, the tor versions where the changes were introduced.
This is interesting both from a history perspective and for understanding how a specific Tor version can be blocked.
That's a good idea. I added the page "Changes in Tor" to the Censorship Wiki and started by covering the cipher list change introduced in version 0.2.3.17-beta.
It looks like the censorship wiki isn't being maintained (for a few years now) and the needs are now being addressed by OONI.
Trac:
Resolution: N/A to wontfix
Cc: asn, runa, arma, g.koppen@jondos.de to asn, runa, arma, g.koppen@jondos.de, cass
Sponsor: N/A to N/A
Status: new to closed
Reviewer: N/A to N/A
Severity: N/A to Normalmentioned in issue #28531 (moved)
mentioned in issue tpo/community/outreach#28531 (moved)
mentioned in issue tpo/anti-censorship/team#87 (closed)