GFW actively probes obfs2 bridges

It looks like the GFW is now actively probing obfs2. After hearing rumours yesterday, I wasn't able to reproduce this. Today, however, I got my private obfs2 bridge probed just milliseconds after my own connection from China. I got hit by two random Chinese addresses as we already know it from the Tor probing. After the probing, my obfs2 connection timed out and the SYN/ACK segments from the bridge were dropped when trying to establish a new connection. I could reproduce all of this several times.

I haven't tested obfs3 yet and I suppose we can skip the old looking-for-the-fingerprint game. Depending on what protocols they are trying to detect, they might have to probe several times since it's not clear what's behind all that entropy. It might be obfs2, obfs3 or VPN PSK and perhaps even more protocols.

added active censorship china component::circumvention/censorship analysis gfw obfs2 owner::phw priority::medium probing resolution::fixed severity::normal status::closed type::task labels

Fun fun.

Do you know what kind of probes where they? Did they actually complete the obfs2 handshake?

Could #8104 (closed) be related? I've been getting an increased number of such messages since the latest bundles were released.

Trac:
Cc: asn to asn, arma

Trac:
obfs2_probes.txt

obfs2 probes

Do you know what kind of probes where they? Did they actually complete the obfs2 handshake?

I attached a log with some scanners (and slightly more verbose log messages) in it.

I manually started pyobfsproxy with obfs2 and forwarded the traffic to a local echo daemon. I then connected to the bridge from within .cn using telnet and without sending any data. Tor was not involved. As a result, it looks like obfs2's server-side traffic is enough to trigger the probes.

With respect to the attached log: it looks like some of the probes just receive data and send nothing. Others send a little bit and the rest completes the handshake and sends all the promised padding. However, not a single probe seems to send actual application data. So I believe that they are actually fingerprinting obfs2 and don't care what it transports. That is probably smart since some people started tunneling their VPN traffic over obfs2. The GFW can probably catch these poor folks as well.

Replying to asn:

Could #8104 (closed) be related? I've been getting an increased number of such messages since the latest bundles were released.

Hmm, good question. I haven't attracted a lot of probes yet but the small handful I have sent the correct magic. If they actually developed their own obfs2 implementation instead of just getting ours, I wouldn't be surprised if they messed up the implementation a number of times. Also, they tend to send completely random and invalid data to other protocols as well (HTTP, HTTPS, ...). So I think #8104 (closed) could very well be related.

I quickly tested obfs3. It also gets probed but not blocked. The probes think it's talking obfs2 and send obfs2 handshake data to it. Since the handshake fails, it's probably not getting blocked.

Do we have instructions for how to set up obfs3 bridges anywhere? I can update the Tor Cloud images to be normal bridges, obfs2, and obfs3.

I've put some instructions in the future obfsproxy Debian package.

Regarding Tor Cloud images, if you wish to use binary packages, you can probably build pyptlib and obfsproxy (even if the later is not yet pushed to Debian, it works for me™).

Replying to phw:

I quickly tested obfs3. It also gets probed but not blocked. The probes think it's talking obfs2 and send obfs2 handshake data to it. Since the handshake fails, it's probably not getting blocked.

phw: did you figure out what triggers the obfs2 probe? Is it a number of characters? Or a lack of protocol identification? It seems non-obvious to me what they would DPI on to decide to do a probe. (If we're lucky, they're looking for redundancy in the protocol by basically doing a passive mitm on it -- if so, it means DPIing for obfs3 will be significantly messier, even if it isn't any more resistant to the follow-up probe.)

Trac:
Username: jonolumb
Cc: asn, arma to asn, arma, jonolumb@gmail.com

Replying to arma:

Replying to phw:

I quickly tested obfs3. It also gets probed but not blocked. The probes think it's talking obfs2 and send obfs2 handshake data to it. Since the handshake fails, it's probably not getting blocked.

phw: did you figure out what triggers the obfs2 probe? Is it a number of characters? Or a lack of protocol identification? It seems non-obvious to me what they would DPI on to decide to do a probe. (If we're lucky, they're looking for redundancy in the protocol by basically doing a passive mitm on it -- if so, it means DPIing for obfs3 will be significantly messier, even if it isn't any more resistant to the follow-up probe.)

I didn't. They might not even have a fingerprint. While testing, I got inconsistent results for many different setups. And as far as I know, David has even seen probes targeting telnet and plain-text HTTP. It looks like there is a lot of experimentation going on and their infrastructure might scale well. If you can afford to probe a lot, the fingerprints can become less accurate. So I would expect obfs3 to last as long as it'll take them to write active probing code for it.

A cheap thing to do would be to test obfs2+brdgrd and see if you get active probed. Although, reading Philipp's replies it seems that their detection method is so inconsistent that we won't get a clear answer on whether obfs2+brdgrd is beneficial.

Replying to asn:

A cheap thing to do would be to test obfs2+brdgrd and see if you get active probed. Although, reading Philipp's replies it seems that their detection method is so inconsistent that we won't get a clear answer on whether obfs2+brdgrd is beneficial.

I am not sure if brdgrd is still effective. For a number of Chinese users, my bridge was probed despite brdgrd running. The tool did its job but the scanners showed up nevertheless. So perhaps they are now partially able to reassemble TCP streams.

Replying to phw:

David has even seen probes targeting telnet and plain-text HTTP.

I received my first obfs2 probes on 2013-01-23, about two months before the opening of this ticket. On that day, probes were sent to port 25 (smtp) and port 80 (http). Here are the first few probes and their timestamps.

2013-01-23 00:36:35     smtp    obfs2
2013-01-23 00:37:35     smtp    obfs2
2013-01-23 04:47:16     http    obfs2
2013-01-23 05:04:02     http    obfs2
2013-01-24 00:40:27     smtp    obfs2
2013-01-24 05:21:39     http    obfs2
2013-01-24 16:02:25     http    obfs2
2013-01-24 16:02:28     http    obfs2
2013-01-29 01:28:45     smtp    obfs2
2013-01-29 08:03:05     http    obfs2
2013-02-01 03:51:34     smtp    obfs2
2013-02-01 03:51:36     smtp    obfs2

I now know that obfs2 probes have also been sent to (at least) port 22 (ssh), port 23 (telnet), and port 443 (https). My ssh logs do not go back that far, port 23 was not open at the time, and obfs2 sent to the https port would not survive the TLS handshake to be logged.

Shortly after that, the server started receiving dozens of probes daily.

2013-01-23     4
2013-01-24     4
2013-01-25     0
2013-01-26     0
2013-01-27     0
2013-01-28     0
2013-01-29     2
2013-01-30     0
2013-01-31     0
2013-02-01     6
2013-02-02     0
2013-02-03     2
2013-02-04    33
2013-02-05    34
2013-02-06    27
2013-02-07    25
2013-02-08    16
2013-02-09    21
2013-02-10    22

The server did not then, nor has ever run obfs2. It has been a vanilla bridge since January 2011. https://globe.torproject.org/#/bridge/272EB44C8992B8088BD8E8A12DB23B56478EB885 obfs2 probing continues to the present.

Closing this because it has by now been well analyzed in https://www.usenix.org/system/files/conference/foci12/foci12-final2.pdf and https://ensa.fi/active-probing/.

Trac:
Reviewer: N/A to N/A
Sponsor: N/A to N/A
Severity: N/A to Normal
Resolution: N/A to fixed
Status: new to closed

closed

moved to tpo/anti-censorship/censorship-analysis#8591 (closed)