#20495 closed task (fixed)

Unexplained drop in meek users, 2016-10-19 to 2016-11-10

Reported by: dcf Owned by:
Priority: Medium Milestone:
Component: Obfuscation/Censorship analysis Version:
Severity: Normal Keywords: cn us ru de gb meek
Cc: nikita Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

There was a drop in bridge users on October 19 or 20, 2016:
https://metrics.torproject.org/userstats-bridge-country.png?start=2016-07-30&end=2016-10-28&country=cn link

The by-transport graph shows that almost all meek users disappeared:
https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-07-30&end=2016-10-28&country=cn link

Child Tickets

Change History (15)

comment:1 Changed 13 months ago by dcf

Preliminary tests with Wget in two ASes in China worked. So if there is any blocking, it might be like other cases we've seen where it takes into account the TLS fingerprint.

All these commands worked:

wget --no-check-certificate -O - https://72.21.81.200/ --header 'Host: az786092.vo.msecnd.net'
wget -O - https://ajax.aspnetcdn.com/ --header 'Host: az786092.vo.msecnd.net'
wget --no-check-certificate -O - https://52.84.246.240/ --header 'Host: d2zfqthxsdq309.cloudfront.net'
wget -O - https://a0.awsstatic.com/ --header 'Host: d2zfqthxsdq309.cloudfront.net'

comment:2 Changed 13 months ago by nikita

Cc: nikita added

comment:3 Changed 13 months ago by dcf

I tested Firefox 45.2.0esr (the basis of current Tor Browser 6.0.5) and Firefox 45.4.0esr (the basis of current Tor Browser 6.5a3) through a SOCKS proxy into China. In other cases, this has been enough to stimulate blocking, because the TLS fingerprint of meek is (supposed to be) the same as that of the underlying Firefox. However, I was not able to find any blocking.

Firefox 45.2.0 https://a0.awsstatic.com/ not blocked
Firefox 45.2.0 https://ajax.aspnetcdn.com/ not blocked
Firefox 45.4.0 https://a0.awsstatic.com/ not blocked
Firefox 45.4.0 https://ajax.aspnetcdn.com/ not blocked

The TLS fingerprints of Firefox 45.2.0esr and 45.4.0esr differ slightly. 45.4.0esr has one additional Signature Hash Algorithm. Oddly, the TLS fingerprint of meek in Tor Browser 6.5a3 is the same as that of Firefox 45.2.0, despite Tor Browser 6.5a3 being based on Firefox 45.4.0.

Diff of Firefox 45.2.0 and Firefox 45.4.0:

             Extension: signature_algorithms
                 Type: signature_algorithms (0x000d)
-                Length: 22
-                Signature Hash Algorithms Length: 20
-                Signature Hash Algorithms (10 algorithms)
+                Length: 24
+                Signature Hash Algorithms Length: 22
+                Signature Hash Algorithms (11 algorithms)
                     Signature Hash Algorithm: 0x0401
                         Signature Hash Algorithm Hash: SHA256 (4)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0501
                         Signature Hash Algorithm Hash: SHA384 (5)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0601
                         Signature Hash Algorithm Hash: SHA512 (6)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0201
                         Signature Hash Algorithm Hash: SHA1 (2)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0403
                         Signature Hash Algorithm Hash: SHA256 (4)
                         Signature Hash Algorithm Signature: ECDSA (3)
                     Signature Hash Algorithm: 0x0503
                         Signature Hash Algorithm Hash: SHA384 (5)
                         Signature Hash Algorithm Signature: ECDSA (3)
                     Signature Hash Algorithm: 0x0603
                         Signature Hash Algorithm Hash: SHA512 (6)
                         Signature Hash Algorithm Signature: ECDSA (3)
                     Signature Hash Algorithm: 0x0203
                         Signature Hash Algorithm Hash: SHA1 (2)
                         Signature Hash Algorithm Signature: ECDSA (3)
+                    Signature Hash Algorithm: 0x0502
+                        Signature Hash Algorithm Hash: SHA384 (5)
+                        Signature Hash Algorithm Signature: DSA (2)
                     Signature Hash Algorithm: 0x0402
                         Signature Hash Algorithm Hash: SHA256 (4)
                         Signature Hash Algorithm Signature: DSA (2)
                     Signature Hash Algorithm: 0x0202
                         Signature Hash Algorithm Hash: SHA1 (2)
                         Signature Hash Algorithm Signature: DSA (2)
Last edited 13 months ago by dcf (previous) (diff)

comment:4 Changed 13 months ago by dcf

Keywords: censorship block cn added

Changed 13 months ago by dcf

Attachment: meek-by-country.png added

Graph showing meek users in many countries.

comment:5 Changed 13 months ago by dcf

Some other countries show a decrease on or about the same time, though not to the same relative extent: us, ru, de, gb. (Notably absent is br, which is way off the chart at the top.)

One possibility is that this is another case of some botnet or other widespread software that used meek but then suddenly stopped. Perhaps China was simply more affected than other countries.

Graph showing meek users in many countries.

library(ggplot2)

# https://metrics.torproject.org/userstats-combined-data.html
x <- read.csv("userstats-combined.csv")
x$date <- as.Date(x$date)
x$avg <- ave((x$low + x$high)/2, x$country)

p <- ggplot(x[x$node=="bridge" & x$transport=="meek" & x$avg>=50, ], aes(x=date, y=(high+low)/2, ymax=high, ymin=low, color=country, fill=country))
p <- p + geom_ribbon(alpha=0.5)
p <- p + geom_text(aes(label=country))
p <- p + theme(legend.position="none")
p <- p + coord_cartesian(xlim=c(as.Date("2016-09-01"), as.Date("2016-10-28")), ylim=c(0, 1500))
p <- p + scale_x_date(date_breaks="1 month", date_minor_breaks="1 week")
ggsave("meek-by-country.png", p, width=10, height=5, dpi=90)

comment:6 in reply to:  3 Changed 13 months ago by gk

Replying to dcf:

I tested Firefox 45.2.0esr (the basis of current Tor Browser 6.0.5)

Maybe just a minor thing but 6.0.5 is based on 45.4.0esr as well. Both the stable and the alphas are usually using the same Firefox version. The exception so far has been the migration to a new ESR then the alpha was one release earlier already on the new one.

Changed 12 months ago by dcf

comment:7 Changed 12 months ago by dcf

Keywords: us ru de gb added; censorship block removed
Summary: China blocking of meek, 2016-10-19Unexplained drop in meek users, 2016-10-19 to 2016-11-10

Since 2016-11-10, the count of meek users has mysteriously returned to previous levels, not only in China but in the other places mentioned in comment:5.

This no longer looks like a censorship event to me, though I don't know what it could be.

https://metrics.torproject.org/userstats-bridge-country.png?start=2016-07-30&end=2016-11-15&country=cn link

https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-07-30&end=2016-11-15&country=cn link


One possible clue is that TorLandMeek, the meek-amazon bridge, currently has a last restarted date of 2016-11-09 22:11:20, which is close to when the numbers returned to normal. However, TorLandMeek doesn't show a decrease in users during this time.

comment:8 Changed 12 months ago by dcf

The drop might have been caused by problems in an Orbot release. The dates match up:

https://groups.google.com/d/msg/traffic-obf/CSJLt3t-_OI/FnAqWqquAwAJ

Could have been the tumultuous upgrade cycle we had for the last version of Orbot... We released an RC that was pretty broken, taking out a good chunk of our users for awhile starting around October 20th:
https://lists.mayfirst.org/pipermail/guardian-dev/2016-October/004938.html
and finally stabilizing around Nov 7th:
https://lists.mayfirst.org/pipermail/guardian-dev/2016-November/004986.html

comment:9 Changed 12 months ago by dcf

Resolution: fixed
Status: newclosed

Closing because this seems settled.

Note: See TracTickets for help on using tickets.