Opened 7 months ago

Closed 6 months ago

#20495 closed task (fixed)

Unexplained drop in meek users, 2016-10-19 to 2016-11-10

Reported by: dcf Owned by:
Priority: Medium Milestone:
Component: Metrics/Censorship analysis Version:
Severity: Normal Keywords: cn us ru de gb meek
Cc: nikita Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

There was a drop in bridge users on October 19 or 20, 2016:
https://metrics.torproject.org/userstats-bridge-country.png?start=2016-07-30&end=2016-10-28&country=cn link

The by-transport graph shows that almost all meek users disappeared:
https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-07-30&end=2016-10-28&country=cn link

Child Tickets

Change History (15)

comment:1 Changed 7 months ago by dcf

Preliminary tests with Wget in two ASes in China worked. So if there is any blocking, it might be like other cases we've seen where it takes into account the TLS fingerprint.

All these commands worked:

wget --no-check-certificate -O - https://72.21.81.200/ --header 'Host: az786092.vo.msecnd.net'
wget -O - https://ajax.aspnetcdn.com/ --header 'Host: az786092.vo.msecnd.net'
wget --no-check-certificate -O - https://52.84.246.240/ --header 'Host: d2zfqthxsdq309.cloudfront.net'
wget -O - https://a0.awsstatic.com/ --header 'Host: d2zfqthxsdq309.cloudfront.net'

comment:2 Changed 7 months ago by nikita

  • Cc nikita added

comment:3 follow-up: Changed 7 months ago by dcf

I tested Firefox 45.2.0esr (the basis of current Tor Browser 6.0.5) and Firefox 45.4.0esr (the basis of current Tor Browser 6.5a3) through a SOCKS proxy into China. In other cases, this has been enough to stimulate blocking, because the TLS fingerprint of meek is (supposed to be) the same as that of the underlying Firefox. However, I was not able to find any blocking.

Firefox 45.2.0 https://a0.awsstatic.com/ not blocked
Firefox 45.2.0 https://ajax.aspnetcdn.com/ not blocked
Firefox 45.4.0 https://a0.awsstatic.com/ not blocked
Firefox 45.4.0 https://ajax.aspnetcdn.com/ not blocked

The TLS fingerprints of Firefox 45.2.0esr and 45.4.0esr differ slightly. 45.4.0esr has one additional Signature Hash Algorithm. Oddly, the TLS fingerprint of meek in Tor Browser 6.5a3 is the same as that of Firefox 45.2.0, despite Tor Browser 6.5a3 being based on Firefox 45.4.0.

Diff of Firefox 45.2.0 and Firefox 45.4.0:

             Extension: signature_algorithms
                 Type: signature_algorithms (0x000d)
-                Length: 22
-                Signature Hash Algorithms Length: 20
-                Signature Hash Algorithms (10 algorithms)
+                Length: 24
+                Signature Hash Algorithms Length: 22
+                Signature Hash Algorithms (11 algorithms)
                     Signature Hash Algorithm: 0x0401
                         Signature Hash Algorithm Hash: SHA256 (4)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0501
                         Signature Hash Algorithm Hash: SHA384 (5)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0601
                         Signature Hash Algorithm Hash: SHA512 (6)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0201
                         Signature Hash Algorithm Hash: SHA1 (2)
                         Signature Hash Algorithm Signature: RSA (1)
                     Signature Hash Algorithm: 0x0403
                         Signature Hash Algorithm Hash: SHA256 (4)
                         Signature Hash Algorithm Signature: ECDSA (3)
                     Signature Hash Algorithm: 0x0503
                         Signature Hash Algorithm Hash: SHA384 (5)
                         Signature Hash Algorithm Signature: ECDSA (3)
                     Signature Hash Algorithm: 0x0603
                         Signature Hash Algorithm Hash: SHA512 (6)
                         Signature Hash Algorithm Signature: ECDSA (3)
                     Signature Hash Algorithm: 0x0203
                         Signature Hash Algorithm Hash: SHA1 (2)
                         Signature Hash Algorithm Signature: ECDSA (3)
+                    Signature Hash Algorithm: 0x0502
+                        Signature Hash Algorithm Hash: SHA384 (5)
+                        Signature Hash Algorithm Signature: DSA (2)
                     Signature Hash Algorithm: 0x0402
                         Signature Hash Algorithm Hash: SHA256 (4)
                         Signature Hash Algorithm Signature: DSA (2)
                     Signature Hash Algorithm: 0x0202
                         Signature Hash Algorithm Hash: SHA1 (2)
                         Signature Hash Algorithm Signature: DSA (2)
Last edited 7 months ago by dcf (previous) (diff)

comment:4 Changed 7 months ago by dcf

  • Keywords censorship block cn added

Changed 7 months ago by dcf

Graph showing meek users in many countries.

comment:5 Changed 7 months ago by dcf

Some other countries show a decrease on or about the same time, though not to the same relative extent: us, ru, de, gb. (Notably absent is br, which is way off the chart at the top.)

One possibility is that this is another case of some botnet or other widespread software that used meek but then suddenly stopped. Perhaps China was simply more affected than other countries.

Graph showing meek users in many countries.

library(ggplot2)

# https://metrics.torproject.org/userstats-combined-data.html
x <- read.csv("userstats-combined.csv")
x$date <- as.Date(x$date)
x$avg <- ave((x$low + x$high)/2, x$country)

p <- ggplot(x[x$node=="bridge" & x$transport=="meek" & x$avg>=50, ], aes(x=date, y=(high+low)/2, ymax=high, ymin=low, color=country, fill=country))
p <- p + geom_ribbon(alpha=0.5)
p <- p + geom_text(aes(label=country))
p <- p + theme(legend.position="none")
p <- p + coord_cartesian(xlim=c(as.Date("2016-09-01"), as.Date("2016-10-28")), ylim=c(0, 1500))
p <- p + scale_x_date(date_breaks="1 month", date_minor_breaks="1 week")
ggsave("meek-by-country.png", p, width=10, height=5, dpi=90)

comment:6 in reply to: ↑ 3 Changed 7 months ago by gk

Replying to dcf:

I tested Firefox 45.2.0esr (the basis of current Tor Browser 6.0.5)

Maybe just a minor thing but 6.0.5 is based on 45.4.0esr as well. Both the stable and the alphas are usually using the same Firefox version. The exception so far has been the migration to a new ESR then the alpha was one release earlier already on the new one.

Changed 6 months ago by dcf

comment:7 Changed 6 months ago by dcf

  • Keywords us ru de gb added; censorship block removed
  • Summary changed from China blocking of meek, 2016-10-19 to Unexplained drop in meek users, 2016-10-19 to 2016-11-10

Since 2016-11-10, the count of meek users has mysteriously returned to previous levels, not only in China but in the other places mentioned in comment:5.

This no longer looks like a censorship event to me, though I don't know what it could be.

https://metrics.torproject.org/userstats-bridge-country.png?start=2016-07-30&end=2016-11-15&country=cn link

https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-07-30&end=2016-11-15&country=cn link


One possible clue is that TorLandMeek, the meek-amazon bridge, currently has a last restarted date of 2016-11-09 22:11:20, which is close to when the numbers returned to normal. However, TorLandMeek doesn't show a decrease in users during this time.

comment:8 Changed 6 months ago by dcf

The drop might have been caused by problems in an Orbot release. The dates match up:

https://groups.google.com/d/msg/traffic-obf/CSJLt3t-_OI/FnAqWqquAwAJ

Could have been the tumultuous upgrade cycle we had for the last version of Orbot... We released an RC that was pretty broken, taking out a good chunk of our users for awhile starting around October 20th:
https://lists.mayfirst.org/pipermail/guardian-dev/2016-October/004938.html
and finally stabilizing around Nov 7th:
https://lists.mayfirst.org/pipermail/guardian-dev/2016-November/004986.html

comment:9 Changed 6 months ago by dcf

  • Resolution set to fixed
  • Status changed from new to closed

Closing because this seems settled.

Note: See TracTickets for help on using tickets.