Opened 2 years ago

Closed 15 months ago

#20216 closed defect (wontfix)

Iran blocking of direct users, 2016-08 and 2016-09

Reported by: dcf Owned by:
Priority: Medium Milestone:
Component: Obfuscation/Censorship analysis Version:
Severity: Normal Keywords: censorship block ir
Cc: mrphs Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by dcf)

Direct users in Iran dropped from 8,000 to 2,000 between 2016-08-20 and 2016-08-23. The numbers recovered to 4,000, then crashed to 400 on 2016-09-03 and 2016-09-04.

Originally from https://metrics.torproject.org/userstats-relay-country.png?start=2016-06-24&end=2016-09-22&country=ir&events=off link

Edit 2016-10-04: the bridge changes below, on further investigation, appear to be unrelated to anything done by Iran.

Looking at bridge users, there is an increase right around 2016-08-20, the time of the first blocking, then an abrupt return to previous levels around 2016-09-03, the time of the second blocking.

Originally from https://metrics.torproject.org/userstats-bridge-country.png?start=2016-06-24&end=2016-09-22&country=ir link

Looking at the graph of bridge users by transport, obfs4 continued working while obfs3 and vanilla were blocked.

Originally from https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-06-24&end=2016-09-22&country=ir link

Child Tickets

Change History (18)

comment:1 Changed 2 years ago by mrphs

Looking at a couple of other countries, it seems obfs3 has been dropped for everyone at the same rate. Here's Germany and Russia for instance:

DE: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-06-24&end=2016-09-22&country=de

RU: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-06-24&end=2016-09-22&country=ru

So perhaps it's not about obfs3 being blocked.

Last edited 2 years ago by mrphs (previous) (diff)

comment:2 Changed 2 years ago by dcf

The change in direct users was noticed by the Filterprints detector. Iran went to the top of the "most anomalous in the last day" list on 2016-08-30:
http://lists.infolabe.net/archives/infolabe-anomalies/2016-September/000037.html

Last day:
=========
Iran, Islamic Republic of (IR)
Kazakhstan (KZ)
China (CN)
Georgia (GE)
Uzbekistan (UZ)
Bulgaria (BG)
Ethiopia (ET)
Ghana (GH)
Turkmenistan (TM)
Oman (OM)

Here's the 01day graph from that report (originally at http://lists.infolabe.net/archives/infolabe-anomalies/attachments/20160909/135fb600/attachment-0003.pdf).
convert -density 60 2016-09-06-01day.pdf 2016-09-06-01day.png (originally http://lists.infolabe.net/archives/infolabe-anomalies/attachments/20160909/135fb600/attachment-0003.pdf)

Changed 2 years ago by dcf

Attachment: 2016-09-06-01day.png added

convert -density 60 2016-09-06-01day.pdf 2016-09-06-01day.png (originally http://lists.infolabe.net/archives/infolabe-anomalies/attachments/20160909/135fb600/attachment-0003.pdf)

comment:3 in reply to:  description Changed 2 years ago by dcf

Replying to dcf:

Looking at bridge users, there is an increase right around 2016-08-20, the time of the first blocking, then an abrupt return to previous levels around 2016-09-03, the time of the second blocking.

Looking at the graph of bridge users by transport, obfs4 continued working while obfs3 and vanilla were blocked.

I accidentally swapped the images for these two paragraphs. Fixed now.

Last edited 2 years ago by dcf (previous) (diff)

comment:4 in reply to:  1 ; Changed 2 years ago by dcf

Replying to mrphs:

Looking at a couple of other countries, it seems obfs3 has been dropped for everyone at the same rate. Here's Germany and Russia for instance:

DE: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-06-24&end=2016-09-22&country=de

RU: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-06-24&end=2016-09-22&country=ru

So perhaps it's not about obfs3 being blocked.

You're right. Here's the graph of the most popular transports across all countries.

Originally from https://metrics.torproject.org/userstats-bridge-transport.png?start=2016-06-24&end=2016-09-22&transport=obfs3&transport=obfs4&transport=meek&transport=%3COR%3E link

obfs4 took a hit at the same time too, but didn't go to zero.

comment:5 Changed 2 years ago by joss

Hi All,

Just for a little extra context, we also have code to run the anomaly detection system on both normal and bridge usage, and only report a period as anomalous when there is an anomalous drop in normal usage and a corresponding anomalous rise in bridge usage. (Or vice versa, actually.)

The nice thing about the system we're working with here is that it's designed to exclude anomalies that are seen over a reasonable number of other countries, so it should only highlight per-country anomalies in usage.

As noted, Iran did experience this combined bridge/normal anomalous period, but there's been a quick drop back to non-anomalous behaviour for combined bridge usage. I really should be running this on the per-transport graphs...

For isolating events like this in the future, I'd be happy to work at getting this code running with regular outputs like the infolabe-anomalies mailing list that David mentioned. I'm wary of flooding too much data out there, though. Do let me know if there's something that would be of particular interest.

I could just output a list and graph every day for each country that experiences these combined anomalies. Over the five years of data we have, and 102 countries that meet our data criteria, we've spotted about 480 events where both normal and bridge usage shows an anomaly simultaneously, so these events are relatively rare and won't flood people with too much data.

I'll attach a recent combined anomaly Iran graph for reference:

Combined anomaly graph of Iranian standard Tor and bridge usage up to 2016-09-22.

Last edited 2 years ago by joss (previous) (diff)

Changed 2 years ago by joss

Attachment: iran-bridge-combined.png added

Combined anomaly graph of Iranian standard Tor and bridge usage up to 2016-09-22.

comment:6 in reply to:  4 Changed 2 years ago by dcf

Replying to dcf:

Replying to mrphs:

Looking at a couple of other countries, it seems obfs3 has been dropped for everyone at the same rate. Here's Germany and Russia for instance:

DE: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-06-24&end=2016-09-22&country=de

RU: https://metrics.torproject.org/userstats-bridge-combined.html?start=2016-06-24&end=2016-09-22&country=ru

So perhaps it's not about obfs3 being blocked.

You're right. Here's the graph of the most popular transports across all countries.

Karsten writes that 2016-09-02 was the date that the new bridge authority Bifroest took over from Tonga, causing about 40% of public bridges to stop reporting. If that 40% was disproportionately obfs3, that might help explain what we're seeing.

I don't see how the changed bridge authority could have an effect on the number of direct users, so the fact that direct and obfs3 users both decreased on 2016-09-02 could be a coincidence. I just took "obfs3" out of the ticket title.

comment:7 Changed 2 years ago by dcf

Summary: Iran blocking of vanilla and obfs3, 2016-08 and 2016-09Iran blocking of direct users, 2016-08 and 2016-09

comment:8 Changed 2 years ago by joss

It does seem odd behaviour.

The drop in ofs3 usage corresponds to a small decrease in the baseline number, but doesn't really change that much. I'm more interested in why the obsf4 spike was so short-lived. It looks like a clear response to the massive drop in direct usage, but doesn't last when direct usage falls to almost zero after that small recovery.

I wonder if this is some kind of doubled event. An initial block on direct usage, followed by a minor relaxation in which the direct usage climbed again while bridge usage fell, followed by a second block.

If I were speculating wildly, it almost looks like a new group of people tried bridges after an initial block, then didn't need them when direct connections became partially possible again, then didn't try bridges again when the block came back.

Do we have an AS-level breakdown of connections? Could this be some kind of per-network issue?

Last edited 2 years ago by joss (previous) (diff)

comment:9 in reply to:  8 Changed 2 years ago by dcf

Replying to joss:

The drop in ofs3 usage corresponds to a small decrease in the baseline number, but doesn't really change that much. I'm more interested in why the obsf4 spike was so short-lived. It looks like a clear response to the massive drop in direct usage, but doesn't last when direct usage falls to almost zero after that small recovery.

My guess is it went like this:

Iran blocks direct → users switch to obfs4 → change in bridge authority means a fraction of obfs4 bridges stop reporting → apparent but not actual end of obfs4 spike.

When the bridge authority changed, 5/5 default obfs3 bridges stopped reporting, while only 3/16 default obfs4 bridges stopped reporting. That may be why obfs3 crashed almost to zero while obfs4 just seemed to retard an upward trend.

I wonder if this is some kind of doubled event. An initial block on direct usage, followed by a minor relaxation in which the direct usage climbed again while bridge usage fell, followed by a second block.
If I were speculating wildly, it almost looks like a new group of people tried bridges after an initial block, then didn't need them when direct connections became partially possible again, then didn't try bridges again when the block came back.

My guess is that the first block was a naive one, simply blocking the IPs in some snapshot consensus. What looks like a relaxation of blocking is just natural churn in relays bringing new, unblocked entry nodes online. The second block looks like a continually updated blocklist, updated hourly or something.

Do we have an AS-level breakdown of connections? Could this be some kind of per-network issue?

No, there's not AS-level breakdown. The stats get aggregated to the country level at the bridge, as I understand it.

comment:10 Changed 2 years ago by dcf

Description: modified (diff)

comment:12 Changed 15 months ago by dcf

Keywords: ir added
Resolution: wontfix
Status: newclosed

Closing this old ticket.

Note: See TracTickets for help on using tickets.