Opened 4 months ago

Last modified 12 days ago

#30636 new task

Something funky is going in Iran: numbers of relay users flies off to 1M+

Reported by: cypherpunks Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Analysis Version:
Severity: Normal Keywords: ir
Cc: dcf, gaba, phw, mrphs Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Child Tickets

Attachments (13)

userstats-relay-country-ir-2019-01-01-2019-06-01-off.png (23.1 KB) - added by dcf 4 months ago.
https://metrics.torproject.org/userstats-relay-country.png?start=2019-01-01&end=2019-06-01&country=ir
userstats-bridge-country-ir-2019-01-01-2019-06-01.png (36.4 KB) - added by dcf 4 months ago.
https://metrics.torproject.org/userstats-bridge-country.png?start=2019-01-01&end=2019-06-01&country=ir
userstats-bridge-combined-ir-2019-01-01-2019-06-01.png (116.6 KB) - added by dcf 4 months ago.
https://metrics.torproject.org/userstats-bridge-combined.png?start=2019-01-01&end=2019-06-01&country=ir
MTProto Post Reach.png (162.3 KB) - added by dcf 4 months ago.
https://www.dropbox.com/s/ggin9dmkobowkve/MTProto%20Post%20Reach.png?dl=0
userstats-relay-country-ir-2019-03-01-2019-06-11-off.png (20.8 KB) - added by dcf 3 months ago.
https://metrics.torproject.org/userstats-relay-country.png?start=2019-03-01&end=2019-06-11&country=ir
userstats-bridge-country-ir-2019-03-01-2019-06-11.png (33.4 KB) - added by dcf 3 months ago.
https://metrics.torproject.org/userstats-bridge-country.png?start=2019-03-01&end=2019-06-11&country=ir
userstats-bridge-combined-ir-2019-03-01-2019-06-11.png (107.8 KB) - added by dcf 3 months ago.
https://metrics.torproject.org/userstats-bridge-combined.png?start=2019-03-01&end=2019-06-11&country=ir
iran-ioda.png (65.0 KB) - added by phw 3 months ago.
IODA signals for Iran.
userstats-relay-country-ir-2019-04-01-2019-09-03-off.png (26.0 KB) - added by dcf 2 weeks ago.
https://metrics.torproject.org/userstats-relay-country.png?start=2019-04-01&end=2019-09-03&country=ir&events=off
userstats-bridge-country-ir-2019-04-01-2019-09-03.png (24.1 KB) - added by dcf 2 weeks ago.
https://metrics.torproject.org/userstats-bridge-country.png?start=2019-04-01&end=2019-09-03&country=ir
userstats-bridge-combined-ir-2019-04-01-2019-09-03.png (57.9 KB) - added by dcf 2 weeks ago.
https://metrics.torproject.org/userstats-bridge-combined.png?start=2019-04-01&end=2019-09-03&country=ir
history-eRYaZuvY02FpExln-20190903.png (14.4 KB) - added by dcf 2 weeks ago.
bw_months https://metrics.torproject.org/rs.html#details/AA033EEB61601B2B7312D89B62AAA23DC3ED8A34
history-starman-20190903.png (11.2 KB) - added by dcf 2 weeks ago.
bw_months https://metrics.torproject.org/rs.html#details/AA033EEB61601B2B7312D89B62AAA23DC3ED8A34

Download all attachments as: .zip

Change History (47)

comment:1 Changed 4 months ago by cypherpunks

Woops, wrong component, my apologies.

comment:2 Changed 4 months ago by dcf

Description: modified (diff)
Keywords: ir added

Woops, wrong component, my apologies.

What component did you intend? I was thinking Censorship/Censorship Analysis, but possibly this is not related to censorship.

I added an entry on the Metrics Timeline.

comment:3 Changed 4 months ago by gaba

Cc: gaba added

comment:4 Changed 4 months ago by phw

Cc: phw added

comment:5 Changed 4 months ago by mrphs

Cc: mrphs added

comment:6 Changed 4 months ago by mrphs

I just checked from a few different networks and also asked a number of people to test it from different ISPs and it seems like everyone is able to connect directly without a bridge. I'll update the ticket if I find out more.

comment:7 Changed 4 months ago by mrphs

According to the reports I gathered, it seems like all ISPs except for "Pars Online" are able to connect directly.

comment:8 Changed 4 months ago by arma

It now seems that 1/4 of the users in the Tor network are in Iran:
https://metrics.torproject.org/userstats-relay-table.html?start=2019-05-30&end=2019-06-03

I looked at some relay extrainfo descriptors, to see if this is a fluke or what, and it looks like many relays are seeing not just a huge number of v3 consensus fetches from Iran, but also a huge variety of IP addresses fetching these consensus documents. So my current thought is that these really are a bunch of different Tor clients running in different places in Iran.

The shape of the growth makes me think it isn't many hundreds of thousands of people each one at a time deciding to install Tor Browser though. I wonder if Tor is now bundled in some software that many of them already had, and when it upgraded, they became Tor users? See for example how this happened in Ukraine two years ago, where the FreeU browser bundled a Tor client: #22369.

comment:9 Changed 4 months ago by arma

https://metrics.torproject.org/dirbytes.html makes it look like the directory load is growing with the recent pattern of Russian users, but not growing with the recent pattern of Iranian users. I wonder why that is.

comment:10 Changed 4 months ago by mrphs

I wonder how many of these are Orbot installations which were sitting on Android devices and now are suddenly connecting back to the network. Iranian users do have a habit of having all the censorship circumvention tools installed and try to see which one works everyday they pick up their phones. I asked Nathan to see if he can give us any statistics on how many active installation (through Google play store) we currently have in IR.

comment:11 Changed 4 months ago by cypherpunks

Tor Browser for Android without Orbot has been released. So, where are all that new users?

comment:12 Changed 4 months ago by n8fr8

Google Play reports active users of Orbot in Iran has gone from 40k to about 70k in the last two months. This is a big spike, as before we would just grow a few k per month.

Iran is our #5 country, after Russia, India, US, and Ukraine.

Now, this is only measuring users who install Orbot from Google Play. If they are getting it from somewhere else, I don't think we can see those numbers.

comment:13 Changed 4 months ago by xhdix

Some unofficial telegram applications (e.g. Bgram) use Tor.

In the past month, several unofficial telegrams were removed by Google Play Protect service from users' devices. (E.g. talagram, hotgram, mobogram)

In the past few weeks in Iran, the speed of blocking MTProxy servers has increased. So the only remaining option for the unofficial telegram application was to integrate with Tor.

This is my guess. (Also reports from unofficial telegram builders, and reverse engineering of published files in the telegram)

Last edited 4 months ago by xhdix (previous) (diff)

comment:14 Changed 4 months ago by torfone

Don't know about Iran, but for Russia this can be my Torfone (see https://habr.com/ru/post/448856/).
App: http://torfone.org/download/Torfone.apk
Core: https://github.com/gegel/torfone

comment:16 Changed 4 months ago by xhdix

@emmapeel
I think that when the increase began, it was before Tor release date:
https://metrics.torproject.org/userstats-relay-country.html?start=2019-05-13&end=2019-05-23&country=ir&events=off

https://metrics.torproject.org/userstats-relay-country.png

Last edited 4 months ago by dcf (previous) (diff)

comment:17 Changed 4 months ago by difautsch

So, according to a report released on May 30 from Iran Human Rights Watch, the Iranian government is supporting the development of a proxy tool, MTProto, so they can better 'manage' censorship circumvention.
https://iranhumanrights.org/2019/05/why-is-the-iranian-government-aiding-the-development-of-a-censorship-circumvention-tool/

I believe that the new Tor users were previously using MTProto and then realized it's not keeping their privacy from the Iranian gov.
My evidence is a decline in 'Post Reach' on May 15, the same day Iranian Tor users began to grow: https://www.dropbox.com/s/ggin9dmkobowkve/MTProto%20Post%20Reach.png?dl=0
https://www.dropbox.com/s/ggin9dmkobowkve/MTProto%20Post%20Reach.png?dl=0

I don't know what specifically triggered the switch from MTP to Tor, but a significant and sustained drop of Post Reach is definitely suspicious.

You can find all the metrics here: https://tgstat.com/channel/@MTProxies

Last edited 4 months ago by dcf (previous) (diff)

comment:18 Changed 3 months ago by xhdix

Also, according to a query from people, messages similar to the screenshot below were published in telegram groups.
In this message, installing and using Tor is trained to work on Telegram.
https://imgur.com/RqyjccN

https://i.imgur.com/RqyjccNl.jpg

comment:19 Changed 3 months ago by cypherpunks

Title should be edited as the figure is now much more spectacular. To all our Farsi friends: thanks for making it happen, we love you! <3

comment:20 Changed 3 months ago by xhdix

Yeah :D
The number of users is now more than 1 million. (1067335)

All Iranian people are grateful to the members of the Tor Project.

https://metrics.torproject.org/userstats-relay-country.png

comment:21 Changed 3 months ago by dcf

Description: modified (diff)
Summary: Something funky is going in Iran: numbers of relay users flies off to 400KSomething funky is going in Iran: numbers of relay users flies off to 1M+

comment:22 in reply to:  17 ; Changed 3 months ago by difautsch

Replying to difautsch:

So, according to a report released on May 30 from Iran Human Rights Watch, the Iranian government is supporting the development of a proxy tool, MTProto, so they can better 'manage' censorship circumvention.
https://iranhumanrights.org/2019/05/why-is-the-iranian-government-aiding-the-development-of-a-censorship-circumvention-tool/

I believe that the new Tor users were previously using MTProto and then realized it's not keeping their privacy from the Iranian gov.
My evidence is a decline in 'Post Reach' on May 15, the same day Iranian Tor users began to grow: https://www.dropbox.com/s/ggin9dmkobowkve/MTProto%20Post%20Reach.png?dl=0
https://www.dropbox.com/s/ggin9dmkobowkve/MTProto%20Post%20Reach.png?dl=0

From the OONI forums, Iran's internet is down. Apparently, fiber optic cables have been cut.
https://twitter.com/sadjadb/status/1136646856652525568
I wonder if this related to the U.S. - Iran tensions?

I don't know what specifically triggered the switch from MTP to Tor, but a significant and sustained drop of Post Reach is definitely suspicious.

You can find all the metrics here: https://tgstat.com/channel/@MTProxies

Changed 3 months ago by phw

Attachment: iran-ioda.png added

IODA signals for Iran.

comment:23 in reply to:  22 Changed 3 months ago by phw

Replying to difautsch:

From the OONI forums, Iran's internet is down. Apparently, fiber optic cables have been cut.
https://twitter.com/sadjadb/status/1136646856652525568
I wonder if this related to the U.S. - Iran tensions?


The tweet is from June 6. According to IODA, it looks like there was indeed a noticeable drop in connectivity but it affected less than 5% of the address space that Iran announces over BGP:
IODA signals for Iran.

comment:24 Changed 3 months ago by arma

Two more data points here:

(A) I am coming to believe that our user count numbers are based on counting requests, not on counting successful deliveries of the consensus. So if a client gets far enough through the process to request a consensus, but then has their connection cut, (1) we will count them as a user, and (2) they will come back somewhere else soon after to retry, and we'll count that next one as a new user. I am beginning to suspect that's a factor in what's happening here. But that still doesn't explain why the number keeps going up -- it's not like clients start asking more and more frequently as they fail more often.

(B) We do have a count of successful consensus deliveries, vs delivery attempts that time out. Dir mirrors can't tell whether the client receives all the bytes, but because the consensus takes more than one stream window worth of cells to deliver, the dir mirrors can tell that all but the last stream window (250KB) of cells were acknowledged by the client. *But*, with the advent of consensus diffs, the entire diff fits within a single stream window, so every time a client asks for a diff, we're going to count that delivery as a success, even if no bytes actually make it to the client.

(B') And while we're on that topic, there actually *is* a way to learn whether the client received the last part -- see the concern in https://trac.torproject.org/projects/tor/ticket/30926#comment:3 where we get a stream-level sendme for an unknown streamid. That happens when we've finished sending all of the consensus bytes, and then we send the end cell, and then we close the stream on our side, and we get a sendme with an unknown streamid because we've already forgotten about the stream. But all of the data is there for us to be able to confirm that the client has received (nearly all of) the bytes, if we want to.

We should probably make a bunch of tickets out of these various bugs and feature ideas, but I wanted to get them written up somewhere first.

Last edited 3 months ago by arma (previous) (diff)

comment:25 Changed 3 months ago by xhdix

Many users report that they can not connect directly to Tor.
Connections with Bridge also have problems for them. (Connected but not stable)

There is a problem with handshake in SSL / TLS communication that the time of occurrence of the problem is different in each city and ISP.

logs:
https://paste.ubuntu.com/p/jrNQGhqBTx/

https://i.imgur.com/XbyPkPLl.jpg

https://i.imgur.com/G9kZEPtl.jpg

Last edited 3 months ago by xhdix (previous) (diff)

comment:26 Changed 3 months ago by xhdix

Only servers that are listed in metrics.torproject.org have problems.
Even middle-relays
On the first day, from Iran, I could get the authority file from my middle-relay server, but attempting to connect SSH and HTTPS to my server received a timeout error in the handshake.

https://i.imgur.com/fxz4eEl.jpg

https://i.imgur.com/uSLA0GT.jpg

https://i.imgur.com/uP5lcML.jpg

Fall down:
https://metrics.torproject.org/userstats-relay-country.png

comment:27 Changed 3 months ago by cypherpunks

half million just dropped

comment:28 Changed 3 months ago by mrphs

And it's gonna get back to ~0 shortly, since the direct connections to the tor network are blocked again.

comment:29 Changed 3 months ago by xhdix

I can tell people to use the bridge, but there are two problems:

1- The orbot app has not yet released its new update:
https://bugs.torproject.org/30870

2- This solution will be very difficult:
https://github.com/termux/termux-packages/issues/3198
Is there a better solution for installing via termux?

comment:30 Changed 3 months ago by xhdix

Bridge usage is increasing:
https://metrics.torproject.org/userstats-bridge-combined.png

comment:31 Changed 3 months ago by xhdix

Again, users from Iran can connect directly to Tor. :)

comment:32 Changed 3 weeks ago by xhdix

and again..
The ability to connect directly to Tor was blocked, and it was blocked through the list of IPs in the collector.
https://metrics.torproject.org/userstats-relay-country.png

https://metrics.torproject.org/userstats-bridge-combined.png

It looks like an app (probably an unofficial telegram) uses <OR>. (Due to a sharp increase and decrease)

comment:33 in reply to:  32 ; Changed 2 weeks ago by dcf

Description: modified (diff)

Replying to xhdix:

and again..
The ability to connect directly to Tor was blocked, and it was blocked through the list of IPs in the collector.

How do you know that it was blocked using the CollecTor?

It looks like an app (probably an unofficial telegram) uses <OR>. (Due to a sharp increase and decrease)

In the recent past, I've seen two large but brief spikes of users from IP addresses located in Iran, on two of my vanilla bridges. (In the second case, the bridge is actually supposed to be a meek bridge, but the Iranian IP addresses were connecting to one of its exposed ORPorts.)

It would be an interesting experiment to go through CollecTor data and see if any other bridges have experienced spikes like this. A hypothesis is that a Telegram app, or whatever it is, chooses one bridge to use for a day, then changes to another one. If that were the case, we would be able to identify spikes rolling temporally across multiple bridges.

https://metrics.torproject.org/rs.html#details/272EB44C8992B8088BD8E8A12DB23B56478EB885
bw_months https://metrics.torproject.org/rs.html#details/AA033EEB61601B2B7312D89B62AAA23DC3ED8A34

https://metrics.torproject.org/rs.html#details/AA033EEB61601B2B7312D89B62AAA23DC3ED8A34
bw_months https://metrics.torproject.org/rs.html#details/AA033EEB61601B2B7312D89B62AAA23DC3ED8A34

comment:34 in reply to:  33 Changed 12 days ago by xhdix

Replying to dcf:

Sorry. I didn't receive email about this replay!

How do you know that it was blocked using the CollecTor?

This is just the closest conjecture to this method of blocking.

It would be an interesting experiment to go through CollecTor data and see if any other bridges have experienced spikes like this.

Yes, It would be interesting. And I'm interested to know.

Note: See TracTickets for help on using tickets.