Opened 22 months ago

Closed 14 months ago

Last modified 3 months ago

#21014 closed task (wontfix)

Turkey blocking of direct connections, 2016-12-12

Reported by: mrphs Owned by: metrics-team
Priority: Medium Milestone:
Component: Obfuscation/Censorship analysis Version:
Severity: Normal Keywords: censorship block tr Turkey UX
Cc: dcf Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by dcf)

Turkey Blocks article: https://turkeyblocks.org/2016/12/18/tor-blocked-in-turkey-vpn-ban/

After getting some reports on twitter about Tor being blocked in Turkey and some chat on IRC, <bypassemall> aka <trdpi> aka <kzdpi> ran some tests and found some interesting information about how Turkey is blocking vanilla Tor connections. I paste their findings here:

16:48 < trdpi> 10 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
16:48 < trdpi> after less than 10 seconds
...
16:55 < trdpi> this isp injects rst it seems
16:56 < trdpi> to both side, as i got 2 rst one legit and 2 not
16:57 < mrphs> oh apparently today is an special day in turkey
...
17:00 < trdpi> telneting to or port, no rsts. it triggered by something more than ip:port connection
17:01 < trdpi> yay, window trick for split req works for tr
17:02 < trdpi> magic tool allows to bypass vanilla tor censorship
17:04 < trdpi> so it's about ciphersuits or something
17:07 < trdpi> it's like kz, but obfs4 works
17:07 < trdpi> and kz do not rsts
17:07 < trdpi> it controlls connection
17:07 < trdpi> and tr like do not controlls and to inject fraud only

Child Tickets

Attachments (12)

userstats-relay-country-tr-2016-09-18-2016-12-17-off.png (9.0 KB) - added by dcf 22 months ago.
https://metrics.torproject.org/userstats-relay-country.png?start=2016-09-18&end=2016-12-17&country=tr&events=off
userstats-bridge-country-tr-2016-09-18-2016-12-17.png (9.2 KB) - added by dcf 22 months ago.
https://metrics.torproject.org/userstats-bridge-country.png?start=2016-09-18&end=2016-12-17&country=tr
userstats-bridge-combined-tr-2016-09-18-2016-12-17.png (28.7 KB) - added by dcf 22 months ago.
https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-09-18&end=2016-12-17&country=tr
tr-tor-20161215.png (17.8 KB) - added by dcf 22 months ago.
obfs3.jpg (42.8 KB) - added by mrphs 22 months ago.
obfs3_connection-error.jpg
obfs4.jpg (43.5 KB) - added by mrphs 22 months ago.
obfs4_connection-error
tor_dirauth_dir_v3_direct_dl_timeout-pinpoint=1479226024,1482077224.png (38.2 KB) - added by dcf 22 months ago.
http://ygzf7uqcusp4ayjs.onion/munin-cgi/munin-cgi-graph/tor-health/tor-health/tor_dirauth_dir_v3_direct_dl_timeout-pinpoint=1479226024,1482077224.png?&lower_limit=&upper_limit=&size_x=800&size_y=200
tor_dirauth_dir_v3_direct_dl_timeout-pinpoint=1479226024-1482077224.png (38.2 KB) - added by dcf 22 months ago.
Re-upload because the Image macro can't handle filenames with commas.
userstats-relay-country-tr-2018-03-31-2018-06-29-off.png (25.8 KB) - added by dcf 4 months ago.
userstats-bridge-country-tr-2018-03-31-2018-06-29.png (24.9 KB) - added by dcf 4 months ago.
tr-tor-20180704.png (19.3 KB) - added by dcf 3 months ago.
tr-tor-asns-20180704.png (27.3 KB) - added by dcf 3 months ago.

Download all attachments as: .zip

Change History (32)

comment:1 Changed 22 months ago by mrphs

Keywords: censorship Turkey added

comment:2 Changed 22 months ago by dcf

Component: Core Tor/TorMetrics/Censorship analysis
Keywords: block tr added
Owner: set to metrics-team
Type: defecttask

comment:3 Changed 22 months ago by dcf

Tor metrics graphs show a large increase in users (both relay and bridge) in recent days, starting on 2016-12-12. I first heard of it from Joss Wright's twitter reporting a find of their anomaly detector.

The brief spike in relay users and sustained jump in bridge users on November 4 was the same date as government orders to block Tor and VPNs. The more recent increase on December 12, I don't know what might have caused.

https://metrics.torproject.org/userstats-relay-country.png?start=2016-09-18&end=2016-12-17&country=tr&events=off link

https://metrics.torproject.org/userstats-bridge-country.png?start=2016-09-18&end=2016-12-17&country=tr link

https://metrics.torproject.org/userstats-bridge-combined.png?start=2016-09-18&end=2016-12-17&country=tr link

Changed 22 months ago by dcf

Attachment: tr-tor-20161215.png added

comment:4 Changed 22 months ago by dcf

Summary: TR is blocking Tor connectionsTurkey blocking of direct connections, 2016-12-12

There is one ooniprobe in Turkey, and it reports that it has not been able to make a vanilla Tor connection starting on 2016-12-13 (T means success and F means failure).

test_start_time,success,probe_cc,transport_name,test_runtime
...
2016-12-11 00:01:04,T,TR,vanilla,91.0027029514
2016-12-12 00:01:01,T,TR,vanilla,80.7238359451
2016-12-13 09:16:40,F,TR,vanilla,300.1475861073
2016-12-13 10:20:30,F,TR,vanilla,300.1046140194
2016-12-15 17:14:54,F,TR,vanilla,300.1320888996
2016-12-16 00:00:35,F,TR,vanilla,300.1127798557


source code for graph

# mkdir -p reports
# wget -O - 'http://staging.measurements.ooni.io/api/v1/files?probe_cc=TR&test_name=vanilla_tor' | jq -r '.results[].download_url' | wget --no-http-keep-alive -P reports -c -i -
# (echo "test_start_time,success,probe_cc,transport_name,test_runtime"; jq -j '.test_start_time,",",if .test_keys.success then "T" else "F" end,",",.probe_cc,",",.test_keys.transport_name,",",.test_runtime,"\n"' reports/*) > tr.csv
library(ggplot2)
x <- read.csv("tr.csv")
x$test_start_time <- as.POSIXct(x$test_start_time, tz="GMT")
p <- ggplot(x)
p <- p + geom_linerange(aes(test_start_time, ymin=0, ymax=test_runtime, color=success), size=1.5, alpha=0.8, stat="identity")
p <- p + scale_x_datetime(date_breaks="1 month", date_minor_breaks="1 week")
p <- p + ggtitle("time to bootstrap Tor in Turkey")
p <- p + theme_bw()
p <- p + theme(legend.position="top")
filename <- sprintf("tr-tor-%s.png", strftime(max(x$test_start_time), "%Y%m%d"))
ggsave(filename, p, dpi=120, width=5, height=3)

comment:5 Changed 22 months ago by dcf

If what trdpi says is correct, that the firewall is breaking connections that are already partly underway, that could account for the seemingly increased number of users. Users are counted indirectly by counting directory requests. Connections might be getting broken after a directory request is sent but before the connection becomes useful. This is just a guess. The OONI reports say that bootstrapping failed at 10%, which is where you make a directory request, but you can also get to 10% even with no connectivity, I believe.

comment:6 Changed 22 months ago by cypherpunks

turk telekom, vanilla tor detected by ciphers, injected rst after client hello, split of segment works, obfs4 works.
tellcom, vanilla tor detected by ciphers, connection stalled after client hello, split of segment doesn't works, obfs4 works.

comment:7 Changed 22 months ago by cypherpunks

Connections might be getting broken after a directory request is sent but before the connection becomes useful.

No, it seems connection broken before successful tls handshake, client can't get server hello.

Changed 22 months ago by mrphs

Attachment: obfs3.jpg added

obfs3_connection-error.jpg

Changed 22 months ago by mrphs

Attachment: obfs4.jpg added

obfs4_connection-error

comment:8 Changed 22 months ago by mrphs

ISPs and ASnumbers tested:

Turksat uydunet: 47524
Turk Telekom: 9121

Here are some tests from inside TR on Pluggable Transports:

obfs4

12/17/2016 12:16:46 PM.900 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:16:46 PM.900 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:16:46 PM.900 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:16:46 PM.900 [NOTICE] Opening Socks listener on 127.0.0.1:9150 
12/17/2016 12:16:46 PM.900 [NOTICE] Renaming old configuration file to "C:\Users\X\Desktop\Tor Browser\Browser\TorBrowser\Data\Tor\torrc.orig.1" 
12/17/2016 12:16:46 PM.900 [NOTICE] Bootstrapped 5%: Connecting to directory server 
12/17/2016 12:16:47 PM.000 [NOTICE] Bootstrapped 10%: Finishing handshake with directory server 
12/17/2016 12:19:32 PM.800 [NOTICE] Closing no-longer-configured Socks listener on 127.0.0.1:9150 
12/17/2016 12:19:32 PM.800 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:19:32 PM.800 [NOTICE] Closing old Socks listener on 127.0.0.1:9150 
12/17/2016 12:19:32 PM.900 [NOTICE] Delaying directory fetches: DisableNetwork is set. 
12/17/2016 12:21:40 PM.800 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:21:40 PM.800 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:21:40 PM.800 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:21:40 PM.800 [NOTICE] Opening Socks listener on 127.0.0.1:9150 
12/17/2016 12:21:55 PM.900 [NOTICE] Bootstrapped 15%: Establishing an encrypted directory connection 
12/17/2016 12:21:57 PM.500 [NOTICE] Bootstrapped 20%: Asking for networkstatus consensus 
12/17/2016 12:22:00 PM.600 [NOTICE] new bridge descriptor 'LeifEricson' (fresh): $A09D536DD1752D542E1FBB3C9CE4449D51298239~LeifEricson at 83.212.101.3 
12/17/2016 12:22:00 PM.600 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no usable consensus. 
12/17/2016 12:22:03 PM.200 [NOTICE] Bootstrapped 25%: Loading networkstatus consensus 
12/17/2016 12:22:03 PM.900 [NOTICE] Bridge 'NX01' has both an IPv4 and an IPv6 address.  Will prefer using its IPv4 address (85.17.30.79:443) based on the configured Bridge address. 
12/17/2016 12:22:03 PM.900 [NOTICE] new bridge descriptor 'NX01' (fresh): $FC259A04A328A07FED1413E9FC6526530D9FD87A~NX01 at 85.17.30.79 
12/17/2016 12:22:03 PM.900 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no usable consensus. 
12/17/2016 12:22:08 PM.700 [WARN] Proxy Client: unable to connect to 154.35.22.9:12166 ("general SOCKS server failure") 
12/17/2016 12:22:08 PM.800 [WARN] Proxy Client: unable to connect to 154.35.22.13:16815 ("general SOCKS server failure") 
12/17/2016 12:22:11 PM.200 [NOTICE] new bridge descriptor 'noether' (fresh): $7B126FAB960E5AC6A629C729434FF84FB5074EC2~noether at 122.99.11.54 
12/17/2016 12:22:11 PM.200 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no usable consensus. 
12/17/2016 12:22:17 PM.700 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no usable consensus. 
12/17/2016 12:22:19 PM.800 [NOTICE] Bootstrapped 40%: Loading authority key certs 
12/17/2016 12:22:21 PM.800 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no recent usable consensus. 
12/17/2016 12:22:53 PM.100 [WARN] Problem bootstrapping. Stuck at 40%: Loading authority key certs. (DONE; DONE; count 10; recommendation warn; host 752CF7825B3B9EA6A98C83AC41F7099D67007EA5 at 128.245.60.50:443) 
12/17/2016 12:22:53 PM.100 [WARN] 12 connections have failed: 
12/17/2016 12:22:53 PM.100 [WARN]  8 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
12/17/2016 12:22:53 PM.100 [WARN]  2 connections died in state handshaking (proxy) with SSL state (No SSL object) 
12/17/2016 12:22:53 PM.100 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:22:53 PM.100 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [NOTICE] Closing no-longer-configured Socks listener on 127.0.0.1:9150 
12/17/2016 12:28:40 PM.300 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:28:40 PM.300 [NOTICE] Closing old Socks listener on 127.0.0.1:9150 
12/17/2016 12:28:40 PM.300 [WARN] Problem bootstrapping. Stuck at 40%: Loading authority key certs. (DONE; DONE; count 11; recommendation warn; host 00DC6C4FA49A65BD1472993CF6730D54F11E0DBB at 154.35.22.12:4304) 
12/17/2016 12:28:40 PM.300 [WARN] 13 connections have failed: 
12/17/2016 12:28:40 PM.300 [WARN]  9 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
12/17/2016 12:28:40 PM.300 [WARN]  2 connections died in state handshaking (proxy) with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [WARN] Problem bootstrapping. Stuck at 40%: Loading authority key certs. (DONE; DONE; count 12; recommendation warn; host 8FB9F4312E89E5C6223052AA525A122AFBC85D55 at 154.35.22.10:15937) 
12/17/2016 12:28:40 PM.300 [WARN] 14 connections have failed: 
12/17/2016 12:28:40 PM.300 [WARN]  10 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
12/17/2016 12:28:40 PM.300 [WARN]  2 connections died in state handshaking (proxy) with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [WARN] Problem bootstrapping. Stuck at 40%: Loading authority key certs. (DONE; DONE; count 13; recommendation warn; host A832D176ECD5C7C6B58825AE22FC4C90FA249637 at 154.35.22.11:80) 
12/17/2016 12:28:40 PM.300 [WARN] 15 connections have failed: 
12/17/2016 12:28:40 PM.300 [WARN]  10 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
12/17/2016 12:28:40 PM.300 [WARN]  2 connections died in state handshaking (proxy) with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state handshaking (TLS) with SSL state SSLv3 read finished A in HANDSHAKE 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:28:40 PM.300 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:28:40 PM.300 [NOTICE] Delaying directory fetches: DisableNetwork is set. 

obfs4_connection-error

After switching to obfs3

12/17/2016 12:29:44 PM.100 [WARN]  2 connections died in state handshaking (TLS) with SSL state SSLv3 read finished A in HANDSHAKE 
12/17/2016 12:29:44 PM.100 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:29:44 PM.100 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:29:44 PM.100 [WARN] Problem bootstrapping. Stuck at 40%: Loading authority key certs. (DONE; DONE; count 16; recommendation warn; host 7B126FAB960E5AC6A629C729434FF84FB5074EC2 at 122.99.11.54:443) 
12/17/2016 12:29:44 PM.100 [WARN] 27 connections have failed: 
12/17/2016 12:29:44 PM.100 [WARN]  11 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
12/17/2016 12:29:44 PM.100 [WARN]  11 connections died in state handshaking (proxy) with SSL state (No SSL object) 
12/17/2016 12:29:44 PM.100 [WARN]  3 connections died in state handshaking (TLS) with SSL state SSLv3 read finished A in HANDSHAKE 
12/17/2016 12:29:44 PM.100 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:29:44 PM.100 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:29:44 PM.100 [NOTICE] Delaying directory fetches: DisableNetwork is set. 
12/17/2016 12:48:55 PM.400 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:48:55 PM.400 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:48:55 PM.400 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:48:55 PM.400 [NOTICE] Opening Socks listener on 127.0.0.1:9150 
12/17/2016 12:49:06 PM.500 [NOTICE] new bridge descriptor 'ndnop0' (fresh): $1E05F577A0EC0213F971D81BF4D86A9E4E8229ED~ndnop0 at 109.105.109.163 
12/17/2016 12:49:06 PM.500 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no recent usable consensus. 
12/17/2016 12:49:07 PM.000 [NOTICE] new bridge descriptor 'ndnop2' (fresh): $4C331FA9B3D1D6D8FB0D8FBBF0C259C360D97E6A~ndnop2 at 109.105.109.163 
12/17/2016 12:49:07 PM.000 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no recent usable consensus. 
12/17/2016 12:49:08 PM.000 [NOTICE] new bridge descriptor 'Unnamed' (fresh): $AF9F66B7B04F8FF6F32D455F05135250A16543C9~Unnamed at 169.229.59.75 
12/17/2016 12:49:08 PM.000 [NOTICE] I learned some more directory information, but not enough to build a circuit: We have no recent usable consensus. 
12/17/2016 12:49:15 PM.700 [NOTICE] Bootstrapped 45%: Asking for relay descriptors 
12/17/2016 12:49:15 PM.700 [NOTICE] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/7221, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of exit bw = 0% of path bw.) 
12/17/2016 12:49:18 PM.100 [NOTICE] Bootstrapped 50%: Loading relay descriptors 
12/17/2016 12:49:45 PM.200 [WARN] Problem bootstrapping. Stuck at 50%: Loading relay descriptors. (DONE; DONE; count 17; recommendation warn; host A09D536DD1752D542E1FBB3C9CE4449D51298239 at 83.212.101.3:50002) 
12/17/2016 12:49:45 PM.200 [WARN] 28 connections have failed: 
12/17/2016 12:49:45 PM.200 [WARN]  12 connections died in state handshaking (TLS) with SSL state SSLv2/v3 read server hello A in HANDSHAKE 
12/17/2016 12:49:45 PM.200 [WARN]  11 connections died in state handshaking (proxy) with SSL state (No SSL object) 
12/17/2016 12:49:45 PM.200 [WARN]  3 connections died in state handshaking (TLS) with SSL state SSLv3 read finished A in HANDSHAKE 
12/17/2016 12:49:45 PM.200 [WARN]  1 connections died in state handshaking (Tor, v3 handshake) with SSL state SSL negotiation finished successfully in OPEN 
12/17/2016 12:49:45 PM.200 [WARN]  1 connections died in state connect()ing with SSL state (No SSL object) 
12/17/2016 12:50:11 PM.700 [NOTICE] Closing no-longer-configured Socks listener on 127.0.0.1:9150 
12/17/2016 12:50:11 PM.700 [NOTICE] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections. 
12/17/2016 12:50:11 PM.700 [NOTICE] Closing old Socks listener on 127.0.0.1:9150 
12/17/2016 12:50:11 PM.700 [NOTICE] Delaying directory fetches: DisableNetwork is set.

obfs3_connection-error.jpg

Last edited 22 months ago by mrphs (previous) (diff)

comment:9 Changed 22 months ago by mrphs

Keywords: UX added

comment:10 Changed 22 months ago by dcf

Subgraph OS found no blocking of vanilla Tor from a VPN in Turkey: https://twitter.com/subgraph/status/810222377292484608

Video: Testing to see if Tor is being blocked in Turkey from the desktop in Subgraph OS using Oz + coming OpenVPN support.
https://support.subgraph.com/turkey-tor.mp4

comment:11 in reply to:  7 Changed 22 months ago by dcf

Replying to cypherpunks:

Connections might be getting broken after a directory request is sent but before the connection becomes useful.

No, it seems connection broken before successful tls handshake, client can't get server hello.

Maybe it's only pluggable transport connections. comment:8 shows pluggable transports getting to around 50% bootstrapped. Especially if tor is retrying failed connections, that may account for an increase in the number of directory requests.

comment:12 Changed 22 months ago by dgoulet

Extra data point. Since December 13th, our directory authorities have seen a significant increase in consensus direct download timeout. Below is a graph that shows you the stat over time for the "div-v3-direct-dl-timeout" statistics reported by dirauth I'm collecting:

http://ygzf7uqcusp4ayjs.onion/static/dynazoom.html?cgiurl_graph=/munin-cgi/munin-cgi-graph&plugin_name=tor-health/tor-health/tor_dirauth_dir_v3_direct_dl_timeout&size_x=800&size_y=400&start_epoch=1479226024&stop_epoch=1482077224

(More high level view: http://ygzf7uqcusp4ayjs.onion/tor-health/tor-health/tor_dirauth_dir_v3_direct_dl_timeout.html)

It does matches the timeline of Turkey with this ticket.

Changed 22 months ago by dcf

Re-upload because the Image macro can't handle filenames with commas.

comment:13 in reply to:  12 Changed 22 months ago by dcf

Replying to dgoulet:

Extra data point. Since December 13th, our directory authorities have seen a significant increase in consensus direct download timeout. Below is a graph that shows you the stat over time for the "div-v3-direct-dl-timeout" statistics reported by dirauth I'm collecting:

Re-upload because the Image macro can't handle filenames with commas.

dgoulet, good find! This indeed makes me think that the metrics graphs are overcounting users in this case where consensus downloads are being interrupted. That's why the direct-user graphs show more users when in reality there are probably fewer.

#18203 is a proposal to base direct-user counts on directory responses, rather than directory requests. Doing that might solve this overcounting issue. (Apparently bridge counts are already based on responses.) In normal operation, it doesn't matter, because the number of requests should be very close to the number of responses—karsten showed this in a graph of responses vs. requests, which is almost a perfect y=x line. Presumably, if we made the same graph again today, there would be a lot of points beneath the line (more requests than responses, because some responses fail).

comment:14 Changed 22 months ago by dcf

Description: modified (diff)

dgoulet points to this Turkey Blocks article: https://turkeyblocks.org/2016/12/18/tor-blocked-in-turkey-vpn-ban/. They tested vanilla, obfs3, and obfs4, and also noted that the apparent rise in the metrics graphs may be caused by miscounting. I append some quotations.

The Turkey Blocks internet censorship watchdog has identified and verified that restrictions on the Tor anonymity network and Tor Browser are now in effect throughout Turkey.

Other circumvention methods, including Tor’s bridged modes built to evade similar restrictions imposed by the regime in Syria, as well as custom VPN deployments, continue to remain available to technically skilled users in the short-term.

Turkey Blocks finds that the Tor direct access mode is now restricted for most internet users throughout the country; Tor usage via bridges including obfs3 and obfs4 remains viable, although we see indications that obfs3 is being downgraded by some service providers with scope for similar on restrictions obfs4. The restrictions are being implemented in tandem with apparent degradation of commercial VPN service traffic.

Direct Tor access restrictions started around 12 December 2016. Tor’s direct mode is now entirely unusable via providers TTNet and UyduNet on the residential broadband connections we tested. Deep Packet Inspection (DPI) is likely used to disrupt the connection phase, which stalls around the 10% mark.

Connection is possible using obfs3 and obfs4 Tor bridges with both providers. While obfs4 is effective across all configurations, obfs3 intermittently fails with TTNet.

Where we expected a fall in usage corresponding to widespread reports of failure to access the Tor network, charts instead show a huge increase in Tor usage over the same period.

During tests we saw over a hundred connection attempts associated with a single user connection request, leading us to favour the theory Tor metrics have incorrectly counted these failed attempts in their overall usage tally.

comment:15 Changed 22 months ago by mrphs

The graph seems to be falling back down while bridge and PT users continue to go high. Though number of bridge users seems to be 1/3 of typical direct connections to Tor.

Last edited 22 months ago by mrphs (previous) (diff)

comment:16 in reply to:  15 Changed 22 months ago by dgoulet

Replying to mrphs:

The graph seems to be falling back down. I don't know if it's due to sudden censorship being lifted or what.

Just keep in mind that there is a roughly 3 days delay for metrics to actually show the right results for a specific day. So right now, we are seeing the ~Dec 17th and before results appearing.

comment:17 in reply to:  15 Changed 22 months ago by mrphs

Replying to mrphs:

Though number of bridge users seems to be 1/3 of typical direct connections to Tor.

I'm guessing this could be due to 2 factors: 1) People not knowing about bridges or how to configure them. Tor stops working they assume it's blocked and there's no way around and 2) Pluggable Transport being too slow for them to function.

comment:18 Changed 14 months ago by dcf

Resolution: wontfix
Status: newclosed

comment:19 Changed 4 months ago by dcf

19 months later, there's a very similar pattern in relay users from Turkey, jumping from 5k to 30k in about a day, on 2018-06-09. Is it another blocking event that's resulting in an illusory increase in the number of users? A Reddit user reports on 2018-06-29 that Tor is blocked.

https://metrics.torproject.org/userstats-relay-country.html?start=2018-03-31&end=2018-06-29&country=tr&events=off

https://metrics.torproject.org/userstats-bridge-country.html?start=2018-03-31&end=2018-06-29&country=tr&events=off

Changed 3 months ago by dcf

Attachment: tr-tor-20180704.png added

Changed 3 months ago by dcf

Attachment: tr-tor-asns-20180704.png added

comment:20 in reply to:  19 Changed 3 months ago by dcf

Replying to dcf:

19 months later, there's a very similar pattern in relay users from Turkey, jumping from 5k to 30k in about a day, on 2018-06-09. Is it another blocking event that's resulting in an illusory increase in the number of users?

Here are some OONI graphs (update of comment:4). Unfortunately there are no reports from the past two months, when the apparent big increase in users began.



# ooni-sync -xz -directory tr probe_cc=TR since=2016-06-01 test_name=vanilla_tor
# xz -dc tr/*.json.xz | jq -r '[.test_start_time,.test_keys.success,.probe_cc,.probe_asn,.test_runtime]|@csv' > tr.csv

library(ggplot2)
end.date <- as.POSIXct("2018-07-05", tz="GMT")
x <- read.csv("tr.csv", header=F, col.names=c("test_start_time", "success", "probe_cc", "probe_asn", "test_runtime"), stringsAsFactors=FALSE)
x$test_start_time <- as.POSIXct(x$test_start_time, tz="GMT")
x$success <- as.logical(x$success)
x[is.na(x$success), ]$success <- "NA"

p <- ggplot(x, aes(test_start_time, probe_asn, color=success))
p <- p + geom_point(alpha=0.5)
p <- p + scale_color_manual(values=c("TRUE"="blue", "FALSE"="red", "NA"="gray"))
p <- p + scale_x_datetime(limits=c(min(x$test_start_time), end.date), date_breaks="4 months", date_minor_breaks="1 month")
p <- p + ggtitle("bootstrap success in Turkey")
p <- p + theme_minimal()
p <- p + theme(legend.position="top")
filename <- sprintf("tr-tor-asns-%s.png", strftime(end.date, "%Y%m%d"))
ggsave(filename, p, width=5, height=2.5, dpi=120)

p <- ggplot(x, aes(test_start_time, ymin=0, ymax=test_runtime, color=success))
p <- p + geom_linerange(size=0.5, alpha=0.8, stat="identity")
p <- p + scale_color_manual(values=c("TRUE"="blue", "FALSE"="red", "NA"="gray"))
p <- p + scale_x_datetime(limits=c(min(x$test_start_time), end.date), date_breaks="4 months", date_minor_breaks="1 month")
p <- p + ggtitle("time to bootstrap Tor in Turkey")
p <- p + theme_minimal()
p <- p + theme(legend.position="top")
filename <- sprintf("tr-tor-%s.png", strftime(end.date, "%Y%m%d"))
ggsave(filename, p, width=5, height=3, dpi=120)
Note: See TracTickets for help on using tickets.