#24350 closed defect (user disappeared)

A fresh compiled tor does not honor MaxCircuitDirtiness

Reported by: Zakhar Owned by:
Priority: Medium Milestone: Tor: 0.3.4.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-client, 034-triage-20180328
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I am having a strange behavior when compiling tor, it does not take into account the MaxCircuitDirtiness I have set in the configuration... nor the default value that is supposed to be 10 minutes.

In fact, it changes identity every 5 minutes!

I tried both the Ubuntu version (0.2.9.11) and the last official one from tor (0.3.1.18)

What is even stranger it that with the Ubuntu repo binary, it works fine.

Steps to reproduce:

  • Start a Live Ubuntu 16.04.3 (x64 in my case) [so that the behavior is easy to reproduce]

Execute that script (as root... no problem we are in a Live session, all is gone in the end), it will install the necessary packages to compile (otherwise configure will complain on libevent-dev, then on libssl-dev), download sources and compile both versions.

#!/bin/sh

cd /tmp
echo  'deb-src http://archive.ubuntu.com/ubuntu/ xenial-updates universe' >>'/etc/apt/sources.list'
apt-get update
apt-get install -y libevent-core-2.0-5 libevent-extra-2.0-5 libevent-openssl-2.0-5 libevent-pthreads-2.0-5 libevent-dev libssl-doc zlib1g-dev libssl-dev
gpg --keyserver keyserver.ubuntu.com --recv-keys 64792D67
gpg --no-default-keyring -a --export 64792D67 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
apt-get source tor
cd "tor-0.2.9.11"
./configure
make

cd /tmp
wget https://www.torproject.org/dist/tor-0.3.1.8.tar.gz
tar xvzf tor-0.3.1.8.tar.gz
cd "tor-0.3.1.8"
./configure
make

You will have some warnings like:

ar: `u' modifier ignored since `D' is the default (see `U')

I am assuming these warnings are benign looking a 'u' and 'D' options in the man of ar.
You will get both versions of tor compiled as documented by the README.
Save them before rebooting.

Now whichever version you try, here is the output tracking the change of IP:

Set MaxCircuitDirtiness to 30 minutes for example with:

sudo echo "MaxCircuitDirtiness 1800" >>/etc/tor/torrc

Then test the ip we have through tor

$ while :; do line="$( date +%H:%M ) == $( curl -s http://whatismyip.akamai.com/ )"; echo "$line"; sleep 60; done
19:27 == 185.100.84.82
19:28 == 185.100.84.82
19:29 == 185.100.84.82
19:30 == 185.100.84.82
19:31 == 185.100.84.82
19:32 == 204.85.191.30
19:34 == 204.85.191.30
19:35 == 204.85.191.30
19:36 == 204.85.191.30
19:37 == 204.85.191.30
19:38 == 192.160.102.168
19:39 == 192.160.102.168
19:40 == 192.160.102.168
19:41 == 192.160.102.168
19:42 == 192.160.102.168
19:43 == 163.172.101.137
19:44 == 163.172.101.137
19:45 == 163.172.101.137
19:46 == 163.172.101.137
19:47 == 163.172.101.137
19:48 == 62.210.105.116

(This is done inside a VM with transparent proxying to Tor, see "middlebox").

We can see that it is changing ip exactly every 5 minutes.

When doing the same exit ip test with the stock binary version of Ubuntu that you get with:

sudo apt-get install tor

... all works well, it changes ip every 30 minutes as the configuration says.

Questions:
So... is there a magic trick to compile so that MaxCircuitDirtiness is taken into account ? If so, that would be a documentation enhancement request. I am thinking something like a flag: compile for "debug"/compile for "production" -didn't find that in the documentation!

Should I ask instead on the Ubuntu Launchpad (apparently they are clever enough to have figured out a way to make it work!)

We can however notice a difference between the versions we compiled and the binary from Ubuntu repo: size!
That is (I am guessing) because the tor we compiled has all the symbols.
But if you do (which is undocumented!):

strip --strip-unneeded tor

you get about the same size of stock binary. Anyway, I don't think having the symbols should change behaviors -except in case you have very very little RAM, which is not my case!-

MaxCircuitDirtiness is not such a big issue per se, but I am afraid that if we have those kind of "silent tricky bugs" (nothing in the log of tor) when compiling ourselves, there might be other more serious bugs that could compromise anonymity.

Child Tickets

Change History (17)

comment:1 Changed 16 months ago by nickm

Are you sure that the "freshly compiled Tor" is actually looking for its configuration file in /etc/tor/torrc? (It should say in its logs where it's getting its configuration from.)

comment:2 Changed 16 months ago by arma

Zakhar: yeah, your self-built tor binary will look at /usr/local/etc/tor/torrc.

If you want it to use /etc/tor/torrc, you'll need to build it with the various build options that the deb uses. (The simpler answer is to just use the deb.)

comment:3 Changed 16 months ago by Zakhar

Yes I have the right config file, since it is started with systemd as show with this ps:

$ ps aux | grep /usr/bin/tor
debian-+  3729  0.1  0.0  59224 30516 ?        Ss   08:08   0:00 /usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0
$ /usr/bin/tor --version
Tor version 0.3.1.8 (git-ad5027f7dc790624).
$ grep 'MaxCir' /etc/tor/torrc
MaxCircuitDirtiness 1800
$ tail -n 30 /var/log/tor/log
Nov 19 08:18:28.000 [notice] Tor 0.3.1.8 (git-ad5027f7dc790624) opening log file.
Nov 19 08:18:28.455 [notice] Tor 0.3.1.8 (git-ad5027f7dc790624) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g, Zlib 1.2.8, Liblzma N/A, and Libzstd N/A.
Nov 19 08:18:28.455 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Nov 19 08:18:28.455 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Nov 19 08:18:28.455 [notice] Read configuration file "/etc/tor/torrc".
(... skipping lines showing other details of my configuration for privacy reason ...)
Nov 19 08:18:30.000 [notice] Bootstrapped 100%: Done

You see here it is tor version 0.3.1.8 which is not supposed to be in the repository of Ubuntu 16.04, so it is indeed my compiled version that is running, and since the service files of systemd force the configuration file to /etc/tor/torrc, the configuration file seems correct too. The log shows that it reads the right file in which we find a MaxCircuitDirtiness 1800 directive as shown by the grep.

And by the way, if I hadn't the right config file, I guess I should have an IP changing every 10 minutes since it is the defaut, and not every 5 minutes!

To simplify the test of the ip exit and run it on the same machine where tor is running, you can do it through socks:

$ while :; do line="$( date +%H:%M ) == $( curl -s --socks5 '127.0.0.1:9050' 'http://whatismyip.akamai.com' )"; echo "$line"; sleep 60; done
08:36 == 185.104.120.7
08:37 == 185.104.120.7
08:38 == 185.104.120.7
08:39 == 193.15.16.4
08:40 == 193.15.16.4
08:41 == 193.15.16.4
08:42 == 193.15.16.4
08:43 == 193.15.16.4
08:44 == 193.90.12.115
08:45 == 193.90.12.115
08:46 == 193.90.12.115
08:47 == 193.90.12.115
08:48 == 193.90.12.115
08:49 == 91.219.237.244

As you can see... always 5 minutes! Not 10 min default, not what is in the config file.

Last edited 16 months ago by Zakhar (previous) (diff)

comment:4 in reply to:  2 Changed 16 months ago by Zakhar

Replying to arma:

Zakhar: yeah, your self-built tor binary will look at /usr/local/etc/tor/torrc.

If you want it to use /etc/tor/torrc, you'll need to build it with the various build options that the deb uses. (The simpler answer is to just use the deb.)

I am doing so because I need to patch for ipv6 link-local reasons (see my other ticket: https://trac.torproject.org/projects/tor/ticket/23819).

Since I saw this strange behavior after patching, I compiled it without my patch to check if this strange thing is still there and eliminate a side effect due to my patching.

Unfortunately, I am observing the same strange behavior with an "unpatched fresh compile".

Last edited 16 months ago by Zakhar (previous) (diff)

comment:5 in reply to:  1 Changed 16 months ago by Zakhar

Replying to nickm:

Are you sure that the "freshly compiled Tor" is actually looking for its configuration file in /etc/tor/torrc? (It should say in its logs where it's getting its configuration from.)

That was a possibility (unfortunately... see above).
Also, since I am using tor as middlebox, if it was reading a non existent torrc, I would have noticed since my other special directives would also be missing. I can confirm it binds correctly where I have asked to in the same config file with MaxCircuitDirtiness.

Indeed looking at the strings in the binary (1st one is the Ubuntu repo binary, second the one I compiled):

$ strings -d tor | grep -e '/.*torrc'
/etc/tor/torrc-defaults
/etc/tor/torrc
~/.torrc
$ strings -d tor_0.3.1.8 | grep -e '/.*torrc'
/usr/local/etc/tor/torrc
~/.torrc
/usr/local/etc/tor/torrc-defaults

It looks like the Ubuntu build removes /usr/local from them.

By the way... I failed to find "build options" for tor (I am sure there are some, as any "big" project !)

Would you have a pointer on the documentation that lists them and how to implement them before building (./configure and make)?

Last edited 16 months ago by Zakhar (previous) (diff)

comment:6 Changed 16 months ago by Zakhar

Here is more investigation !

I set the MaxCircuitDirtiness to 240 (4 minutes) and here is what I get:

$ while :; do line="$( date +%H:%M ) == $( curl -s --socks5 '127.0.0.1:9050' 'http://whatismyip.akamai.com' )"; echo "$line"; sleep 60; done
14:20 == 37.187.129.166
14:21 == 46.183.218.199
14:22 == 46.183.218.199
14:23 == 46.183.218.199
14:24 == 46.183.218.199
14:25 == 51.15.72.53
14:26 == 162.243.166.137
14:27 == 162.243.166.137
14:28 == 162.243.166.137
14:29 == 162.243.166.137
14:30 == 192.160.102.170
14:31 == 197.231.221.211
14:32 == 197.231.221.211
14:33 == 197.231.221.211
14:34 == 197.231.221.211
14:35 == 93.115.95.202
14:36 == 185.100.86.167
14:37 == 185.100.86.167
14:38 == 185.100.86.167
14:39 == 185.100.86.167
14:40 == 185.104.120.2
14:41 == 204.194.29.4
14:42 == 204.194.29.4

A very regular pattern where we have 4 minutes on an identity, then 1 minute on another, and the same pattern again.

So I guess, unrelated to tor, there is a parameter either in libevent_dev or libssl_dev that says: timeout is 5 minutes!

So my compiled tor is indeed reading the MaxCircuitDirtiness, but gets timeouts/error every 5 minutes that make it switch identity.

Then, unless you have already seen this strange behavior and have a clue on how to fix it, I think I'll better ask on Ubuntu/Debian related forums since they apparently have already fixed it.

Although if such tricks are "well known", it would be better to document them in tor's "build how-to"... along with any other "well known" nasty tricks we might have once I'll have figured out where does this 5 minutes timeout comes from!

Last edited 16 months ago by Zakhar (previous) (diff)

comment:7 Changed 16 months ago by Zakhar

Question filed on AskUbuntu: https://askubuntu.com/questions/978064/where-to-find-compile-build-configuration-problem-with-tor-from-source

But if you know how to compile correctly (I mean with the right settings/configuration/parameters) don't hesitate to drop a line!

Last edited 16 months ago by Zakhar (previous) (diff)

comment:8 Changed 16 months ago by Zakhar

Priority: MediumLow
Severity: NormalMinor
Type: defectenhancement

[(Almost) Solved]

I make this a low priority documentation enhancement!

The README says that we must do

To build Tor from source:
        ./configure && make && make install

Proposition:

To build Tor from source:
        ./configure && make && make install

   (or use the build procedure specific to your distribution)

Indeed, the documentation could also hint other specific ways of building/compiling according to the distribution. Otherwise the trouble is that with Debian/Ubuntu it works but in the end you get an executable with strange behavior like the one above.

For those who might search, here is how to make it work instead of ./configure && make:

apt-get install build-essentials fakeroot dpkg-dev
apt-get build-dep tor
fakeroot debian/rules binary

In the end you get a .deb repackaged with your newly compiled source.

  • same size as the Ubuntu stock binary
  • same default config files
  • works with MaxCircuitDirtiness

Interesting link: https://www.cyberciti.biz/faq/rebuilding-ubuntu-debian-linux-binary-package/

(Almost) because I am now wondering how I will get a working Debian Package when I'll try the 0.3.1.8 (or more recent) from the tor git/tar.gz

Also a small signature issue since I don't have the private key of the initial Debian/Ubuntu maintainer, I can't obviously re-sign the package... but that is non tor related!

comment:9 Changed 16 months ago by Zakhar

Component: Core Tor/TorCore Tor/DocTor
Owner: set to atagar

comment:10 Changed 16 months ago by asn

Component: Core Tor/DocTorCore Tor/Tor
Keywords: docs added

comment:11 Changed 16 months ago by atagar

Owner: atagar deleted
Status: newassigned

comment:12 in reply to:  8 Changed 16 months ago by teor

Keywords: tor-client added; docs removed
Milestone: Tor: 0.3.3.x-final
Priority: LowMedium
Severity: MinorNormal
Type: enhancementdefect

It looks like this might be a bug in the new 0.3.x guard code.

Is there some reason that our circuits change every 5 minutes?
Do we build a new circuit every 5 minutes?
Do new streams go on new circuits in 0.3.1, but old circuits in earlier versions?

Replying to Zakhar:

[(Almost) Solved]

I make this a low priority documentation enhancement!

The README says that we must do

To build Tor from source:
        ./configure && make && make install

Proposition:

To build Tor from source:
        ./configure && make && make install

   (or use the build procedure specific to your distribution)

Indeed, the documentation could also hint other specific ways of building/compiling according to the distribution. Otherwise the trouble is that with Debian/Ubuntu it works but in the end you get an executable with strange behavior like the one above.

For those who might search, here is how to make it work instead of ./configure && make:

apt-get install build-essentials fakeroot dpkg-dev
apt-get build-dep tor
fakeroot debian/rules binary

In the end you get a .deb repackaged with your newly compiled source.

  • same size as the Ubuntu stock binary
  • same default config files
  • works with MaxCircuitDirtiness

Interesting link: https://www.cyberciti.biz/faq/rebuilding-ubuntu-debian-linux-binary-package/

(Almost) because I am now wondering how I will get a working Debian Package when I'll try the 0.3.1.8 (or more recent) from the tor git/tar.gz

Also a small signature issue since I don't have the private key of the initial Debian/Ubuntu maintainer, I can't obviously re-sign the package... but that is non tor related!

Please open a separate ticket for this issue.
We lose track of issues when there is more than one issue in a ticket.

comment:13 Changed 16 months ago by dgoulet

Also lets consider #24228. A fresh relay without a Guard state will open/close many circuits to ramp up as quickly as possible the CBT (circuit build timeout) and it don't honor MaxCircuitDirtiness but rather a random interval usually around 60 seconds.

It should stabilize after 100 circuits are created and the CBT is then used normally. It takes ~30 minutes.

comment:14 Changed 16 months ago by teor

Status: assignednew

Mark all tickets that are assigned to nobody as "new".

comment:15 Changed 14 months ago by nickm

Milestone: Tor: 0.3.3.x-finalTor: 0.3.4.x-final
Status: newneeds_information

comment:16 Changed 12 months ago by nickm

Keywords: 034-triage-20180328 added

comment:17 Changed 10 months ago by nickm

Resolution: user disappeared
Status: needs_informationclosed
Note: See TracTickets for help on using tickets.