Opened 3 years ago

Last modified 21 months ago

#20007 needs_information defect

Sandbox causing crash when setting HidServAuth when there is a hidden service running

Reported by: segfault Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.2.9.2-alpha
Severity: Normal Keywords: regression, crash, tor-hs, 029-proposed, sandbox, seccomp2, linux, authentication, hs-auth
Cc: segfault, asn Actual Points:
Parent ID: Points: 1
Reviewer: Sponsor:

Description (last modified by segfault)

When the sandbox is enabled and there is a hidden service configured, setting HidServAuth via SETCONF results in a permission error.

Steps to reproduce:

Start Tor with a hidden service:

/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --RunAsDaemon 0 --Log debug --CookieAuthentication 0  --Sandbox 1 --HiddenServiceDir /var/lib/tor/hidden_service/ --HiddenServicePort 80

Try setting HidServAuth via the control port:

echo "AUTHENTICATE
SETCONF HidServAuth=\"prkszpeygn2a3kxo.onion iGwsXkMwZEHuq/0YCD6IGQ\"" | nc -U /var/run/tor/control

Output:

250 OK
513 Unacceptable option value: Failed to configure rendezvous options. See logs for details.

Log:

Aug 27 15:31:55.000 [warn] Directory /var/lib/tor/hidden_service/ cannot be read: Permission denied
Aug 27 15:31:55.000 [warn] Controller gave us config lines that didn't validate: Failed to configure rendezvous options. See logs for details.

If we start Tor without a hidden service or without the sandbox, it works without errors:

Without hidden service:

/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --RunAsDaemon 0 --Log debug --CookieAuthentication 0  --Sandbox 1

or without sandbox:

/usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --RunAsDaemon 0 --Log debug --CookieAuthentication 0  --Sandbox 0 --HiddenServiceDir /var/lib/tor/hidden_service/ --HiddenServicePort 80

Set HidServAuth via the control port:

echo "AUTHENTICATE
SETCONF HidServAuth=\"prkszpeygn2a3kxo.onion iGwsXkMwZEHuq/0YCD6IGQ\"" | nc -U /var/run/tor/control

Output:

250 OK
250 OK

Child Tickets

Change History (15)

comment:1 Changed 3 years ago by nickm

Component: - Select a componentCore Tor/Tor

comment:2 Changed 3 years ago by teor

Status: newneeds_information

What happens when you turn sandbox off and hidden service auth on?

comment:3 Changed 3 years ago by asn

Cc: asn added

comment:4 Changed 3 years ago by segfault

Description: modified (diff)
Status: needs_informationnew

What happens when you turn sandbox off and hidden service auth on?

Without the sandbox it works as expected. I updated the description to include this case.

comment:5 Changed 3 years ago by teor

Keywords: regression tor-hs added
Status: newneeds_information

I don't have enough information to diagnose this issue.

What are you really trying to do?

  • HidServAuth is a client option, but you are running a hidden service. Did you mean to use HiddenServiceAuthorizeClient instead?

What is actually happening?

  • What is the full log? I'd like to see notice level.
  • Does the hidden service work with the Sandbox on startup (without the SETCONF)? Is this simply a permissions problem on the directory?

What is actually causing the issue?

  • rend_parse_service_authorization() parses client HidServAuth entries, it doesn't read any service HiddenServiceDirectory files. So it might be that any SETCONF is your problem here, not specifically HidServAuth. What happens when you issue a SETCONF ClientOnly=1 instead of HidServAuth? (ClientOnly is ignored on clients, it has no effect).
  • rend_services_add_filenames_to_lists() implements the Sandbox for each HiddenServiceDirectory, using the following lines:
          rend_service_add_filenames_to_list(open_lst, s);
          smartlist_add(stat_lst, tor_strdup(s->directory));
    

As far as I can see, this code is working correctly, and should make the hidden service directory available via the sandbox at startup. Maybe this directory isn't being added to the sandbox at startup? Maybe SETCONF isn't using the sandbox-approved way to access the directory?

  • The log error you provided is logged by check_private_dir(), which is called by the HiddenServiceDirectory-handling code, not the HidServAuth code.
      /* Open directory.
       * O_NOFOLLOW to ensure that it does not follow symbolic links */
      fd = open(sandbox_intern_string(dirname), O_NOFOLLOW);
    
      /* Was there an error? Maybe the directory does not exist? */
      if (fd == -1) {
    
        if (errno != ENOENT) {
          /* Other directory error */
          log_warn(LD_FS, "Directory %s cannot be read: %s", dirname,
                   strerror(errno));
          return -1;
        }
    

comment:6 Changed 3 years ago by segfault

What are you really trying to do?
HidServAuth is a client option, but you are running a hidden service. Did you mean to use HiddenServiceAuthorizeClient instead?

No, I did not confuse HidServAuth with HiddenServiceAuthorizeClient. I'm developing the Tails Server, which is an application run inside Tails to start hidden services (with client authentication). There is also a small client application, which primarily sets the client authentication cookie via `SETCONF HidServAuth`, to be able to connect to a service started via Tails Server. Now it's possible that a user who runs a hidden service also wants to connect to another hidden service, which is when this issue would occur.

What is actually happening?
What is the full log? I'd like to see notice level.

The two warn level messages are the only ones related to this issue. Here is the full notice level log:

# /usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --[wiki:RunAsDaemon] 0 --Log notice --[wiki:CookieAuthentication] 0  --Sandbox 1 --[wiki:HiddenServiceDir] /var/lib/tor/hidden_service/ --[wiki:HiddenServicePort] 80
Sep 09 14:42:01.037 [notice] Tor 0.2.9.2-alpha (git-0b1c884a450cad98) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2h and Zlib 1.2.8.
Sep 09 14:42:01.037 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Sep 09 14:42:01.037 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Sep 09 14:42:01.037 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Sep 09 14:42:01.037 [notice] Read configuration file "/etc/tor/torrc".
Sep 09 14:42:01.044 [notice] Opening Socks listener on 127.0.0.1:9050
Sep 09 14:42:01.044 [notice] Opening Socks listener on 127.0.0.1:49050
Sep 09 14:42:01.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Sep 09 14:42:01.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Sep 09 14:42:01.000 [notice] Bootstrapped 0%: Starting
Sep 09 14:42:01.000 [notice] Bootstrapped 80%: Connecting to the Tor network
Sep 09 14:42:02.000 [notice] Opening Control listener on /var/run/tor/control
Sep 09 14:42:02.000 [notice] Bootstrapped 85%: Finishing handshake with first hop
Sep 09 14:42:02.000 [notice] Bootstrapped 90%: Establishing a Tor circuit
Sep 09 14:42:03.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
Sep 09 14:42:03.000 [notice] Bootstrapped 100%: Done
Sep 09 14:42:06.000 [notice] New control connection opened.
Sep 09 14:42:06.000 [warn] Directory /var/lib/tor/hidden_service/ cannot be read: Permission denied
Sep 09 14:42:06.000 [warn] Controller gave us config lines that didn't validate: Failed to configure rendezvous options. See logs for details.

Does the hidden service work with the Sandbox on startup (without the SETCONF)? Is this simply a permissions problem on the directory?

The hidden service works with the Sandbox without the SETCONF, so this is not only a permissions problem.

What is actually causing the issue?

rend_parse_service_authorization() parses client HidServAuth entries, it doesn't read any service HiddenServiceDirectory files. So it might be that any SETCONF is your problem here, not specifically HidServAuth. What happens when you issue a SETCONF ClientOnly=1 instead of HidServAuth? (ClientOnly is ignored on clients, it has no effect).

Indeed, it seems like every SETCONF fails like this (I tested it with SETCONF ClientOnly=1 and SETCONF Log debug). I'm sorry I missed this - I discovered this issue while investigating #20003, which is an issue regarding the sandbox and HidServAuth (but it does not occur in recent tor versions anymore). So I somehow assumed that this issue was also about HidServAuth.

rend_services_add_filenames_to_lists() implements the Sandbox for each HiddenServiceDirectory, using the following lines:

rend_service_add_filenames_to_list(open_lst, s);smartlist_add(stat_lst, tor_strdup(s->directory));

As far as I can see, this code is working correctly, and should make the hidden service directory available via the sandbox at startup. Maybe this directory isn't being added to the sandbox at startup? Maybe SETCONF isn't using the sandbox-approved way to access the directory?

These seem like probable causes for this issue. Should I investigate this further? (I don't have too much time to work on this stuff these days, and have a lot of other issues to work on.)

Note that the output from above is not from Tails, but from a vanilla Debian Stretch (I wanted to make sure that this is not a Tails specific issue), but I also tested it in Tails today with the same Tor version (0.2.9.2-alpha) and the output differs. In my Debian Stretch there are only the warning messages, but tor doesn't actually crash. In Tails, there are not warning messages, tor just crashes with the following traceback:

============================================================ T= 1473422698
(Sandbox) Caught a bad syscall attempt (syscall open)
/usr/bin/tor(+0x15990c)[0xf769790c]
/lib/i386-linux-gnu/libpthread.so.0(__open64+0x3d)[0xf71d8dbd]
/lib/i386-linux-gnu/libpthread.so.0(__open64+0x3d)[0xf71d8dbd]
/usr/bin/tor(check_private_dir+0x58)[0xf7690558]
/usr/bin/tor(rend_config_services+0x402)[0xf75a39a2]
/usr/bin/tor(+0xd84d3)[0xf76164d3]
/usr/bin/tor(options_trial_assign+0x8d)[0xf761a01d]
/usr/bin/tor(+0xfcf31)[0xf763af31]
/usr/bin/tor(connection_control_process_inbuf+0x8a3)[0xf763ef03]
/usr/bin/tor(+0xe1d4d)[0xf761fd4d]
/usr/bin/tor(+0xeb11b)[0xf762911b]
/usr/bin/tor(+0x32c09)[0xf7570c09]
/usr/lib/i386-linux-gnu/libevent-2.0.so.5(event_base_loop+0x73d)[0xf745a36d]
/usr/bin/tor(do_main_loop+0x2b7)[0xf7571d27]
/usr/bin/tor(tor_main+0xdb5)[0xf75746f5]
/usr/bin/tor(main+0x35)[0xf756d365]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf706b723]
/usr/bin/tor(+0x2f3c4)[0xf756d3c4]__

It seems like in my Debian Stretch the permission error is caught somewhere, but in Tails it is not.

comment:7 Changed 3 years ago by segfault

Status: needs_informationnew

Meta: Since I'm new to Tor but plan to continue contributing, please tell me if I'm not using the tracker as expected. For example, should I reset the status to "new" or assign it to someone (who?)? Is there a guideline for contributors?

comment:8 in reply to:  7 Changed 3 years ago by teor

Keywords: crash 029-proposed TorCoreTeam201609 added
Milestone: Tor: 0.2.???
Points: 1

As this is a possible regression, and a crash on some OSs, I'm proposing it for 0.2.9.

Replying to segfault:

Meta: Since I'm new to Tor but plan to continue contributing, please tell me if I'm not using the tracker as expected. For example, should I reset the status to "new" or assign it to someone (who?)?

"new" is fine.
Assignment is only used once we're sure who is going to work on it.

Is there a guideline for contributors?

I'm not sure. We should write this kind of thing down somewhere around here:

https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam

comment:9 in reply to:  6 Changed 3 years ago by teor

Replying to segfault:

What are you really trying to do?
HidServAuth is a client option, but you are running a hidden service. Did you mean to use HiddenServiceAuthorizeClient instead?

No, I did not confuse HidServAuth with HiddenServiceAuthorizeClient. I'm developing the Tails Server, which is an application run inside Tails to start hidden services (with client authentication). There is also a small client application, which primarily sets the client authentication cookie via `SETCONF HidServAuth`, to be able to connect to a service started via Tails Server. Now it's possible that a user who runs a hidden service also wants to connect to another hidden service, which is when this issue would occur.

I'm not sure if we recommend running hidden services and hidden service clients from the same Tor instance. I think there are some anonymity implications. But not so severe that we prevent it entirely.

...

What is actually causing the issue?

rend_parse_service_authorization() parses client HidServAuth entries, it doesn't read any service HiddenServiceDirectory files. So it might be that any SETCONF is your problem here, not specifically HidServAuth. What happens when you issue a SETCONF ClientOnly=1 instead of HidServAuth? (ClientOnly is ignored on clients, it has no effect).

Indeed, it seems like every SETCONF fails like this (I tested it with SETCONF ClientOnly=1 and SETCONF Log debug). I'm sorry I missed this - I discovered this issue while investigating #20003, which is an issue regarding the sandbox and HidServAuth (but it does not occur in recent tor versions anymore). So I somehow assumed that this issue was also about HidServAuth.

OK, so we are clearly not sandboxing that directory correctly, or SETCONF is doing something unnecessary.

rend_services_add_filenames_to_lists() implements the Sandbox for each HiddenServiceDirectory, using the following lines:

rend_service_add_filenames_to_list(open_lst, s);smartlist_add(stat_lst, tor_strdup(s->directory));

As far as I can see, this code is working correctly, and should make the hidden service directory available via the sandbox at startup. Maybe this directory isn't being added to the sandbox at startup? Maybe SETCONF isn't using the sandbox-approved way to access the directory?

These seem like probable causes for this issue. Should I investigate this further? (I don't have too much time to work on this stuff these days, and have a lot of other issues to work on.)

That would be helpful, but others will step in at some point if you can't.

Note that the output from above is not from Tails, but from a vanilla Debian Stretch (I wanted to make sure that this is not a Tails specific issue), but I also tested it in Tails today with the same Tor version (0.2.9.2-alpha) and the output differs. In my Debian Stretch there are only the warning messages, but tor doesn't actually crash. In Tails, there are not warning messages, tor just crashes with the following traceback:

============================================================ T= 1473422698
(Sandbox) Caught a bad syscall attempt (syscall open)
/usr/bin/tor(+0x15990c)[0xf769790c]
/lib/i386-linux-gnu/libpthread.so.0(__open64+0x3d)[0xf71d8dbd]
/lib/i386-linux-gnu/libpthread.so.0(__open64+0x3d)[0xf71d8dbd]
/usr/bin/tor(check_private_dir+0x58)[0xf7690558]
/usr/bin/tor(rend_config_services+0x402)[0xf75a39a2]
/usr/bin/tor(+0xd84d3)[0xf76164d3]
/usr/bin/tor(options_trial_assign+0x8d)[0xf761a01d]
/usr/bin/tor(+0xfcf31)[0xf763af31]
/usr/bin/tor(connection_control_process_inbuf+0x8a3)[0xf763ef03]
/usr/bin/tor(+0xe1d4d)[0xf761fd4d]
/usr/bin/tor(+0xeb11b)[0xf762911b]
/usr/bin/tor(+0x32c09)[0xf7570c09]
/usr/lib/i386-linux-gnu/libevent-2.0.so.5(event_base_loop+0x73d)[0xf745a36d]
/usr/bin/tor(do_main_loop+0x2b7)[0xf7571d27]
/usr/bin/tor(tor_main+0xdb5)[0xf75746f5]
/usr/bin/tor(main+0x35)[0xf756d365]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf706b723]
/usr/bin/tor(+0x2f3c4)[0xf756d3c4]__

It seems like in my Debian Stretch the permission error is caught somewhere, but in Tails it is not.

I would love to see debug symbols here.
Can you install tor-dbg?

Are the directory permissions for /var/lib/tor/hidden_service/ and /var/lib/tor/ identical on Stretch and Tails?
Is the owner of /var/lib/tor/hidden_service/ and /var/lib/tor/ the same user tor runs as on both Stretch and Tails?
(Tor might act differently depending on these permissions.)

Is the sandboxing config in the kernel identical?
(There might be a deny vs terminate setting.)

An error on SETCONF is bad, but a crash is really nasty.

comment:10 Changed 3 years ago by teor

Status: newneeds_information

comment:11 Changed 2 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:12 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:13 Changed 22 months ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:14 Changed 22 months ago by nickm

Keywords: TorCoreTeam201609 removed

comment:15 Changed 21 months ago by nickm

Keywords: sandbox seccomp2 linux authentication hs-auth added
Note: See TracTickets for help on using tickets.