Opened 12 months ago

Last modified 8 months ago

#26181 new defect

Apparmor + systemd failures when loading included service files + DisableAllSwap Fix

Reported by: d3m0nkingx Owned by:
Priority: Very High Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.3.3.6
Severity: Normal Keywords: apparmor, 035-removed-20180711
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Environment:

Ubuntu 16.04.4 (linux kernel 4.16.0-041600-generic)

Tor version 0.3.3.6 (git-c9903102c98cd028).

systemd version 229

+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN


I had to completely redo the service files in order to get it to actually load with systemd

Firstly I disabled the init script for tor in /etc/init.d/tor:

update-rc.d disable tor

Then made changes to tor.service torrc and system_tor files as everything was basically conflicting with each other, mainly with apparmor denying access to /var/run because the apparmor system_tor authorized paths for the cookie file was not matching with what the torrc default path is, so I had to update either system_tor or change the path in torrc this file to reflect this, so I went with updating apparmor's profile as well as adding a couple more variations of the cookie file. 

I had to also add the following lines to tor.service:

RuntimeDirectory=tor

RuntimeDirectoryMode=0700

In order for tor to actually initialize and create the files in the /var/run directory as anything with permissions allowing more than the owner would issue a warning and fail.


I also added some of the configuration from /etc/default/tor to torrc because that file was only used by the /etc/init.d/tor scrip, which is not touched when using systemd to start tor.

From what I tested, tor will not start with systemd if defining user/group as any other than root when it tries to create/read/write to the /var/run/tor directory, as I get the warning for example setting  'User demon' in torrc and User=demon Group=sudo in tor.service results in: 

  • [notice] Opening Socks listener on /var/run/tor/socks
  • [warn] Unable to chown() /var/run/tor/socks socket: Operation not permitted.
  • [notice] Opening Socks listener on 127.0.0.1:9050
  • [warn] /var/run/tor is not owned by this user (root, 0) but by demon (1000). Perhaps you are running Tor as the wrong user?
  • [warn] Before Tor can create a control socket in "/var/run/tor/socks", the directory "/var/run/tor" needs to exist, and to be accessible only by the user and group account that is running Tor.  (On some Unix systems, anybody who can list a socket can connect to it, so Tor is being careful.)
  • [notice] Closing partially-constructed Socks listener on 127.0.0.1:9050
  • [warn] Failed to parse/validate config: Failed to bind one of the listener ports.
  • [err] Reading config failed--see warnings above.

This seems to be an issue with tor and not systemd. T

The files I've attached are setup where tor successfully loads with systemd using User root group=root.

Lastly the other modification I had to make was in /etc/apparmor.d/system_tor wherefore the default cookie locations are mismatched between apparmor and tor:

apparmor.d/system_tor sets the cookie path as:

  /{,var/}run/tor/control.authcookie w,

  /{,var/}run/tor/control.authcookie.tmp rw,

tor's default sets the cookie path as:

  /var/run/tor/control_auth_cookie

Which causes apparmor to trigger and deny tor from writing/reading the cookie file and tor fails to start. Thus I had to add to system_tor the lines:

/{,var/}run/tor/control_auth_cookie w,

/{,var/}run/tor/control_auth_cookie.tmp rw,


The other issue is with starting tor with systemd is the option DisableAllSwap doesn't work and gets the error:

  • [warn] You appear to lack permissions to change memory limits. Are you root?
  • [warn] Unable to raise RLIMIT_MEMLOCK: Operation not permitted
  • [notice] Unable to lock all current and future memory pages: Cannot allocate memory
  • [warn] Failed to parse/validate config: DisableAllSwap failure. Do you have proper permission

I haven't been able to resolve the cause of this. However, the option does work when starting tor from the command line with DisableAllSwap enabled.

Hopefully the maintainers of tor will address and correct this for the next release.

Child Tickets

Change History (11)

comment:1 Changed 12 months ago by d3m0nkingx

https://drive.google.com/open?id=1WnhhUDSGPd4evC1D5U3MbCVMCvEmUzLc

files modified to make systemd work with tor

comment:2 Changed 12 months ago by d3m0nkingx

I was able to resolve the RLIMIT_MEMLOCK permission issue for enabling tor's 'DisableAllSwap 1' option:

To fix this it required to modify systemd's control groups memory management by doing the following at command line:

  • $ systemctl set-property tor.service MemoryAccounting=true

then in the tor.service file under [Service] heading add:

  • LimitMEMLOCK=infinity

Then limit the amount of memory used with:

  • $ systemctl set-property tor.service MemoryLimit=64M

Afterwards, do systemctl daemon-reload; system service tor restart

Last edited 12 months ago by d3m0nkingx (previous) (diff)

comment:3 Changed 12 months ago by d3m0nkingx

Summary: Systemd fails to load included service files tor@.service or tor@default.serviceSystemd fails to load included service files tor@.service or tor@default.service + DisableAllSwap Fix

comment:4 Changed 12 months ago by d3m0nkingx

  • $ systemctl set-property tor.service MemoryLimit=64M

Also set the memory limit to 64 Megabytes rather than allowing all available. Setting less than 47MB resulted in tor having an oom and kernel kills the process:

out_of_memory+0x2ce/0x4f0

kernel:  mem_cgroup_out_of_memory+0x4b/0x80

kernel:  mem_cgroup_oom_synchronize+0x2e8/0x320

kernel:  ? mem_cgroup_css_online+0x40/0x40

kernel:  pagefault_out_of_memory+0x36/0x7b

kernel:  mm_fault_error+0x90/0x180

kernel:  __do_page_fault+0x4a5/0x4d0

kernel:  do_page_fault+0x2d/0xf0

kernel:  ? page_fault+0x2f/0x50

kernel:  page_fault+0x45/0x50

kernel: RIP: 0033:0x7efc1d3de786

kernel: RSP: 002b:00007ffcd6ed4e50 EFLAGS: 00010206

kernel: RAX: 000000000001c001 RBX: 00007efc1d720b20 RCX: 0000000000000021

kernel: RDX: 0000559948ceefe0 RSI: 0000559948cef000 RDI: 0000000000000000

kernel: RBP: 0000000000000021 R08: 0000559948ceef50 R09: 2e33363120746100

kernel: R10: 2e3637312e323731 R11: ffffffffffffffff R12: 00007efc1d720b78

kernel: R13: 00007efc1d720b78 R14: 0000000000002710 R15: 00007efc1d720b88

kernel: Task in /system.slice/tor.service killed as a result of limit of /system.slice/tor.service

kernel: memory: usage 1024kB, limit 1024kB, failcnt 162

kernel: memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0

kernel: kmem: usage 224kB, limit 9007199254740988kB, failcnt 0

kernel: Memory cgroup stats for /system.slice/tor.service: cache:0KB rss:792KB rss_huge:0KB shmem:0KB mapped_file:0KB dirty:0

kernel: [ pid ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name

kernel: [ 5884]     0  5884    11802     2277   139264        0             0 tor

kernel: Memory cgroup out of memory: Kill process 5884 (tor) score 1 or sacrifice child

kernel: Killed process 5884 (tor) total-vm:47208kB, anon-rss:800kB, file-rss:8308kB, shmem-rss:0kB

kernel: oom_reaper: reaped process 5884 (tor), now anon-rss:540kB, file-rss:7016kB, shmem-rss:0kB

systemd[1]: tor.service: Control process exited, code=killed status=9

comment:5 Changed 12 months ago by d3m0nkingx

Along with disabling the init script for /etc/init.d/tor, I removed the service files tor@ default.service and tor@.service from the systemd directory.

Last edited 12 months ago by d3m0nkingx (previous) (diff)

comment:6 Changed 12 months ago by asn

Keywords: apparmor added
Severity: MajorNormal
Summary: Systemd fails to load included service files tor@.service or tor@default.service + DisableAllSwap FixApparmor + systemd failures when loading included service files + DisableAllSwap Fix

Trying to triage this ticket. Pretty complicated summary.

comment:7 Changed 12 months ago by asn

Milestone: Tor: 0.3.3.x-finalTor: 0.3.5.x-final

comment:8 Changed 12 months ago by Hello71

Keywords: apparmor removed
Milestone: Tor: 0.3.5.x-finalTor: 0.3.3.x-final
Severity: NormalMajor
Summary: Apparmor + systemd failures when loading included service files + DisableAllSwap FixSystemd fails to load included service files tor@.service or tor@default.service + DisableAllSwap Fix

what is the actual problem here? please use the following report format:

brief description:

how to reproduce:

expected results:

actual results:

comment:9 Changed 12 months ago by Hello71

Keywords: apparmor added
Milestone: Tor: 0.3.3.x-finalTor: 0.3.5.x-final
Severity: MajorNormal
Summary: Systemd fails to load included service files tor@.service or tor@default.service + DisableAllSwap FixApparmor + systemd failures when loading included service files + DisableAllSwap Fix

comment:10 Changed 10 months ago by nickm

Keywords: 035-removed-20180711 added
Milestone: Tor: 0.3.5.x-finalTor: unspecified

These tickets are being triaged out of 0.3.5. The ones marked "035-roadmap-proposed" may return.

comment:11 Changed 8 months ago by traumschule

group tickets related to AppArmorForTBB/tor packages

Note: See TracTickets for help on using tickets.