Opened 5 years ago

Closed 5 years ago

#13702 closed enhancement (fixed)

Adding OpenBSD to doc/TUNING

Reported by: mmcc Owned by:
Priority: Medium Milestone: Tor: 0.2.6.x-final
Component: Core Tor/Tor Version: Tor: unspecified
Severity: Keywords: OpenBSD, tuning, doc/TUNING lorax
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

As described in the below draft, OpenBSD is a little more persnickety about maximum file descriptor counts than most OSs. It's also the most-used Unix OS not listed in doc/TUNING thus far, so it makes sense to include it.

I wrote the below section, which turned out to be a little long. Please let me know what you think of it; I'd be fine cutting much of it out or doing a total rewrite if that's the best option.


OpenBSD


For recent versions of OpenBSD (5.5 and 5.6, and probably older releases
as well), the maximum number of file descriptors that can be opened is
7030:

http://unix.stackexchange.com/questions/104929/does-openbsd-have-a-limit-to-the-number-of-file-descriptors/104948#104948

This limit is kernel-level. To change it, you would have to modify the
relevant constants and recompile the kernel.

However, there are stricter limits set on users. This is a security
feature intended to prevent one user from choking out others by opening
all possible file descriptors.

The stricter limits are set in /etc/login.conf. This config file sets
resource access rules for user classes. You should be running
Tor as a non-privileged daemon user '_tor', which belongs to the 'daemon'
class. It will therefore be subject to the 'default' and 'daemon' rules.
There are two relevant rules: openfiles-cur and openfiles-max. The prior
is the initial limit upon login - the soft limit. The latter is the maximum limit
that can be set using 'ulimit -n' or setrlimit() without editing /etc/login.conf and
rebooting. This is known as the hard limit.

Without editing /etc/login.conf, daemon-owned processes have a
soft limit of 512 open files and a hard limit
of 1024 open files. Tor can increase the soft limit as needed, so
you will therefore eventually get warnings about running
out of available file descriptors once Tor reaches ~1024 open files.

To increase the hard limit, add the following line to the daemon class
rules in /etc/login.conf:

:openfiles-max=6500:\

Upon restarting the machine, Tor will be able to open up to 6500 file
descriptors.

Be aware that, by doing this, you are bypassing a security and stability
feature of the OS. If you are running your relay on a weak or old system, watch
your system load to ensure that it can handle this many open files.
Also, Tor may interfere with any other programs that open many files.

Child Tickets

Change History (21)

comment:1 Changed 5 years ago by nickm

Keywords: lorax added
Milestone: Tor: 0.2.6.x-final
Status: newneeds_review

comment:2 Changed 5 years ago by rl1987

I added this section to doc/TUNING. See the following branch:

My question is: is it really necessary to edit source and recompile kernel in order to increase maximum allowed number of file descriptors? Maybe there is an easier way to raise it?

comment:3 Changed 5 years ago by mmcc

rl1987: Yeah, I'm doing more research now, but I may have made a newb mistake reading the OpenBSD documentation. It seems that one can change the sysctl variable kern.maxfiles with a command like sudo sysctl kern.maxfiles=20000. I thought I had double-checked this - sorry for not doing a more rigorous review before it got committed.

I'll come back to confirm soon. If I did indeed make a mistake, we can change the first three paragraphs (through "recompile the kernel") with:

The maximum number of file descriptors that an OpenBSD machine can have open is stored in the sysctl variable kern.maxfiles. This value defaults to 7030 - to verify this, run sysctl kern.maxfiles.

To immediately change a running system's file descriptor limit to, for example, 20,000 files, run sudo sysctl kern.maxfiles=20000. All sysctl variables are reset upon reboot using defaults and /etc/sysctl.conf, so to make your change permanent you must add the line kern.maxfiles=20000 to /etc/sysctl.conf.

comment:4 Changed 5 years ago by mmcc

It may also be worth mentioning that kern.nfiles tells the user how many files the system has open, which would make it easier to diagnose whether kern.maxfiles or openfiles-max is the limiting factor. This section's pretty long as it is, though.

comment:5 Changed 5 years ago by mmcc

For what it's worth, I got confirmation on IRC that sysctl can change the kernel's maximum.

comment:6 Changed 5 years ago by nickm

rl1987, mmcc -- please let me know once you're reasonably confident that there's something I should include? It doesn't need to be perfect, but it should be accurate.

comment:7 Changed 5 years ago by mmcc

I got a confirmation on Freenode's #OpenBSD that this is correct when the update is made. I can look for another conformation now.

comment:8 Changed 5 years ago by mmcc

Someone else tipped me off that you can make a rule specifically for daemons defined in /etc/rc.d/. The below change would be better because it's tor-specific:

tor:\
        :openfiles-max=8192:\
        :tc=daemon:

This has become increasingly messy, so I'll just gather a final edition from the proposed edits.

comment:9 Changed 5 years ago by nickm

Is there something I can apply on this ticket?

comment:10 Changed 5 years ago by rl1987

Status: needs_reviewneeds_revision

comment:11 Changed 5 years ago by nickm

Waiting for the revision to happen. Is there something I can merge soon, or will there be? If so I can take it in 0.2.6. Otherwise it will have to wait.

comment:12 Changed 5 years ago by rl1987

I got the the above branch updated to no longer tell the user they have to recompile a kernel.

comment:13 Changed 5 years ago by rl1987

Status: needs_revisionneeds_review

comment:14 Changed 5 years ago by nickm

Resolution: fixed
Status: needs_reviewclosed

Okay, squashed + merged. Let me know if there's more to add here, or if anything looks wrong.

Thanks!

comment:15 Changed 5 years ago by mmcc

Hi, Nick. Thanks for this! I just came back across this ticket a couple days ago and was planning on updating it this weekend.

Suggesting a kernel compile was dumb, so after noticing that mistake I decided to step away until I understood both Tor and OpenBSD a little better.

The current version looks accurate, but it's definitely longer than it has to be. Additionally, I and others have exhausted the available mbufs, so kern.maxclusters should be increased. Also, PF limits the number states to 10,000, which also probably makes sense to increase for high-capacity exits. I'll make these changes soon.

mmcc

Last edited 5 years ago by mmcc (previous) (diff)

comment:16 Changed 5 years ago by mmcc

Below is (IMO) a better, clearer, and more thorough version. Feel free to make any necessary corrections. I know you're busy, so obviously feel free to leave it as is until 0.2.6 is frozen.

Because OpenBSD is primarily focused on security and stability, it uses default resource limits stricter than those of more popular Unix-like operating systems.

OpenBSD stores a kernel-level file descriptor limit in the sysctl variable kern.maxfiles. It defaults to 7,030. To change it to, for example, 16,000 while the system is running, use the command 'sudo sysctl kern.maxfiles=16000'. kern.maxfiles will reset to the default value upon system reboot unless you also add 'kern.maxfiles=16000' to the file /etc/sysctl.conf.

There are stricter resource limits set on user classes, which are stored in /etc/login.conf. This config file also allows limit sets for daemons started with scripts in the /etc/rc.d directory, which presumably includes Tor.

To increase the file descriptor limit from its default of 1,024, add the following to /etc/login.conf:

tor:\
	:openfiles-max=13500:\
	:tc=daemon:

Upon restarting Tor, it will be able to open up to 13,500 file descriptors.

This will work *only* if you are starting Tor with the script /etc/rc.d/tor. If you're using a custom build instead of the package, you can easily copy the rc.d script from the Tor port directory. Alternatively, you can ensure that the Tor's daemon user has its own user class and make a /etc/login.conf entry for it.

High-bandwidth relays sometimes give the syslog warning:

/bsd: WARNING: mclpools limit reached; increase kern.maxclusters

In this case, increase kern.maxclusters with the sysctl command and in the file /etc/sysctl.conf, as described with kern.maxfiles above. Use 'sysctl kern.maxclusters' to query the current value. Increasing by about 15% per day until the error no longer appears is a good guideline.

Last edited 5 years ago by mmcc (previous) (diff)

comment:17 Changed 5 years ago by mmcc

I just edited my previous comment to include kern.maxclusters.

comment:18 Changed 5 years ago by rl1987

Resolution: fixed
Status: closedreopened

comment:19 Changed 5 years ago by rl1987

I got the OpenBSD section updated in the following branch:

  • https://github.com/rl1987/tor/commits/doc13702_revisited

Looks okay? Can we get it merged into Tor 0.2.6.3?

comment:20 Changed 5 years ago by mmcc

@rl1987: Looks good to me. Thanks!

comment:21 Changed 5 years ago by nickm

Resolution: fixed
Status: reopenedclosed

Applied; thanks!

Note: See TracTickets for help on using tickets.