Opened 5 years ago

Closed 5 years ago

#11790 closed defect (duplicate)

Our zlib deflate objects require lots of memory per directory connection

Reported by: nickm Owned by:
Priority: Medium Milestone: Tor: 0.2.6.x-final
Component: Core Tor/Tor Version:
Severity: Keywords: tor-relay zlib 025-backport
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Compressing descriptors on the fly requires a lot of memory with zlib's default settings. According to zconf.h:

/* The memory requirements for deflate are (in bytes):
            (1 << (windowBits+2)) +  (1 << (memLevel+9))
 that is: 128K for windowBits=15  +  128K for memLevel = 8  (default values)
 plus a few kilobytes for small objects.

We are using windowbits == 15 and memLevel == 8. This might explain in part why busy directories are so memory-hungry. In addition to our investigations for #11787, we should see whether we can safely reduce this to 32K or so with windowBits == 12 and memlevel == 5. It appears in preliminary investigation that this compression level would not hurt compression very much (0-5% for microdescs, 3-7% for server descriptors).

We could also choose our window size and memlevel based on current memory availability.

Child Tickets

Change History (1)

comment:1 Changed 5 years ago by nickm

Resolution: duplicate
Status: newclosed

Whoops; looks like I made this twice. Closing as a dup of #11791.

Note: See TracTickets for help on using tickets.