Opened 7 weeks ago

Last modified 2 weeks ago

#25372 assigned defect

relay: Allocation for compression goes very high

Reported by: dgoulet Owned by: ahf
Priority: High Milestone: Tor: 0.3.4.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-relay, compression, tor-dos, review-group-34, 033-backport?, 034-triage-20180328, 034-must performance oom
Cc: Actual Points:
Parent ID: Points:
Reviewer: nickm Sponsor:

Description

My relay just OOMed some circuits with filled up queue (#25226) but then a useful log was printed showing that the compress total allocation is huge.

Feb 27 20:02:55.718 [notice] We're low on memory (cell queues total alloc: 232279872 buffer total alloc: 1937408, tor compress total alloc: 878586075 rendezvous cache total alloc: 4684497). Killing circuits withover-long queues. (This behavior is controlled by MaxMemInQueues.)

That 878586075 = ~838MB. My relay is hovering around 1.4GB of RAM right now which means ~60% of the RAM used is in the compression subsystem.

I'm not sure where it all comes, the relay is serving directory data but I have my doubt that *compressed*, it comes down to 800+ MB...

Datapoint:

$ du -sh diff-cache/
131M	diff-cache/

Child Tickets

Change History (13)

comment:1 Changed 7 weeks ago by nickm

Which compression methods is your tor compiled with?

comment:2 Changed 7 weeks ago by dgoulet

Which compression methods is your tor compiled with?

Feb 13 20:15:54.919 [notice] Tor 0.3.3.2-alpha-dev (git-86f461e362480bb5) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g, Zlib 1.2.8, Liblzma 5.1.0alpha, and Libzstd N/A.
Last edited 7 weeks ago by dgoulet (previous) (diff)

comment:3 Changed 7 weeks ago by ahf

Owner: set to ahf
Status: newassigned

comment:4 Changed 7 weeks ago by ahf

Looks like this isn't related to #24368 if Zstandard is disabled.

comment:5 Changed 7 weeks ago by ahf

Status: assignedneeds_review

A patch that might help us in the future: https://gitlab.com/ahf/tor/merge_requests/24

Please don't close the ticket if this is applied since it doesn't solve the issue, but it might help us debug issues in the future.

comment:6 Changed 7 weeks ago by nickm

Keywords: review-group-34 added

comment:7 Changed 5 weeks ago by dgoulet

Reviewer: nickm

Assigning reviewer for week 03/19.

comment:8 Changed 5 weeks ago by nickm

Status: needs_reviewneeds_revision

ahf: lgtm but please add a changes file?

comment:9 Changed 4 weeks ago by ahf

Status: needs_revisionneeds_review

Should be fixed now.

comment:10 Changed 4 weeks ago by nickm

Keywords: 033-backport? added
Status: needs_reviewassigned

Cherry-picked this to 0.3.3. Thank you! Putting this back into "assigned" per earlier instructions.

comment:11 Changed 3 weeks ago by nickm

Keywords: 034-triage-20180328 added

comment:12 Changed 3 weeks ago by nickm

Keywords: 034-removed-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

comment:13 Changed 2 weeks ago by nickm

Keywords: 034-must performance oom added; 034-removed-20180328 removed
Note: See TracTickets for help on using tickets.