Opened 2 years ago

Last modified 6 months ago

#22706 new enhancement

Compress cached descriptors and consensus

Reported by: nickm Owned by:
Priority: Low Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: disk-usage, 034-triage-20180328, 034-removed-20180328, 035-removed-20180711
Cc: Actual Points:
Parent ID: #25503 Points: 4
Reviewer: nickm Sponsor:

Description

We can save disk space on clients (perhaps?) if we store more of these objects compressed.

Child Tickets

Attachments (1)

002-gzipped_cache.patch (32.7 KB) - added by nickm 2 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 2 years ago by nickm

Component: - Select a componentCore Tor/Tor
Status: newneeds_review

Changed 2 years ago by nickm

Attachment: 002-gzipped_cache.patch added

comment:2 Changed 2 years ago by nickm

Keywords: review-group-18 removed
Status: needs_reviewnew

Review:

  • I think this presumes #22704.
  • This patch seems to have the same issue with anonymous mmaps as I noted in #22704: is it really saving RAM? I can believe it saves disk when Tor isn't running.
  • This patch should IMO be off-by-default, and I don't think it's safe to turn it on for directory caches.
  • The compression parts need to be rewritten to use the new compress.c APIs.
  • There are bigger memory savings available on clients: once we have the (micro)descriptors parsed, we can just leave them on disk, parse them, and not keep their text in memory at all. (After parsing we only use the text to answer controller requests and directory requests.)
  • We shouldn't add our own functions in the gz* namespace.
  • Cache compression won't work very well: it only handles one (micro)descriptor at a time.

If we decide to take this approach, we should start with a fresh patch, using parts of the design here as needed.

comment:3 Changed 2 years ago by nickm

Sponsor: Sponsor8-can

comment:4 Changed 22 months ago by nickm

Milestone: Tor: 0.3.2.x-finalTor: 0.3.3.x-final

comment:5 Changed 18 months ago by nickm

Milestone: Tor: 0.3.3.x-finalTor: 0.3.4.x-final
Priority: MediumLow
Type: defectenhancement

comment:6 Changed 16 months ago by nickm

Keywords: 034-triage-20180328 added

comment:7 Changed 16 months ago by nickm

Keywords: 034-removed-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

comment:8 Changed 16 months ago by nickm

Milestone: Tor: 0.3.4.x-finalTor: 0.3.5.x-final
Parent ID: #7176#25503

comment:9 Changed 12 months ago by nickm

Keywords: 035-removed-20180711 added
Milestone: Tor: 0.3.5.x-finalTor: unspecified

These tickets are being triaged out of 0.3.5. The ones marked "035-roadmap-proposed" may return.

comment:10 Changed 6 months ago by gaba

Keywords: sponsor8-maybe removed
Sponsor: Sponsor8-can
Note: See TracTickets for help on using tickets.