prop278: Refactor the torgzip module to support additional compression schemes

The current torgzip module should be refactored such that the new compression schemes needed for prop#278 can fit nicely into the code.

This is the tracking bug for this task.

changed milestone to %Tor: 0.3.1.x-final

added TorCoreTeam201703 component::core tor/tor milestone::Tor: 0.3.1.x-final owner::ahf points::3 priority::medium prop278 resolution::implemented severity::normal sponsor::4 status::closed type::task labels

Trac:
Owner: N/A to ahf
Status: new to accepted

Trac:
Actualpoints: 3 to N/A
Points: N/A to 3

Trac:
Keywords: N/A deleted, TorCoreTeam201703 added

Add prop278 keyword.

Trac:
Keywords: N/A deleted, prop278 added

Starting to make my big zstd/xz branch reviewable in https://gitlab.com/ahf/tor/compare/master...prop278%2F21663

Still a few things missing before I mark this as needs_review, but this will make it easier for other people to track the progress here.

Awesome! Is the list of missing features recorded somewhere? That will help distinguish "absent but planned" from "absent and not planned"

The patches in my prop278/21663 branch is the cleaned up versions of the patches in bugs/21663 branch which is currently a bit of a mess.

The only part missing in the bugs/21663 branch is currently the streaming code for Zstd/LZMA, but the tor_compress() and tor_uncompress() functionality is working (and passing tests).

I think it's realistic that this bug will be marked needs_review tonight (Danish time) when I'm done splitting things up into something that is reviewable, which will include patches for #21664 (moved) and #21662 (moved).

#21665 (moved) should be a very short fix, but is not included in the bugs/21663 branch currently; we currently pass UINT64_MAX to disable the upper bound of memory consumption for LZMA.

I looked it over, and what's there looked fine, but I didn't see any actual lzma2 or zstd support. That's coming in the streaming code? If so I'll have another look later in the day

Refactoring and clean-up code for this issue and issue #21664 (moved) can be found in:

https://gitlab.com/ahf/tor/merge_requests/1/commits

Trac:
Status: accepted to needs_review

The Zstd/LZMA code is in the big messy branch (bugs/21663) which I'm turning into smaller reviewable patches in the respective issues here on trac.

Looks pretty good to me! The only issues I noted on the ticket were pretty minor.

Trac:
Status: needs_review to needs_revision

Merging!

Trac:
Status: needs_revision to closed
Resolution: N/A to implemented

closed

changed time estimate to 24h

mentioned in issue #21665 (moved)

moved to tpo/core/tor#21663 (closed)