Prop278: Handle new headers in directory.c

Handle the newly defined headers and their new values from Prop#278 in the directory server/client code.

changed milestone to %Tor: 0.3.1.x-final

added TorCoreTeam201703 component::core tor/tor milestone::Tor: 0.3.1.x-final owner::ahf points::2 priority::medium prop278 resolution::implemented severity::normal sponsor::4 status::closed type::task labels

Trac:
Milestone: N/A to Tor: 0.3.1.x-final

Trac:
Keywords: N/A deleted, TorCoreTeam201703 added

Trac:
Owner: N/A to ahf
Status: new to accepted

Add prop278 keyword.

Trac:
Keywords: N/A deleted, prop278 added

Part of this is partially done in my parse_accept_encoding branch. That only handles the parsing: it doesn't actually do anything with compression types other than deflate. I'll add some tests for it when I get back to it.

Starting to track this in https://gitlab.com/ahf/tor/tree/bugs/21667

I think I could use some feedback on this now: https://gitlab.com/ahf/tor/merge_requests/11?expand_all_diffs=1

I'm missing two essentials: lift my usage of srv_meth_pref_precompressed into a separate constant and deny LZMA during request negotiation for the dynamic endpoints. I'll add these changes as fixups.

Trac:
Status: accepted to needs_review

I added some comments there, but one thing I don't understand -- when does this ever actually serve compressed consensus documents correctly? It changes streaming compression, but it doesn't serve the right spooled_resource_t objects for cached consensuses. Is that coming in a separate commit?

Trac:
Status: needs_review to needs_revision

That will arrive in a separate commit; working on that right now while having pulled your changes from the compress_consensus_more_ways branch in.

Do you agree on having a variable like srv_meth_pref_precompressed where we define the available order, excluding LZMA, for non-precompressed documents?

Yes, that sounds good. Maybe srv_meth_pref_streaming ?

Works for me.

The code from bug #21668 (moved) and the changes already reviewed in PR11 on Gitlab have been moved into https://gitlab.com/ahf/tor/merge_requests/12/commits

Currently tracing a bug when running this code in chutney where we initially receive a ZLIB compressed object, which we claim is compressed with LZMA.

One of the patches contains a new API in consdiff manager to query metadata around a consensus document. These new functions are currently just stubs as per discussion on IRC tonight.

I have made a branch 21667_2_plus in my public repository to fill in the blank API functions. I also had to change them a bit; I made corresponding changes to the callers. Untested; sorry!

I also left some comments on the Gitlab PR. Enjoy!

Patches added to GL - your comments should be addressed as well.

5 of the dir_handle_get/... tests are currently failing, looking into that now.

Trac:
Status: needs_revision to needs_review

patches look fine -- once the tests are working, let's merge.

My branch 21667_2_testing fixes 2 of the failing tests, and adds XXXX comments explaining how to fix the other 3

Patch to fix test execution landed. Not exactly as per the instructions since a lot of the code in handle_get_current_consensus() does various NULL checks for cached_consensus, so we need to inject a consensus anyway - unless I'm overseeing something.

Squashed and merged. Wooo!

Trac:
Status: needs_review to closed
Resolution: N/A to implemented

closed

changed time estimate to 16h

mentioned in issue #21668 (moved)

mentioned in issue #22128 (moved)

mentioned in issue #22148 (moved)

moved to tpo/core/tor#21667 (closed)