Opened 5 months ago

Last modified 5 days ago

#29004 assigned enhancement

PrivCount proof of concept: implement check counters

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: 0.4.2.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: privcount, 040-unreached-20190109, 041-accepted-20190115, network-team-unreached-2019-Q1Q2
Cc: Actual Points: 0.4
Parent ID: #27908 Points: 1
Reviewer: asn Sponsor:

Description

We can start building a PrivCount proof of concept with 3 check counters:

  • ZeroCount - all Data Collectors (DCs) store zero in this counter. If aggregation succeeds, the result is zero. Otherwise, all aggregated results for this round are meaningless.
  • DataCollectorCount - all Data Collectors (DCs) store one in this counter. If aggregation succeeds, the result is the number of DCs that reported results.
  • SampledNoiseAmount - all Data Collectors (DCs) store some noise in this counter, using their standard processes for generating noise, including noise weighting, sampling, and rounding (if needed). If aggregation succeeds, and the standard deviation is large enough, the result will almost always be non-zero, and almost always be within a few standard deviations of zero. We need this counter, because bugs in noise generation may not be obvious in the results.

Child Tickets

TicketTypeStatusOwnerSummary
#29027enhancementassignedPrivCount proof of concept: put the PrivCount statistics in a stats/ file

Change History (20)

comment:1 Changed 5 months ago by teor

Type: defectenhancement

These should all be enhancements

comment:2 Changed 5 months ago by teor

Points: 1

If we hard-code a lot of things, this should not take more than a day or so.

comment:3 Changed 5 months ago by teor

Actual Points: 0.4
Status: assignedneeds_review

I have a proof-of-concept for PrivCount in extra-info descriptors in https://github.com/torproject/tor/pull/640

Here's what it looks like in chutney:

extra-info test000a C83D24A1EC44B1D48089CEBD7E31CB4611775027
...
pc-zero 0
pc-dc-count 1
pc-noise-sample TODO
pc-rw-bytes 0
pc-g-rw-bytes TODO
pc-e-rw-bytes TODO
pc-ge-rw-bytes TODO
pc-m-rw-bytes TODO
pc-stats-end
router-sig-ed25519 ...

comment:4 Changed 5 months ago by dgoulet

Reviewer: nickm

comment:5 Changed 5 months ago by dgoulet

Reviewer: nickmasn

comment:6 Changed 5 months ago by teor

Keywords: 040-unreached-20190109 041-proposed added
Milestone: Tor: 0.4.0.x-finalTor: unspecified

These tasks are on the 0.4.1 roadmap for PrivCount and Sponsor V.

comment:7 Changed 5 months ago by teor

Keywords: 041-proposed-on-roadmap added; 041-proposed removed
Milestone: Tor: unspecifiedTor: 0.4.1.x-final

Let's review these tickets at the next meeting using our 041-proposed process.

They're on the roadmap, so the review should focus on ticket size and team capacity (and sponsor expectations).

comment:8 Changed 5 months ago by asn

Status: needs_reviewmerge_ready

Patch looks really nice and tidy!

Not sure how the TODOs are meant to be addressed, but what I'm seeing looks good!

comment:9 Changed 5 months ago by teor

Status: merge_readyneeds_revision

Thanks for the review!

I talked to nickm about the names, we're going to go for something between pc-ge-rw-bytes and privcount-guard-and-exit-read-and-write-byte-count.

I will look at the TODOs. Maybe I should have flagged this as a work in progress.

comment:10 Changed 5 months ago by teor

Keywords: 041-accepted-20190115 added; 041-proposed-on-roadmap removed

These PrivCount tickets are on the 041 roadmap, we accepted their points estimates in 041 without discussion.

comment:11 Changed 5 months ago by nickm

Sponsor: SponsorV

comment:12 Changed 3 months ago by gaba

Keywords: network-team-roadmap-2019-Q1Q2 added

comment:13 Changed 3 months ago by teor

I have part of a patch for this ticket, but it needs to be revised after #29018 and #29019, and with #29005.

comment:14 Changed 3 months ago by teor

Keywords: backlog added

comment:15 Changed 8 weeks ago by teor

I have an incomplete branches for #29004, #29005, and #29019 here:
https://github.com/teor2345/tor/tree/ticket29004-wip
https://github.com/teor2345/tor/tree/ticket29005
https://github.com/teor2345/tor/tree/ticket29019

I think all the necessary code is present in these branches.

But it needs some cleanup:

  • rebase on to the current master,
  • put the commits on the right branches
  • make sure it does what these tickets say it should do

I'm happy to do that after I come back from leave. I am also happy if Nick wants to clean up this code.

comment:16 Changed 4 weeks ago by nickm

Milestone: Tor: 0.4.1.x-finalTor: 0.4.2.x-final

Move privcount tickets to 0.4.2

comment:17 Changed 6 days ago by teor

Keywords: network-team-roadmap-unreached-2019-Q1Q2 added; backlog network-team-roadmap-2019-Q1Q2 removed

comment:18 Changed 6 days ago by teor

Keywords: network-team-unreached-2019-Q1Q2 added; network-team-roadmap-unreached-2019-Q1Q2 removed

comment:19 Changed 6 days ago by teor

Owner: teor deleted
Status: needs_revisionassigned

comment:20 Changed 5 days ago by gaba

Sponsor: SponsorV
Note: See TracTickets for help on using tickets.