Opened 9 months ago

Last modified 4 weeks ago

#28804 assigned defect

Add circuit padding to padding-spec.txt and write a doc for researchers

Reported by: asn Owned by: mikeperry
Priority: High Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: wtf-pad, tor-relay, tor-cell, padding, tor-spec, 041-proposed, network-team-roadmap-august, scalability-roadmap
Cc: nickm Actual Points:
Parent ID: Points: 2
Reviewer: asn Sponsor: Sponsor2

Description

There are a few things that can be done to the spec to improve them.

For the basics, we can improve the sectioning and also add some indentation to make it more clear how sections are structured (for example, to make it clear the distinctions between histograms and prob distrs).

Child Tickets

Change History (19)

comment:1 Changed 9 months ago by asn

Also update spec (and maybe code) with the discussion from https://github.com/torproject/tor/pull/461#discussion_r239949925 .

comment:2 Changed 8 months ago by mikeperry

Keywords: 041-proposed added

comment:3 Changed 8 months ago by mikeperry

Points: 2

Prop 254 also needs to be folded into padding-spec.txt.

comment:4 Changed 8 months ago by mikeperry

Priority: MediumHigh
Summary: Improve quality of prop254 specWrite padidng spec and a doc for researchers

Let's just supercede the proposal with the padding-spec and a doc for creating state machines. The actual final spec is more important than the historical proposal archive anyway.

comment:5 Changed 8 months ago by mikeperry

Summary: Write padidng spec and a doc for researchersAdd circuit padding to padding-spec.txt and write a doc for researchers

comment:6 Changed 8 months ago by nickm

Sponsor: Sponsor2

comment:7 Changed 8 months ago by nickm

Parent ID: #28637#28631

Reparent children of #28637 into #28631

comment:8 Changed 4 months ago by mikeperry

Parent ID: #28631

comment:9 Changed 4 months ago by gaba

Keywords: network-team-roadmap-2019-Q1Q2 added

comment:10 Changed 4 months ago by mikeperry

Ideas that are popping into my head now that #28634 is done:

One section of the doc should walk the reader through the fields of circpad_machine_spec_t, circpad_state_t, and their interaction with circpad_machine runtime_t (which is invisible to them but feature choice has consequences on how much memory and CPU time circpad_machine_runtime_t requires). This section of the doc should also describe potential features from the wtf-pad tag ticket list, but make it clear we're unlikely to implement them unless someone tells us them need them. And also mention that we're happy to accept patches for any useful feature or flag.

The second section of the doc should have a pile of example machines, to show that the system can do much much more than we're currently using it for. We should not only explain #28634 and why it chose the options it did for performance reasons, but also explain that there's more features we can implement easily. We should highlight a few of those from the wtf-pad tag ticket list, and also show some example machines that do the following:

  • Netflow/channelpadding as circuitpadding to the guard
  • Original WTF-PAD
  • APE (https://www.cs.kau.se/pulls/hot/thebasketcase-ape/)
  • Constant rate cover traffic from the guard (will require #29494 implementation to make cell delivery decisions based on cmux queue status, but as far as machines are concerned, this will just be a flag they set)
  • A reduced padding machine vs not (perhaps using a heavier-padding-count #28634 example)

We also want to discuss some high level interactions with Tor:

  • Explain our load balancing plans
  • Explain how machine precedence is meant to work when multiple machines exist for the same conditions
Last edited 4 months ago by mikeperry (previous) (diff)

comment:11 Changed 4 months ago by mikeperry

Other things to do: I want to stop referring to what we implemented as "WTF-PAD" in technical documentation. WTF-PAD was the implementation of the system described by the research paper http://arxiv.org/pdf/1512.00524. The stuff in Tor now is NOT WTF-PAD. If people start writing research papers about how they attacked the new WTF-PAD as implemented in Tor, all hell will break loose and nobody will know what anybody is talking about anymore.

We should update the Tor spec/proposal (aka https://lists.torproject.org/pipermail/tor-dev/2019-May/013822.html) accordingly.

Additionally, I updated the comments in https://github.com/torproject/tor/blob/master/src/core/or/circuitpadding_machines.c#L60 to make it easier to understand, but these did not get updated in the spec. So we should do that too.

comment:12 Changed 4 months ago by mikeperry

One more thing to cover in the research doc: We should explain how we will accept and deploy changes to the padding machine featureset itself (one potentially useful change is for all state transitions have a probability associated with them, to explicitly support HMM representations).

comment:13 Changed 3 months ago by gaba

Owner: set to mikeperry
Status: newassigned

comment:14 Changed 6 weeks ago by gaba

Keywords: network-team-roadmap-august scalability-roadmap added; network-team-roadmap-2019-Q1Q2 removed

comment:15 Changed 5 weeks ago by mikeperry

We should be sure to document potential shutdown sync issues of #30992 in a "known bugs/implementation issues" section. I bet there will be other things for such a section, too.

comment:16 Changed 5 weeks ago by mikeperry

Ok, here's a branch with an updated padding-spec. Has a couple XXX's (we need the new NSF numbers and it needs to cite the researcher developer doc that still needs to be written).

https://github.com/mikeperry-tor/torspec/tree/hs-padding-spec

comment:18 Changed 4 weeks ago by asn

Reviewer: asn

Putting myself as the reviewer here.

comment:19 Changed 4 weeks ago by asn

Oops I thought this was needs_review so I took a look, but it seems like it's not.

Anyhow, spec patch looks good.

And also some high-level comments on the developer doc:

  • Let's make it markdown since both github and gitlab support md files these days and it seems to be the norm? I doubt researchers appreciate text files that much.
  • Let's add a "High-level system overview" section before the "Header walkthrough" that explains how this whole system works in two paragraphs or so.
  • On the 2. What to consider when creating a new machine, I'd also like an explanation of how machine numbers should work, and how to choose your machine number to avoid conflicts with other tor versions etc.
  • Other than that, looks good to me.
Note: See TracTickets for help on using tickets.