Opened 8 years ago

Closed 7 years ago

#4485 closed project (wontfix)

Research: can we get rid of the stream-level sendme cells

Reported by: arma Owned by: arma
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Keywords: performance flowcontrol SponsorF20121101
Cc: sjmurdoch, nickm, mikeperry, iang, kevin, malsabah Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Ian and Mashael opined a year or so ago that our stream-level sendmes are not helpful, and perhaps harmful.

I'm inclined to agree. All they do is introduce yet another reason for us to sometimes not send a cell that we would otherwise send.

They were originally designed for fairness between streams in a given circuit, but once you have more than two streams in the circuit, I think the stream-level sendmes start to get in the way rather than help.

Step one is that we should analyze whether this is really true. Are there any cases where stream-level sendmes are still necessary? Do they help enough in the two-stream case to merit keeping them around? Do they harm enough in the three-stream case to merit taking them out?

Step two is to write a patch that rips them out assuming everybody in the network is running that patch, and then runs some private networks to make sure nothing crashes (easy to notice) or wedges (harder to notice).

Step three is to simulate this patched Tor in Shadow and/or ExperimenTor to see if there are any surprises on the performance front.

Step four is to design a backward-compatible way to phase them out, and get that started.

Child Tickets

Change History (15)

comment:1 Changed 8 years ago by arma

Keywords: performance, flowcontrolperformance flowcontrol

comment:2 Changed 8 years ago by karsten

Cc: sjmurdoch nickm mikeperry added
Milestone: Sponsor F: July 15, 2012
Type: taskproject

This ticket is the same as sponsor F deliverable 12. Turning it into a project ticket and optimistically assigning it to the July milestone. Related tickets should become children of this ticket.

comment:3 Changed 8 years ago by arma

Steps one and two are plausible to do in the July timeframe, if we decide we want to.

Step three is going to run into the "make the simulations be actually accurate" brick wall that is stalling our other performance work.

Really, I'm hoping that this question is moot because n23 will turn out to be the better plan. #4485 is on the list at https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/Performance

I'd suggest we table this one until we've made more progress on #4506.

comment:4 Changed 8 years ago by arma

Owner: set to arma
Status: newassigned

comment:5 in reply to:  3 Changed 8 years ago by karsten

Milestone: Sponsor F: July 1, 2012Sponsor F: November 1, 2012

Replying to arma:

Steps one and two are plausible to do in the July timeframe, if we decide we want to.

Step three is going to run into the "make the simulations be actually accurate" brick wall that is stalling our other performance work.

Step three is what makes me think we should move the deliverable to November. Doing that.

Really, I'm hoping that this question is moot because n23 will turn out to be the better plan. #4485 is on the list at https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/Performance

I'd suggest we table this one until we've made more progress on #4506.

Okay. Let's revisit this ticket around, say, end of May and see how to proceed.

comment:6 Changed 8 years ago by karsten

Some notes from the #tor-dev meeting on April 12:

  • Roger believes the answer to the question in this deliverable is "yes, we can get rid of stream-level congestion windows." The main result would be a) slightly improved performance for clients who do one thing at once, and b) unknown performance hits for clients who do two or more streams at once. To be sure of answering this question well, Roger would need to drop all his hats and be a Tor researcher for the next months.
  • The simulators need more work before we can answer this question well. Roger says he should help Rob with his CSET submission and move forward the various tickets on simulator fail.
  • Roger hopes that removing stream-level congestion windows might be unnecessary because we will decide N23 is better.

comment:7 Changed 7 years ago by karsten

Keywords: SponsorF20121101 added
Milestone: Sponsor F: November 1, 2012

Switching from using milestones to keywords for sponsor deliverables. See #6365 for details.

comment:8 Changed 7 years ago by karsten

We should hold on with this ticket until #4486 and #4487 are done. If N23 really makes this task moot, we shouldn't spend too much effort on this task.

comment:9 Changed 7 years ago by arma

I just wrote proposal 213, to discuss getting rid of stream-level sendmes.

Branch task4485 in my git repo rips out stream-level sendmes. It would be interesting to run it in Shadow-land, to see if a) something unexpected breaks, and b) the performance changes when we effectively double the window sizes.

comment:10 Changed 7 years ago by arma

Component: AnalysisTor
Milestone: Tor: unspecified
Status: assignednew
Summary: Research: can we get rid of the stream-level sendme cells?Get rid of the stream-level sendme cells

comment:11 Changed 7 years ago by arma

Summary: Get rid of the stream-level sendme cellsResearch: can we get rid of the stream-level sendme cells

comment:12 in reply to:  description Changed 7 years ago by arma

Cc: iang kevin malsabah added

Replying to arma:

Step one is that we should analyze whether this is really true. Are there any cases where stream-level sendmes are still necessary?

Ah ha! Based on discussion on tor-dev, I believe the answer here is yes, there is such a case: circuit-level flow control regulates the number of cells that can be in flight, but stream-level flow control is needed to push back on streams that shouldn't send any more cells yet because the other end hasn't reading enough yet. We don't want to block the circuit just because one of its streams is blocked, so we need some separate control mechanism.

In fact, if I am understanding things correctly, I believe this is a flaw with the n23 design as well -- it lacks any stream-level pushback for streams that are blocking on writes.

(It's interesting to note that none of the experiments in the Defenestrator paper, and also none of the standard torperf experiments, expose the "how do we push back if the application stops reading" issue.)

comment:14 in reply to:  9 ; Changed 7 years ago by karsten

Replying to arma:

I just wrote proposal 213, to discuss getting rid of stream-level sendmes.

Great! I assume proposal 213 is the result of sponsor F year 2 item 12? Can you suggest a deliverable summary of a few sentences for the wiki page? And can we close this sponsor deliverable ticket then (it's well past the November 1 deadline now)? Any further discussion would have to go into a new ticket, too.

Branch task4485 in my git repo rips out stream-level sendmes. It would be interesting to run it in Shadow-land, to see if a) something unexpected breaks, and b) the performance changes when we effectively double the window sizes.

Did you want me to run this simulation? It would need a new ticket though, unless it's still part of the deliverable. In that case, as November 1 was 5 days ago, we better finish this simulation and analysis of its results really soon. If it's not part of the deliverable, let's move this to a new ticket.

comment:15 in reply to:  14 Changed 7 years ago by arma

Resolution: wontfix
Status: newclosed

Replying to karsten:

Great! I assume proposal 213 is the result of sponsor F year 2 item 12? Can you suggest a deliverable summary of a few sentences for the wiki page?

I updated the wiki page.

And can we close this sponsor deliverable ticket then (it's well past the November 1 deadline now)?

Closing.

Any further discussion would have to go into a new ticket, too.

I opened #7346 for one of the next steps.

Did you want me to run this simulation?

No need to run the simulation at this point. We need more research/design work first.

Note: See TracTickets for help on using tickets.