Opened 18 months ago

Last modified 4 months ago

#25227 new enhancement

Avoid storing all Tor nodes in RAM

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-client, 034-triage-20180328, 034-removed-20180328
Cc: Actual Points:
Parent ID: Points: 5
Reviewer: Sponsor:

Description

<teor4>	ahf: I've been thinking about reducing Tor's RAM usage by sampling from each consensus, rather than keeping it all in RAM
<teor4>	there are ways to stream the consensus, pick a weighted sample, and just keep those nodes
<teor4>	https://en.wikipedia.org/wiki/Reservoir_sampling
<teor4>	it would really help the iOS VPN, and other embedded impls
<teor4>	The catch is that if you run out of sampled nodes, you need to re-parse the consensus
<+ahf>	teor4: i had been thinking a bit in the back of my head if we could refactor the code that is used for accessing the consensus to be a bit more pluggable to experiment with different ways of backing them
<+ahf>	teor4: on ios i think we can mmap() files without them adding to the memory limit of the application too
<teor4>	that would be helpful, because we could only keep the selected node queues in RAM
<teor4>	if we got the queue size right, we could regenerate them once per consensus
<+ahf>	do you want to create a ticket with this? maybe add sponsor8-can to the keywords

We could be smart about node selection, and just store some weights for each node. And then when we use too much RAM for nodes, we could evict older nodes.

Child Tickets

Change History (5)

comment:1 Changed 17 months ago by nickm

Keywords: 034-triage-20180328 added

comment:2 Changed 17 months ago by nickm

Keywords: 034-removed-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

comment:3 Changed 17 months ago by nickm

Milestone: Tor: 0.3.4.x-finalTor: unspecified

These tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if time permits.

comment:4 Changed 7 months ago by gaba

Sponsor: Sponsor8-can

comment:5 Changed 4 months ago by cypherpunks

reconsider any of them, if time permits.

Note: See TracTickets for help on using tickets.