Opened 4 years ago

Closed 2 years ago

#17919 closed task (not a bug)

Document history objects better

Reported by: seansaito Owned by: karsten
Priority: Medium Milestone:
Component: Metrics/Onionoo Version:
Severity: Normal Keywords: documentation
Cc: karsten, virgil, fmap Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

In the | documentation of Onionoo, all objects except for the history object have links to example requests (e.g., | bandwidth object example).

I'm wondering if the history object should have an analogous link so that people (like myself presently) can understand how to query for and use history objects.

Here are a few questions I have about the history object:

  • Can I query the history object for individual relays?
  • What data does a history object contain?
  • How far back does a history object go?

Thanks!

Child Tickets

Change History (9)

comment:1 Changed 4 years ago by seansaito

Component: - Select a componentOnionoo

comment:2 in reply to:  description Changed 4 years ago by karsten

Status: newneeds_information
Summary: Implement "example request" for history object in documentation of OnionooDocument history objects better

Hi seansaito,

Replying to seansaito:

  • Can I query the history object for individual relays?

Ah, no. History objects are part of other documents, like the bandwidth document.

  • What data does a history object contain?

A history object contains graph data, which can be anything from written bytes over consensus weights to uptime.

  • How far back does a history object go?

5 years is the upper limit, though we could extend that to 10 or 15 years if needed in the future. But for now, 5 years.

Would you want to take a look at the various history objects contained in the different document types and then suggest how we might be able to document this part better in the spec?

All the best,
Karsten

comment:3 Changed 4 years ago by virgil

What data does a history object contain?

A history object contains graph data, which can be anything from written bytes over consensus weights to uptime.

Q: Just to be clear, this means Onionoo provides access to the graph data points, but does not provide access to the raw historical data?

If the answer is yes: could we start making the historical data available? The most we'd ever want to go back is 6-12 months. If this is too niche a need to be on Onionoo we could keep this data on the Roster server itself.

comment:4 Changed 4 years ago by karsten

Onionoo provides bandwidth data from the last 1 month on the same granularity as reported by recent Tor versions, which is 1 data point for 4 hours. Earlier data is aggregated to 1 data point for 12 hours, then 2 days, then 10 days. This is an effective means to keep Onionoo documents at a roughly fixed size. If there's a good reason to keep details around for longer, let's discuss that. What's your use case here?

Keeping data on Roster itself is not a good option, as I stated before. It should be stateless (except for caching) and rely on Onionoo to keep all state it needs.

comment:5 Changed 4 years ago by virgil

Keeping data on Roster itself is not a good option, as I stated before. It should be stateless (except for caching) and rely on Onionoo to keep all state it needs.

We are currently on course for Roster to have a state when it comes to replacing Tor Weather (knowing which operators have redeemed their shirts), but I hear you loud and clear.  We (Sean and I) will go above and beyond to minimize any kept state.

What's your use case here?

We want the points to extend back in time (for simplicity, lets just say 100 days).  The current badges are: {bandwidth, exit bandwidth, consensus weight}(probably more later).   So we'd like to be able to get each of these per extended_family (ideally per relay, but we'll take what we can get), calculated per day, for the past 100 days.  Worst comes to worst, we could just retain the relevant Onionoo data from the past 100 days.  We are not 100% opposed to this, but that's obviously a lot of state to keep, and per above you want us to minimize state.  What's your suggested path forward?  No wrong answers.

comment:6 Changed 4 years ago by karsten

If the last 92 days are enough (roughly 3 months), Onionoo will give you two data points per day. If you need those 100 days or more, Onionoo will give you one data point every two days for up to 366 days. Does that work for you?

comment:7 Changed 4 years ago by virgil

If the last 92 days are enough (roughly 3 months), Onionoo will give you two data points per day. If you need those 100 days or more, Onionoo will give you one data point every two days for up to 366 days. Does that work for you?

Yeah that data-granularity works just fine.  However, it *sounds like* the that is the only data available is the graph-points normalized between [0,1000].  We'd need the raw data at the level of granularity you specified above.  Do'able?

comment:8 Changed 4 years ago by karsten

Sure, you'd have to multiply the normalized values with the provided factor. Assuming that granularity is sufficient. If not, please explain why you think it isn't.

comment:9 Changed 2 years ago by karsten

Resolution: not a bug
Status: needs_informationclosed

Assuming the documentation is sufficient, or that future readers will open a new ticket. Closing.

Note: See TracTickets for help on using tickets.