Opened 5 years ago

Last modified 10 months ago

#9778 assigned enhancement

Add votes document type

Reported by: karsten Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Onionoo Version:
Severity: Normal Keywords:
Cc: phw, rndm, wfn, arma Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

A few weeks ago, I changed the web output of the consensus-health checker to not display relay flags anymore. Here's an archived example (with broken CSS, sorry for that):

https://people.torproject.org/~karsten/volatile/consensus-health-2013-08-12-07-00-00.html

The main reason was that the page loads forever, which makes the remaining information on the page less useful. Though this will soon be irrelevant, because I'm planning to shut down the consensus health website entirely in favor of Damian's Python consensus-health checker.

We should try to rescue all information from that page that we want to keep. Roger says on IRC that he's interested in the flags that the directory authorities assign to a relay in a vote.

I wonder if we should add a new document type to Onionoo that contains useful information from votes, like flags and measured bandwidths. (I briefly thought about adding more stuff to details documents, but they're already pretty overloaded right now.) Here's how a votes document could look like for gabelmoo:

{
    "relays_published": "2013-09-19 09:00:00",
    "relays": [
        {
            "nickname": "gabelmoo",
            "fingerprint": "F2044413DAC2E02E3D6BCF4735A19BCA1DE97281",
            "flags": {
                "Faravahar": [
                    "Authority",
                    "HSDir",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ],
                "dannenberg": [
                    "Authority",
                    "HSDir",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ],
                "dizum": [
                    "Authority",
                    "HSDir",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ],
                "gabelmoo": [
                    "Authority",
                    "Named",
                    "Running",
                    "V2Dir",
                    "Valid"
                ],
                "maatuska": [
                    "Authority",
                    "HSDir",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ],
                "moria1": [
                    "Authority",
                    "HSDir",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ],
                "tor26": [
                    "Authority",
                    "HSDir",
                    "Named",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ],
                "urras": [
                    "Authority",
                    "HSDir",
                    "Running",
                    "Stable",
                    "V2Dir",
                    "Valid"
                ]
            },
            "measured": {
                "moria1": 9
            }
        }
    ],
    "bridges_published": "2013-09-19 08:37:03",
    "bridges": []
}

What else might be worth adding to these documents? Here's what moria1 thinks about gabelmoo:

r gabelmoo 8gREE9rC4C49a89HNaGbyh3pcoE goM6QETDkFBEjjhRURvUUn2QDkU 2013-09-18 22:01:15 212.112.245.170 443 80
s Authority HSDir Running Stable V2Dir Valid
v Tor 0.2.5.0-alpha-dev
w Bandwidth=20 Measured=9
p reject 1-65535
m 8,9,10,11,12,13,14,15 sha256=lLaUe8rgCb1VRPKWKSC7dP+DLae6r62FyIOh/6+Mm8c
m 16,17 sha256=he0Eot/jfJv/uxqCnUFjkTLLbLbiwRtWIae9s0gyUYU

phw, rndm: If I add these documents to Onionoo, would you want to include them in relay detail pages of Atlas and Globe? Would you expect the data format to be different? Would you want less/more information from votes?

wfn: I think it makes sense to add these documents to good-old-Java-Onionoo, so that there's just one more document type that your shiny-new-Python-Onionoo can treat like summary, details, bandwidth, and weights documents. Does that work for you?

arma: Is there anything missing in this proposal that you'd want to have included?

Child Tickets

Change History (9)

comment:1 Changed 5 years ago by wfn

Karsten:

wfn: I think it makes sense to add these documents to good-old-Java-Onionoo, so that there's just one more document type that your shiny-new-Python-Onionoo can treat like summary, details, bandwidth, and weights documents. Does that work for you?

Introducing a completely separate document type / API point for votes makes sense to me. As we briefly discussed on IRC, I'd be curious to maybe try and eventually implement this new votes document in the new python-onionoo / torsearch itself. But for now, I'll assume that the new votes document should be treated like the Onionoo weights and bandwidth documents (i.e., "outsourced" to the java-onionoo proper.)

I think Roger hinted at the idea that it might be useful to be able to query Onionoo/whatever for vote info on past consensuses as well, to be able to compare things, etc. It would indeed (as Karsten said) be probably a very DB-expensive (and disk i/o-bound) operation / API point; but if there's interest, I could try later on looking at possible solutions (as I said, Redis and/or PostgresSQL's 'hstore' might be the proper tool(s) for the job.)

comment:2 Changed 5 years ago by karsten

I can see how it would be interesting to add a small portion of vote history, but I don't see yet how this would scale. Let's postpone this discussion until we know that current vote information is useful, and then open a new ticket for it if we still think it's a good idea. This ticket should only be about the most recent vote information we have about a relay.

comment:3 in reply to:  description Changed 5 years ago by phw

Replying to karsten:

phw, rndm: If I add these documents to Onionoo, would you want to include them in relay detail pages of Atlas and Globe? Would you expect the data format to be different? Would you want less/more information from votes?

Yes, I could include it (depending on how much JavaScript hackery this would be, I might need some external help). The data format looks OK to me.

However, from an Atlas user's point of view, it might be more interesting to just display to which extent the authorities (dis)agree. If the flags are all identical, I would just display the flags together with something along the lines of "all authorities agree". This could even be reflected in the data format already but that probably complicates things too much.

comment:4 Changed 5 years ago by karsten

Actually, we could reflect this in the data format. It would reduce document size a lot. Here's the example from above if we only include flags that the authorities disagree about:

{
    "relays_published": "2013-09-19 09:00:00",
    "relays": [
        {
            "nickname": "gabelmoo",
            "fingerprint": "F2044413DAC2E02E3D6BCF4735A19BCA1DE97281",
            "missing_flags": {
                "gabelmoo": [
                    "HSDir",
                    "Stable"
                ],
            },
            "measured": {
                "moria1": 9
            }
        }
    ],
    "bridges_published": "2013-09-19 08:37:03",
    "bridges": []
}

The missing_flags field would only list flags that are contained in the consensus but which an authority didn't vote on. (If an authority doesn't vote on a certain flag, that flag wouldn't be listed here; otherwise the dict would contain Faravahar, dannenberg, etc. not voting on Named, and that doesn't make sense.)

There would also be an extra_flags field if one of the authorities voted on a flag that didn't make it into the consensus. But if that's not the case, that field would be omitted.

I wonder if we should simply include these three new fields (missing_flags, extra_flags, and measured) in the details documents. Adding a new document type has some overhead that we'd avoid here. I guess I'll write a quick implementation and see how much bigger details documents will be.

But before I do this, I'd like to know what we're going to build as alternative to the consensus-health web page: https://lists.torproject.org/pipermail/tor-dev/2013-September/005476.html. This ticket was suggested as one solution to replace that web page, though I can see how it would be useful to add this information to Atlas et al. regardless of that discussion. Still, let's see how that goes before building the wrong thing.

Thanks for your response!

comment:5 Changed 5 years ago by karsten

Damian says on @tor-dev: "Actually, it would be pretty slick if hovering your mouse over the flags showed the votes for each authority. I still disagree that any except a vanishingly small group will find this useful, but if this is a use case we want to support then Atlas is the spot."

I like the idea of displaying this information in a tooltip. Though I see two problems here:

  1. We don't include which authorities voted on a given flag. So if we decide for the missing_flags format, Atlas can't tell which authorities did vote on the flag. Though it could display in the tooltip which authorities did not vote on the flag.
  1. There's nothing that the user could hover over to learn about flags in votes that didn't make it into the consensus. This would be the tooltip that would display extra_flags. Not sure how to solve this.

comment:6 in reply to:  5 ; Changed 4 years ago by alphawolf

Replying to karsten:

I like the idea of displaying this information in a tooltip. Though I see two problems here:

  1. We don't include which authorities voted on a given flag. So if we decide for the missing_flags format, Atlas can't tell which authorities did vote on the flag. Though it could display in the tooltip which authorities did not vote on the flag.

What if we include an array of authorities that voted in the current consensus? Since the document is designed for machine consumption, we don't actually need to repeat the authority names under each "missing_flags"/"extra_flags" property. Instead, these properties become arrays of arrays, with the index of each subarray corresponding to the index of the authority. Like so:

{
    "relays_published": "2013-09-19 09:00:00",
    "authorities": [
        "Faravahar",
        "dannenberg",
        "dizum",
        "gabelmoo",
        "maatuska",
        "moria1",
        "urras"
    ],
    "relays": [
        {
            "nickname": "gabelmoo",
            "fingerprint": "F2044413DAC2E02E3D6BCF4735A19BCA1DE97281",
            "missing_flags": [
                [],[],[],["HSDir","Stable"],[],[],[] 
            ],
            "measured": {
                "moria1": 9
            }
        }
    ],
    "bridges_published": "2013-09-19 08:37:03",
    "bridges": []
}

In this case, missing_flags[3] corresponds to authorities[3] ("gabelmoo") and contains ["HSDir","Stable"]. The remaining 6 listed authorities can be assumed to have voted for the flags. Turtles and tor26 took the day off, so they are not listed.

The "Named" and "Unnamed" flags present a problem currently, but it is my understanding that they are on their way out. Are there other flags that only a subset of the authorities vote on?

  1. There's nothing that the user could hover over to learn about flags in votes that didn't make it into the consensus. This would be the tooltip that would display extra_flags. Not sure how to solve this.

Atlas could show the flags grayed out, in red, or with a strike through (or a combination) if a flag was not in the consensus. Hovering over the "inactive" flag could then show the missing/extra flags tooltip.

While we're working out the document format, I propose shortening the flags to a 1-2 character token each. The property names under "relays" could also be shortened. Saving a few bytes in this area will end up saving megabytes when votes for all relays are requested.

comment:7 in reply to:  6 Changed 4 years ago by karsten

Replying to alphawolf:

What if we include an array of authorities that voted in the current consensus? Since the document is designed for machine consumption, we don't actually need to repeat the authority names under each "missing_flags"/"extra_flags" property. Instead, these properties become arrays of arrays, with the index of each subarray corresponding to the index of the authority.

Hmm, I think we can't move authority names to the top level, for several reasons. Most importantly, it may always be the case that an authority doesn't vote on a flag, even when "Named" and "Unnamed" are gone. To make this even more complicated, there's an open Tor proposal (proposal 177) that suggests allowing authorities to abstain from voting on flags for only a subset of relays.

Another reason why I'd like to avoid adding new fields to the top level is that all response types return exactly these four fields at the top level: "relays_published", "relays", "bridges_published", "bridges". This is done to simplify parsing responses by clients. Adding a new "authorities" field just for details responses would break that simplification.

Yet another reason is that we're putting together responses from previously written JSON documents. These pre-written documents may be days old, and it might well be the case that some authorities took that day off but not the current day. So, any object contained in "relays" or "bridges" must be self-contained.

  1. There's nothing that the user could hover over to learn about flags in votes that didn't make it into the consensus. This would be the tooltip that would display extra_flags. Not sure how to solve this.

Atlas could show the flags grayed out, in red, or with a strike through (or a combination) if a flag was not in the consensus. Hovering over the "inactive" flag could then show the missing/extra flags tooltip.

Right, we can do that! Good idea!

While we're working out the document format, I propose shortening the flags to a 1-2 character token each. The property names under "relays" could also be shortened. Saving a few bytes in this area will end up saving megabytes when votes for all relays are requested.

Not sold on that one. The actual saving shouldn't be that big with compression turned on (yes, yes, that's a different ticket). And while documents are designed for machine consumption, there's always a tiny fraction of people who read machine-readable documents because there's no good user interface (yet). There's also the issue that clients will have to understand both full and shortened field names at least for a while. That requires quite some coordination, and in practice we'll still end up with broken clients.

Are there any other ways to improve the document format for including vote information? And should we consider additional information contained in the proposed "vote-info" document type (proposal 164)?

Thanks for your feedback here!

comment:8 Changed 11 months ago by karsten

Owner: changed from karsten to metrics-team
Status: newassigned

Handing over to metrics-team, because I'm not currently working on this.

comment:9 Changed 10 months ago by karsten

Severity: Normal
Summary: Adding votes documents to OnionooAdd votes document type
Note: See TracTickets for help on using tickets.