Allow filtering relays by version ranges
It would be much easier to find out how many relays are still vulnerable to bugs like #6811 (moved) if there were an easy way to search for relays running versions of Tor in a specified range.
Activity
changed milestone to %Onionoo 1.18.0
That use case (How much overall tor bandwidth is provided by vulnerable relays) was also my motivation for #6855 (moved) - great to see someone else seeing that use case too.
You are probably interested in karsten's reply in #6855 (moved) - which somehow covers also your feature request in this ticket.
Trac:
Username: cypherpunkx-
Trac:
Username: tyseom
Cc: N/A to nusenu@openmailbox.org 
In #23517 (moved) it is planned to merge Compass functionality with Relay Search (formerly known as Atlas). These tickets may be relevant to that work and so these are being reassigned to the Metrics/Atlas component.
Trac:
Component: Metrics/Compass to Metrics/Atlas
I started to think about this today and came up with a few design questions that we need to answer first:
-
Dashes in tor versions: As of now, a query for
/details?version=0.3.4.5-rcreturns relays with version "0.3.4.5-rc". But the dash is also a quite intuitive notation for a range, just like we're using a dash in thefirst_seen_daysandlast_seen_daysparameters. If we're going to support version ranges, we might want to use the dash for queries like/details?version=0.3.4.5-0.3.4.6. If we do, the first query won't work anymore, and would have to be changed to/details?version=0.3.4.5. The result set will remain the same, it's just that people will get it wrong in the beginning. Needs a major protocol update. -
Prefix matching: The query
/details?version=0.3.2.1returns relays with version 0.3.2.1 as well as 0.3.2.10, 0.3.2.11, and so on. Following that logic, which versions would we expect that the query/details?version=0.3.2.2-0.3.2.1returns? Everything from 0.3.2.1 to 0.3.2.29, plus 0.3.2.100 to 0.3.2.199, etc. Maybe we should consider changing the matching algorithm to attempt to parse the version (prefix) and do the matching numerically. In this case the example above would simply be rejected, and a query/details?version=0.3.2.1-0.3.2.2would just return those two versions and nothing else. Needs a major protocol update. -
Comma-separated versions or version ranges: I could imagine use cases where somebody wants all relays with non-consecutive version numbers. While we're changing this parameter for version ranges we could as well start supporting comma-separated versions or version ranges. As in:
/details?version=0.3.2.1,0.3.2.5. Needs a minor protocol update unless combined with the changes above.
Thoughts?
Trac:
Cc: nusenu@openmailbox.org to nusenu@openmailbox.org, metrics-team-
-
Replying to irl:
We have version ordering logic now thanks to the
version_statusfield.Yes! It might that we need to write some more code to also support version prefixes, but most of the code should already be there.
We can have a
from_versionandto_versionparameter?Hmm, let's ignore for a moment that we already have a
versionparameter and let's discuss how we would like to build version-related parameters. Wouldn't it be more intuitive if we had a singleversionparameter that supports single versions, dash-separated version ranges, and possibly even comma-separated versions or version ranges? Keep in mind that some users may want to use these parameters as qualified search terms.Maybe my list of design questions looked too alarming, but that was not my intention. I think it's perfectly fine to make a major protocol update, if it's a useful change. We just need to give folks the chance to update their clients, bookmarks, etc. But if the result is a more intuitive interface, that's worth the effort.
However, I'm not sure what the most intuitive interface is. Maybe it's something else, like a different character for ranges, such as
/details?version=0.3.2.1:0.3.2.5. (If so, we should consider changingfirst_seen_daysandlast_seen_days, too, for consistency.)We can then separately also have comma separated versions for prefix matches.
-
Replying to karsten:
However, I'm not sure what the most intuitive interface is. Maybe it's something else, like a different character for ranges, such as
/details?version=0.3.2.1:0.3.2.5. (If so, we should consider changingfirst_seen_daysandlast_seen_days, too, for consistency.)I think using
:for ranges is better, as long as we can be certain that we'll never see a:in a released Tor version. We should probably require this in torspec somewhere. If we do a major version change, which I think we should for this, then we would also change{first,last}_seen_days. -
Replying to irl:
Replying to karsten:
However, I'm not sure what the most intuitive interface is. Maybe it's something else, like a different character for ranges, such as
/details?version=0.3.2.1:0.3.2.5. (If so, we should consider changingfirst_seen_daysandlast_seen_days, too, for consistency.)I think using
:for ranges is better, as long as we can be certain that we'll never see a:in a released Tor version. We should probably require this in torspec somewhere. If we do a major version change, which I think we should for this, then we would also change{first,last}_seen_days.Hmm, on second thought,
:wouldn't work so well with qualified search terms. Imagine a search forversion:0.3.2.1:0.3.2.5(for versions 0.3.2.1, 0.3.2.2, 0.3.2.3, 0.3.2.4, and 0.3.2.5) or evenversion::0.3.2.5(for everything until 0.3.2.5) orversion:0.3.2.1:for 0.3.2.1 or newer).New idea (sorry for the brainstorming, I'm just trying to find the best solution):
.., as inversion:0.3.2.1..0.3.2.5orversion:..0.3.2.5orversion:0.3.2.1..for the examples given above. Yet unimplemented, so no guarantee that it will work well in all cases. We'd only have to require that versions don't contain two consecutive., but that should be a safe assumption. What do you think? Are there better alternatives? -
Replying to karsten:
Hmm, on second thought,
:wouldn't work so well with qualified search terms. Imagine a search forversion:0.3.2.1:0.3.2.5(for versions 0.3.2.1, 0.3.2.2, 0.3.2.3, 0.3.2.4, and 0.3.2.5) or evenversion::0.3.2.5(for everything until 0.3.2.5) orversion:0.3.2.1:for 0.3.2.1 or newer).Good point.
New idea (sorry for the brainstorming, I'm just trying to find the best solution):
.., as inversion:0.3.2.1..0.3.2.5orversion:..0.3.2.5orversion:0.3.2.1..for the examples given above. Yet unimplemented, so no guarantee that it will work well in all cases. We'd only have to require that versions don't contain two consecutive., but that should be a safe assumption. What do you think? Are there better alternatives?Yes, I think a double
.is safer than a:. I know that:is perfectly valid in Debian package version numbers and perhaps there would be other clients that use it in the future. -
Please review commit 9740ce7 in my task-6947 branch. Once this patch is merge-ready, let's schedule the next major version for a month later and put the merge on hold for that time. Thanks!
Trac:
Status: accepted to needs_review mentioned in issue #27442 (moved)