Opened 5 months ago

Last modified 4 months ago

#30174 needs_information defect

possible SBWS measurement quality regression

Reported by: starlight Owned by:
Priority: Medium Milestone: sbws: 1.2.x-final
Component: Core Tor/sbws Version: sbws: 1.1.0
Severity: Normal Keywords:
Cc: juga Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

With the release of v1.1 to longclaw seeing evidence of measurement quality degradation, regression. Have not had time to work up an analysis.

Recommend running Matt's original scanner and comparing raw bandwidth results.

Child Tickets

Change History (3)

comment:1 in reply to:  description Changed 5 months ago by juga

Milestone: sbws: unspecified
Status: newneeds_information
Version: sbws: 1.1.0

Hi starlight,

Thanks for reporting this.

Replying to starlight:

With the release of v1.1 to longclaw seeing evidence of measurement quality degradation, regression. > Have not had time to work up an analysis.

If you have time it would be great to know:

  • respect to which scanner and version you see degradation and regression
  • where/how do you see that


Recommend running Matt's original scanner and comparing raw bandwidth results.

  • which is Matt's original scanner. I guess you mean sbws < 1.x.x?. Most of the changes respect to previous versions have been done on the scaling part, not the measuring one.

comment:2 Changed 5 months ago by starlight

According to the vote document it's "software_version=1.1.0", what I say above in the description (which you copy in reply), is the current git release and you posted about it on tor-relays. This ticket is under SBWS so what doubt inspires the scanner/version question?

The out-of-whack numbers I observed reverted--SBWS requires a substantial ramp-up time before quality is achieved. Torflow has an explicit mechanism to delay publishing until equilibrium is present, suggest that SBWS employ something similar.

My recommendation is to examine raw votes in comparison to the POC scanner Matt authored as a QA validation. I still like the idea, even with the vote quality improvement.

comment:3 Changed 4 months ago by teor

Milestone: sbws: unspecifiedsbws: 1.2.x-final
Note: See TracTickets for help on using tickets.