Opened 4 years ago

Closed 4 years ago

#13565 closed task (fixed)

Work on script that generates the data formats for visualisation team

Reported by: hellais Owned by: kudrom
Priority: Medium Milestone:
Component: Archived/Ooni Version:
Severity: Keywords: ooni_data_analytics_team
Cc: asn, sysrqb, kudrom, aagbsn, infinity0, joelanders, shidash, otr Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

This involves querying the database tables reports and measurements and adding data to a new table called "bridge_reachability" or "bridge_reachability_timeline" where all the data that is needed to visualise the data is placed.

The data format that is needed by them is the following: https://transparencytoolkit.org/pretty_data.json with the addition of the key "status" that can have 4 values:

  • working (the bridge works, control == true, test == true)
  • blocked (the bridge is offline, control == true, test == false)
  • offline (the bridge is offline, control == false, test == false)
  • inconsistency (there is probably a bug in ooni or tor or obfsproxy, control == false, test == true)

Child Tickets

Attachments (1)

new.json (616 bytes) - added by joelanders 4 years ago.
example json format

Download all attachments as: .zip

Change History (6)

comment:1 Changed 4 years ago by hellais

Cc: otr added

Changed 4 years ago by joelanders

Attachment: new.json added

example json format

comment:2 Changed 4 years ago by hellais

The schema for the measurement will be this: https://github.com/TheTorProject/ooni-spec/blob/master/test-specs/ts-011-bridge-reachability.md#semantics-1.

The control measurement will always be assumed to be a measurement performed on the bridge in question from the country code NL.

comment:3 Changed 4 years ago by joelanders

My point with the new.json was that I think we want an extra level of nesting, "PT type," under "Country" to match the visualization.

CN:
  obfs3:
    bridge1:
      meas1
      meas2
    bridge2:
      meas1
      meas2
  obfs4:
    bridge3:
      meas1
      meas2

(each measurement might have probe_success:true/false and control_success:true/false keys)

Last edited 4 years ago by joelanders (previous) (diff)

comment:4 Changed 4 years ago by kudrom

Owner: changed from hellais to kudrom
Status: newassigned

comment:5 Changed 4 years ago by hellais

Resolution: fixed
Status: assignedclosed

This has been implemented

Note: See TracTickets for help on using tickets.