Opened 8 years ago

Closed 5 years ago

#4180 closed enhancement (wontfix)

Move statistical portions of censorship detector to R from python

Reported by: tomb Owned by: tomb
Priority: Medium Milestone:
Component: Metrics/Analysis Version:
Severity: Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I would like to make our approach to censorship detection more inline with how we do torperf: I'd like to do pre-processing of the data in python, and then do the actual stats in R. I think it will be easier to follow and extend the actual math. I am in the process of considering additional statistical methods of anomaly detection, and R will be better suited towards examining multiple methods and graphing them for comparison and visualization

Child Tickets

Change History (4)

comment:1 Changed 8 years ago by karsten

Component: Metrics UtilitiesAnalysis

Rewriting parts of George's Python-based censorship detector in R sounds useful if it helps researching alternative approaches. Whenever you have code for that, I'll add it to the metrics-tasks repository.

I'm changing the component to Analysis though. The goal is still to compare detection algorithms and determine which one we want to deploy. Turning the algorithm(s) we like most into a tool is the next step and probably requires a new ticket. And even then I'm not sure if this will be a stand-alone tool for users (which is what Metrics Utilities is for) or rather a web-based service, e.g., part of the Metrics Website.

comment:2 in reply to:  1 Changed 8 years ago by tomb

Replying to karsten:

Rewriting parts of George's Python-based censorship detector in R sounds useful if it helps researching alternative approaches. Whenever you have code for that, I'll add it to the metrics-tasks repository.

Yes, we are working right now on a couple of alternative algorithms with the goal of contrasting them. We may find that one is best overall, or we may find that different approaches work better under specific conditions.

The port to R is intended to make it easier to prototype and graph quickly.

I'm changing the component to Analysis though. The goal is still to compare detection algorithms and determine which one we want to deploy. Turning the algorithm(s) we like most into a tool is the next step and probably requires a new ticket. And even then I'm not sure if this will be a stand-alone tool for users (which is what Metrics Utilities is for) or rather a web-based service, e.g., part of the Metrics Website.

Sounds good!

comment:3 Changed 7 years ago by karsten

Parent ID: #2718

Removing the parent ticket relationship to #2718, because I want to close that ticket (which Trac doesn't permit while there are open child tickets). We have an approach for detecting country-wide blockings deployed. There will always be room for improvements. If this ticket comes up with a better approach, we'll deploy it.

comment:4 Changed 5 years ago by karsten

Resolution: wontfix
Status: newclosed

Not seeing any relevance of this ticket anymore.

Note: See TracTickets for help on using tickets.