Opened 5 months ago

Last modified 12 days ago

#28152 new enhancement

Gettor code refactor with Python Twisted

Reported by: ilv Owned by: ilv
Priority: Very High Milestone:
Component: Applications/GetTor Version:
Severity: Major Keywords:
Cc: hiro Actual Points:
Parent ID: #28232 Points:
Reviewer: Sponsor:


Code refactor

Gettor needs some love. It should be more robust to make it: easier to maintain (by me or somebody else), to know when it is working or not, and to allow more developers to contribute to it.

For the above, I propose to refactor the current code and turn it into a twisted daemon [1, 2]. This would preserve the main logic of the current system and add all the benefits of having a daemonized application. This service approach considers two main parts:

  1. Distribution channels. Whenever gettor receives a request or sends a reply it uses a channel (e.g. e-mail). Each channel could be handled by one or more services. These services would be constantly fetching and updating information in a SQLite database to know how to proceed.

In the case of e-mail, there should be a script that receives messages forwarded by the MTA, process them, and add a request with a given status to the SQLite database. On the other hand, a service running on background will be fetching ready-to-be-sent requests from the database and send e-mails with the requested information.

For a twitter bot, a single service that receives DMs, process them and send replies would be enough.

  1. Tor Browser sync. A service constantly checking new Tor Browser releases, downloading the new packages and updating the SQLite database with the new links.

The logging system provided by twistd is easy to use and works very well. This will solve one of the problems with the current code and the use of logging, also providing useful information for debugging and statistics. Log rotation is automatic.

I have developed a similar service using twistd. Adapting it to gettor would be fairly easy and it would take me a few weeks of spare time.

Twisted is not installed on getulum, so I will collect all the needed packages and ask for them to be installed.


Child Tickets

#1593taskassignedkanerImplement test (-t switch) functionality
#3780enhancementnewkanerGetTor and BridgeDB should share email processing code
#28091taskneeds_revisiontraumschulePort GetTor to python3
#28339defectnewilvLog handling in gettor
#28340defectnewilvPackage gettor for Debian
#28905taskclosedtpaPlease install packages for twisted gettor

Change History (4)

comment:1 Changed 5 months ago by traumschule

See also #28091

comment:2 Changed 5 months ago by traumschule

Parent ID: #28232

#28232 now groups GetTor working areas.

comment:3 Changed 4 months ago by hiro

Hi I have been looking at the code of gettor and bridge db. I am thinking many of the functionalities implemented in twistd in bridgedb could be easily ported.

Also I think it would help to have scripts between:

  • development scripts
  • install scripts

With developmente scripts I would group everything that can be used or is needed while developing or testing gettor.
With install scripts I would group scripts that can be used to install and deploy the service.

Last edited 4 months ago by hiro (previous) (diff)

comment:4 Changed 12 days ago by pili

Cc: hiro added
Note: See TracTickets for help on using tickets.