Opened 5 years ago

Closed 4 years ago

#6171 closed enhancement (implemented)

Export Tor descriptors with Csv, Xml, Json: tor2csv, Tor2xml, tor2json

Reported by: naif Owned by: atagar
Priority: Medium Milestone:
Component: Core Tor/Stem Version:
Severity: Keywords: descriptors
Cc: atagar Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

This ticket to add to Stem a functionality to Stem (as set of small sample applications) that export the tor descriptors in various useful formats:

  • csv
  • xml
  • json

A similar attempt for csv has been done on https://github.com/mrlanrat/TorExport

A post to tor-dev mlist about it is at https://lists.torproject.org/pipermail/tor-dev/2012-June/003633.html

A chat discussion is at:

naif
atagar: for example the TorExport tried to do that goal (by torservers) https://github.com/mrlanrat/TorExport but it's not working properly
atagar
naif: Hm, so you're thinking that the use case is 'write a python app to parse and do something with the descriptors (say, filter), then export the results to a csv'? I could see use in that, though how should we handle new descriptor additions? Add another positional field that might not be expected by callers?
4:16
rransom
atagar: SQL! After all, it works *soooo* nicely for every Tor Python program that uses it.
4:17
naif
atagar: you made the right definition of what i mean
atagar: a set of python app that use stem to do conversion to csv, json, xml for content of tor descriptors
naif
atagar: i mean, i am a sysadmin, not a coder, so for me having something that i can easily sed/awk and/or load with simple CSV library it's *extremely* useful
atagar
naif: Sounds good. I'd be happy to add a 'get_csv()' method to decriptor classes and have a little converter like TorExport. However, I'm still not sure what you would like to have happen when there are descriptor additions.
4:20
atagar
Add the new fields to the end of the csv or something else?
4:20

naif
atagar: it make sense
atagar: tipically within CSV files the 1st line it's used to describe various fields
atagar: we may have as a convention that the 1st line always bring the description of the descriptor addition fields?
4:21
4:21
atagar
naif: Ahh, ok. Please file a ticket, it would be an easy task for one of the new people to pick up. If there's a specific pressing need for it then I could easily hack it out.

Child Tickets

Change History (4)

comment:1 Changed 5 years ago by runa

  • Cc runa removed

comment:2 Changed 4 years ago by atagar

Shame on me for not updating this ticket. The Wesleyan students added csv exporting functionality to stem a while back...

https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/descriptor/export.py

With this we should have parity with the TorExport script that you mentioned, but not export capacity for the other formats. They wouldn't be hard to add but I'd rather hold off until we have someone who needs it for their project.

Leaving this ticket open for now but I'll resolve it eventually if there's no interest.

comment:3 Changed 4 years ago by atagar

  • Keywords descriptors added

comment:4 Changed 4 years ago by atagar

  • Resolution set to implemented
  • Status changed from new to closed

I'm gonna go ahead and resolve this. We can extend the exporter (and maybe add an importer) as demand arises.

Note: See TracTickets for help on using tickets.