Opened 4 years ago

Closed 4 years ago

#12813 closed task (fixed)

Look at a bitmap visualization of relay consensus

Reported by: dcf Owned by:
Priority: Medium Milestone:
Component: Metrics/Website Version:
Severity: Keywords:
Cc: phw Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I tried a visualization of the hourly consensus that might be useful.

Warning, big graphic (5832×39026 pixels)

Each row in the graphic is one fingerprint. Each column is one hour in the consensus, from January 2014 on the left to August 2014 on the right. Black pixels are when the fingerprint was in the consensus. The rows are sorted first by total weight, then by center of mass of the black pixels (so two rows with the same weight will have one that is more to the left, higher).

The rows in the graphic map 1:1 to lines in the text file. You can open the graphic in Gimp and hover over an interesting area to get the row number. Then you can jump to that line number (plus one) in the text file to see what descriptors are there.

Why this visualization is interesting is some visually noticeable features have a meaning. For example, near the top at about rows 1469 through 1508 and 1554 through 1610 are two heavy black areas that start and stop at almost the same time. It turns out, if you look at the corresponding lines in the text file, that those blocks belong to the attacker of https://blog.torproject.org/blog/tor-security-advisory-relay-early-traffic-confirmation-attack. There are other weird and interesting features that I haven't looked at yet.

Child Tickets

Attachments (1)

microdescs-2014-tiny.png (122.6 KB) - added by dcf 4 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 Changed 4 years ago by dcf

Status: newneeds_review

My colleague Roya thought of doing this kind of visualization.

Source code is:

git clone git clone https://www.bamsoftware.com/git/microdescs.git

Usage is:

wget https://collector.torproject.org/archive/relay-descriptors/microdescs/microdescs-2014-01.tar.xz
tar xJf microdescs-2014-01.tar.xz
...
wget https://collector.torproject.org/archive/relay-descriptors/microdescs/microdescs-2014-08.tar.xz
tar xJf microdescs-2014-08.tar.xz
go build
./microdescs --output microdescs-2014 microdescs-2014-??

The main program is microdescs.go. There is also a microdescs.py that is slower and handles just one month at a time.

Karsten suggested that the program could be adapted to non-micro descriptors, so that it would be able to go farther back in time.

Last edited 4 years ago by dcf (previous) (diff)

Changed 4 years ago by dcf

Attachment: microdescs-2014-tiny.png added

comment:2 Changed 4 years ago by dcf

Here's a little thumbnail of the huge graphic at https://people.torproject.org/~dcf/graphs/microdescs/microdescs-2014.png.


comment:3 Changed 4 years ago by phw

Cc: phw added

comment:5 Changed 4 years ago by dcf

Resolution: fixed
Status: needs_reviewclosed

All right it's been looked at.

Note: See TracTickets for help on using tickets.