#25387 closed enhancement (fixed)

Add comment section to CSV file headers

Reported by: karsten Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Website Version:
Severity: Normal Keywords:
Cc: metrics-team Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

From #25382:

Starting suggestion:

##
## The Tor Project
##
# URL:
#  https://metrics.torproject.org/networksize.html?start=2017-11-30&end=2018-02-28
# Parameters:
#  networksize: start=2017-11-30 end=2018-02-28
#
# Legend:
#  date: UTC date (YYYY-MM-DD) when relays or bridges have been listed as running.
#  relays: average number on the given day.
#  bridges: average number on the given day.
#
date,relays,bridges
2017-11-30,6512,1955
2017-12-01,6629,1959
2017-12-02,6647,1963
2017-12-03,6650,1976
...

We need to use write.table, for example:

## the data summary
summary(y)
         date        users       downturns        upturns            lower           upper     
 2018-01-01: 1   Min.   :716.0   Mode :logical   Mode :logical   Min.   :165.0   Min.   :1039  
 2018-01-02: 1   1st Qu.:755.0   FALSE:58        FALSE:58        1st Qu.:403.5   1st Qu.:1106  
 2018-01-03: 1   Median :780.0                                   Median :455.0   Median :1173  
 2018-01-04: 1   Mean   :778.9                                   Mean   :444.1   Mean   :1179  
 2018-01-05: 1   3rd Qu.:802.5                                   3rd Qu.:496.2   3rd Qu.:1225  
 2018-01-06: 1   Max.   :858.0                                   Max.   :593.0   Max.   :1584  
 (Other)   :52                                                                                 


# writing
write("# some comments", file="data.csv")
write("# some more comments", file="data.csv", append = TRUE)
write.table(y, file="data.csv", append = TRUE, quote=FALSE, sep=",", row.names = FALSE)

Yields:

# some comments
# some more comments
date,users,downturns,upturns,lower,upper
2018-01-01,787,FALSE,FALSE,369,1319
2018-01-02,754,FALSE,FALSE,427,1268
2018-01-03,791,FALSE,FALSE,485,1107
2018-01-04,823,FALSE,FALSE,452,1119
...

We'll be able to re-use quite some content from the current stats.html.

Child Tickets

Change History (4)

comment:1 Changed 11 months ago by karsten

Status: newneeds_review

I started an implementation in my task-25387 branch. It's also temporarily deployed right now, but not merged to master yet.

In contrast to the suggestions above, I added the header in Java, not in R. I think that's going to be more flexible if we make the header more complex.

I did not yet include parameters or a legend. That will require more effort and might not be done this week. If the current changes look okay, I'll merge and deploy permanently. We could still keep this ticket open to do more.

Please take a quick look!

comment:2 in reply to:  1 ; Changed 11 months ago by iwakeh

Replying to karsten:

I started an implementation in my task-25387 branch. It's also temporarily deployed right now, but not merged to master yet.

In contrast to the suggestions above, I added the header in Java, not in R. I think that's going to be more flexible if we make the header more complex.

Using Java for all this is surely the better solution here.

I did not yet include parameters or a legend. That will require more effort and might not be done this week. If the current changes look okay, I'll merge and deploy permanently. We could still keep this ticket open to do more.

I understand that legend is some more work, but parameter would be simply adding 'HttpServletRequest::getQueryString()' to the Url: request.getRequestURI() + "?" + request.getQueryString().
The complete URL including parameters is important for reproducing the csv files.

With added parameters this patch is merge ready and this ticket would be finished.
For the legend descriptions we should open a new ticket as part of the sponsor13 work?

comment:3 Changed 11 months ago by iwakeh

Status: needs_reviewneeds_revision

comment:4 in reply to:  2 Changed 11 months ago by karsten

Resolution: fixed
Status: needs_revisionclosed

Replying to iwakeh:

Replying to karsten:

I did not yet include parameters or a legend. That will require more effort and might not be done this week. If the current changes look okay, I'll merge and deploy permanently. We could still keep this ticket open to do more.

I understand that legend is some more work, but parameter would be simply adding 'HttpServletRequest::getQueryString()' to the Url: request.getRequestURI() + "?" + request.getQueryString().
The complete URL including parameters is important for reproducing the csv files.

Changed!

With added parameters this patch is merge ready and this ticket would be finished.

Great! Merged and deployed the changed branch.

For the legend descriptions we should open a new ticket as part of the sponsor13 work?

Yes, I'll do that. Closing this ticket. Thanks!

Note: See TracTickets for help on using tickets.