Opened 11 months ago

Closed 6 months ago

Last modified 6 months ago

#31157 closed enhancement (fixed)

Collect metrics about what type of proxies are running

Reported by: cohosh Owned by: cohosh
Priority: Medium Milestone:
Component: Circumvention/Snowflake Version:
Severity: Normal Keywords: anti-censorship-roadmap-october
Cc: arlolra, cohosh, phw, dcf, karsten, irl Actual Points: 1
Parent ID: Points: 3
Reviewer: karsten Sponsor: Sponsor28

Description

Collect metrics about what type of proxies are running (e.g., are they the standalone proxy-go instances, webextensions, web badges, etc.) and add them to the current set of metrics that are displayed.

This requires sending some additional information from the proxies to the broker in the initial poll step and will require whatever we decide to use for backwards compatable changes in the way different peices of snowflake interact with each other (#30704).

Child Tickets

Change History (17)

comment:1 Changed 11 months ago by gaba

Keywords: anti-censorship-roadmap-october added

comment:2 Changed 7 months ago by cohosh

Owner: set to cohosh
Status: newassigned

comment:3 Changed 7 months ago by cohosh

Status: assignedneeds_review

comment:4 Changed 7 months ago by cohosh

Actual Points: 1

comment:5 Changed 7 months ago by phw

Reviewer: phw
Status: needs_reviewneeds_revision

I left a bunch of comments. Other than that, the patch looks good!

comment:6 Changed 6 months ago by cohosh

I'm not seeing these comments for some reason >.< were they on the pull request itself?

comment:7 in reply to:  6 Changed 6 months ago by phw

Replying to cohosh:

I'm not seeing these comments for some reason >.< were they on the pull request itself?


Oh, I think this was my fault. I started a review but never finished it. It should now be finished. Can you see the comments?

comment:8 Changed 6 months ago by cohosh

Status: needs_revisionneeds_review

Thanks! Pushed some changes.

comment:9 in reply to:  8 Changed 6 months ago by phw

Status: needs_reviewmerge_ready

Replying to cohosh:

Thanks! Pushed some changes.


The changes are all well done!

comment:11 Changed 6 months ago by cohosh

Cc: karsten irl added
Reviewer: phwkarsten
Status: merge_readyneeds_review

Before I deploy this I'd like to hear from the metrics team about whether they or we need to so something before suddenly supplying new metrics.

The new specification is available here: https://gitweb.torproject.org/pluggable-transports/snowflake.git/tree/doc/broker-spec.txt

comment:12 in reply to:  11 Changed 6 months ago by karsten

Replying to cohosh:

Before I deploy this I'd like to hear from the metrics team about whether they or we need to so something before suddenly supplying new metrics.

The new specification is available here: https://gitweb.torproject.org/pluggable-transports/snowflake.git/tree/doc/broker-spec.txt

These changes look okay to me, from the perspective of CollecTor archiving these files. We'll have to extend metrics-lib to parse and provide the new fields, but there's no rush in doing so. CollecTor will ignore those new lines and archive files anyway. And if it doesn't, we'll fix CollecTor. Feel free to deploy these changes. I'll make a note (and later a ticket) to update metrics-lib. Thanks for asking!

comment:13 Changed 6 months ago by cohosh

Status: needs_reviewmerge_ready

Okay, great. Thanks!

comment:14 Changed 6 months ago by cohosh

Updated the broker as of 2019/12/03 19:00:28

Updated the standalone proxy-go instances on the bridge as of 2019/12/03 19:05:27

I also updated both the mozilla and chrome addon stores.

When updating the website I got the following errors:

$ rsync --chown=:snowflake --chmod ug=rw,D+x --perms --delete -crv build/ staticiforme:/srv/snowflake.torproject.org/htdocs/
sending incremental file list
rsync: failed to set permissions on "/srv/snowflake.torproject.org/htdocs/.": Operation not permitted (1)
rsync: failed to set permissions on "/srv/snowflake.torproject.org/htdocs/assets": Operation not permitted (1)
./
embed.js
_locales/es/messages.json
_locales/lt/messages.json
_locales/nl/messages.json
_locales/tr/messages.json
assets/
rsync: mkstemp "/srv/snowflake.torproject.org/htdocs/_locales/nl/.messages.json.c5cJkc" failed: Permission denied (13)

sent 14,890 bytes  received 888 bytes  4,508.00 bytes/sec
total size is 982,821  speedup is 62.29
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1196) [sender=3.1.2]

and when I ran the dry-run again I can see that the nl translations didn't update properly:

$ rsync -n --chown=:snowflake --chmod ug=rw,D+x --perms --delete -crv build/ staticiforme:/srv/snowflake.torproject.org/htdocs/
sending incremental file list
./
_locales/nl/messages.json
assets/

sent 3,220 bytes  received 52 bytes  934.86 bytes/sec
total size is 982,821  speedup is 300.37 (DRY RUN)

comment:15 Changed 6 months ago by cohosh

Resolution: fixed
Status: merge_readyclosed

comment:16 in reply to:  14 Changed 6 months ago by dcf

Replying to cohosh:

When updating the website I got the following errors:

I'm not sure how it got this way but there were some files in the dcf and cohosh groups.

dcf@staticiforme:/srv/snowflake.torproject.org/htdocs$ find . -not -group snowflake | xargs ls -ld
-rw-rw-r-- 1 dcf    dcf    3496 Oct 29 00:53 ./_locales/ca/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/da/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/de/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/en_US/messages.json
-rw-rw-r-- 1 cohosh cohosh 3520 Dec  3 19:17 ./_locales/es/messages.json
-rw-rw-r-- 1 dcf    dcf    3746 Oct 29 00:53 ./_locales/fr/messages.json
-rw-rw-r-- 1 dcf    dcf    3941 Nov 14 20:21 ./_locales/he/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/it/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/ka/messages.json
-rw-rw-r-- 1 cohosh cohosh 3345 Dec  3 19:17 ./_locales/lt/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/messages.json
-rw-rw-r-- 1 dcf    dcf    5006 Oct 29 00:53 ./_locales/mk/messages.json
drwxrwxr-x 2 dcf    dcf    4096 Oct 29 00:53 ./_locales/nl
-rw-rw-r-- 1 dcf    dcf    3305 Oct 29 00:53 ./_locales/nl/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/pt_BR/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Oct 29 00:53 ./_locales/pt_PT/messages.json
-rw-rw-r-- 1 dcf    dcf    3267 Nov 14 20:21 ./_locales/sv/messages.json
-rw-rw-r-- 1 cohosh cohosh 3583 Dec  3 19:17 ./_locales/tr/messages.json
-rw-rw-r-- 1 dcf    dcf    3252 Oct 29 00:53 ./_locales/zh_CN/messages.json

I've just fixed the ones owned by dcf. I think you need to do the ones owned by cohosh:

$ ssh staticiforme 'chgrp -R snowflake /srv/snowflake.torproject.org/htdocs'

comment:17 Changed 6 months ago by cohosh

Thanks! I just ran the same command and the update works now. After I uploaded these changes, I looked at the groups again:

$ find . -not -group snowflake | xargs ls -ld
-rw-rw-r-- 1 cohosh cohosh 3309 Dec  6 21:47 ./_locales/nl/messages.json

So it looks like whenever we upload new files their group gets assigned to us as well. I had to chgrp the new files after the update to fix this.

Note: See TracTickets for help on using tickets.