Expose bridge pool assignments again

[irl]

In the past we archived bridge pool assignments in CollecTor, but we don't do that anymore. It would be nice to do it again. This could be combined with the implementation of #29448.

[phw]

Looks like this only requires #14082 to be reverted. I'll look into it once I have access to the BridgeDB machine and the bridgedb-admin.git repo.

[arma]

I think this would be a really useful step to take -- especially if we then list the distribution bucket in relay-search, so operators can learn how their bridge is being used (which will in turn help them understand why it has the load patterns that it has).

[phw]

Replying to arma:

I think this would be a really useful step to take -- especially if we then list the distribution bucket in relay-search, so operators can learn how their bridge is being used (which will in turn help them understand why it has the load patterns that it has).

I agree. I'm happy to flip the switch and have BridgeDB expose the assignments again. Is there anything else that's needed from the metrics side?

[phw]

Replying to phw:

I agree. I'm happy to flip the switch and have BridgeDB expose the assignments again. Is there anything else that's needed from the metrics side?

It looks like BridgeDB ​changed a bit since metrics last archived these files. What file format were you dealing with? And how did you get access to the assignments file?

What BridgeDB currently has is an assignments.log file that is structured as follows:

bridge-pool-assignment 2019-07-24 00:00:10
C042E360BB0139ABA8C115A61CAC4F5A65D86A9B moat ip=4 ring=2
7A95FB316A8D6FE64F24F1742DE066EEBF04A930 moat ip=4,6 ring=2 flag=stable transport=websocket,fte,obfs3,scramblesuit,obfs4
6CC4EC0313D2D823EC2AEF5F5179579F09CB48B0 moat ip=4 ring=2 transport=obfs4
...

[phw]

Glancing at ​archived bridge pool assignments, the format does not seem to have changed. #14082's description mentions that these files were once rsynced over to metrics. I'm happy to set up a cronjob on polyanthum to make that happen again.

[karsten]

We can include bridge pool assignments in CollecTor again. Yes, please set up an rsync to copy files to colchicifolium to a directory /srv/collector.torproject.org/collector/in/bridge-pool-assignments/polyanthum/. I'll take care of the CollecTor side of things then which includes reverting an earlier commit and refreshing that enough to run again. Thanks!

[phw]

Replying to karsten:

We can include bridge pool assignments in CollecTor again. Yes, please set up an rsync to copy files to colchicifolium to a directory /srv/collector.torproject.org/collector/in/bridge-pool-assignments/polyanthum/. I'll take care of the CollecTor side of things then which includes reverting an earlier commit and refreshing that enough to run again. Thanks!

I ​just updated our script sync-to-colchicifolium to make this happen. You should see assignments.log files in this directory in less than two hours. Please let me know if that's not the case, or if anything else went wrong.

[phw]

I just noticed an error message in the logs:

1566241638:descriptors:<36>Aug 19 19:07:18 collector-ssh-wrap[23644]: The SSH_ORIGINAL_COMMAND ('rsync --server -logDtpre.iLsfxC . /srv/collector.torproject.org/collector/in/bridge-pool-assignments/polyanthum/') is not on the whitelist

This seems like an issue on colchicifolium's side. Do we need an update in its SSH configuration?

[karsten]

I just created #31515 for the sysadmin side of this.

[karsten]

Alright, the rsync is working, now I need to look into resurrecting the CollecTor module. Created #31558 for that part.

[karsten]

Bridge pool assignments are back! Closing.