Opened 2 years ago

Last modified 2 years ago

#26848 closed task

Create Debian package for sbws — at Version 22

Reported by: juga Owned by: juga
Priority: Medium Milestone: sbws: 1.0.x-final
Component: Core Tor/sbws Version:
Severity: Normal Keywords: sbws-1.0-must-closed-moved-20181128
Cc: pastly, juga, teor, stefani@…, arma, mail@…, micah@…, adejoode@… Actual Points:
Parent ID: #25925 Points:
Reviewer: Sponsor:

Description (last modified by juga)

We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275.
This ticket follows the issue created in github [0]

Tasks to have a Debian package:

[0] https://github.com/pastly/simple-bw-scanner/issues/101
[1] https://wiki.debian.org/ITP

Edit:

  • link to ITP bug, mark as done
  • [ ] get ITP accepted
  • [ ] (optionall) provide sbws releases via dist.tpo (#XXX) so that watch does not use Github
  • [ ] get the source reviewed
  • [ ] get the package uploaded to Debian archive (sid)
  • [ ] create debian backport to current stable (stretch)
  • [ ] get backport reviewed
  • [ ] get backport uploaded to Debian archive
  • [ ] (optional) upload the package to deb.tpo
  • [ ] (optional) create bug in Ubuntu to get the package distributed in Ubuntu too
  • Add ticket number for having releases in dist.tpo
  • Add link to team mail
  • Number of ticket blocking this
  • Add link to the package

Child Tickets

TicketStatusOwnerSummaryComponent
#26849closedUpload sbws releases in dist.torproject.orgInternal Services/Service - dist
#26862closedjugasbws should allow a configuration file argumentCore Tor/sbws
#26926closedjugaBinaries should have manual pages (man)Core Tor/sbws

Change History (22)

comment:1 Changed 2 years ago by juga

Description: modified (diff)

comment:2 Changed 2 years ago by juga

Description: modified (diff)

comment:3 Changed 2 years ago by juga

Description: modified (diff)

comment:4 Changed 2 years ago by teor

Instead of modifying the description, you can open child tickets, and close them when they're done.
Then the "child tickets" table in this ticket will automatically show their statuses.

comment:5 Changed 2 years ago by irl

I'm *really* not convinced it is a good idea to do this.

Creating the structure of a Debian package and building a policy-compliant package is a great idea and would make deployment easier, as long the the dirauths are running Debian which at least dannenberg and maatuska are not.

My main objection to this though is that the package is not really useful outside of the directory authorities, not all of which run Debian and not all of which might even run sbws. It could be though that many people decide they want to measure the network themselves just for fun, and generate useless load.

(optionall) provide sbws releases via dist.tpo (#26849) so that watch does not use Github

You could just do this anyway, and have the dirauths fetch the code from here? This would work for all platforms then.

Assuming that you do go ahead with this anyway and there's something I'm missing:

get ITP accepted

I have no idea what this means. If you file an ITP bug it means you intend to package it. No one accepts or rejects these although discussion may happen on the bug.

create debian backport to current stable (stretch)

Note that it would take 10 days after the upload to unstable is accepted (sometimes more) to be able to upload a backport. If the dirauths are running stable, they would be delayed 10 days upgrading while this happens.

(optional) create bug in Ubuntu to get the package distributed in Ubuntu too

If it's in unstable you'll find it just appears in Ubuntu. It's not, in my experience, that quick to get updates to go in. This will be especially apparent with such a niche package.

comment:6 Changed 2 years ago by dkg

irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)

having a debian package can help for identifying problems, for system integration, and for ease of updates.

If there's a real concern about people measuring the network who shouldn't be, then i'm not sure that the presence of software in the debian repository or not is going to stop any even mildly interested actor. If you want to ensure that only "the right" people run sbws, you could have the dedicated debian system service do some sort of verification that it is on a host that is "acceptable", and then decline to run unless the administrator overrides it, but that seems like a lot of work to put in to an antifeature in free software.

as for the cadence of uploads to stable-backports -- packages with a passing [DEP-8 autopkgtest testsuite](https://dep-team.pages.debian.net/deps/dep8/) and no outstanding RC bugs can [migrate from unstable to testing in less time](https://lists.debian.org/debian-devel-announce/2018/05/msg00001.html), which allows for an upload to stretch-backports faster. It also means that we can rely on debian's testing infrastructure to verify basic package functionality on minimal systems. taking advantage of that continuous integration infrastructure seems like a good idea regardless of the package migration times.

what cadence do we expect bandwidth measurement upgrades to take anyway? do current measurement sites deploy them in hours instead of days today?

comment:7 in reply to:  5 ; Changed 2 years ago by juga

Replying to irl:

I'm *really* not convinced it is a good idea to do this.

i'm not totally convinced either, but asked around for some time and we decided to go for it.

Creating the structure of a Debian package and building a policy-compliant
package is a great idea and would make deployment easier, as long the the
dirauths are running Debian which at least dannenberg and maatuska are
not.

the majority are running Debian.

My main objection to this though is that the package is not really useful
outside of the directory authorities, not all of which run Debian and not
all of which might even run sbws. It could be though that many people
decide they want to measure the network themselves just for fun, and
generate useless load.

i share this concern seems months when i first thought to do this. But:

  1. in theory anyone could have been running torflow (just a bit harder

to install) for 6 years

  1. in theory anyone can run sbws now (not hard to install)
  2. some dirauths would refuse to install something from

unsafe/unverified sources

(optionall) provide sbws releases via dist.tpo (#26849) so that watch

does not use Github

You could just do this anyway, and have the dirauths fetch the code from
here? This would work for all platforms then.

agree, the ticket is created. Still upstream releases would miss
important system configuration/dependencies.

Assuming that you do go ahead with this anyway and there's something I'm
missing:

get ITP accepted

I have no idea what this means. If you file an ITP bug it means you intend
to package it. No one accepts or rejects these although discussion may
happen on the bug.

New packages need to close an ITP. AFAICT that discussion is important
to actually decided whether it makes sense to have the package in Debian.

create debian backport to current stable (stretch)

Note that it would take 10 days after the upload to unstable is accepted
(sometimes more) to be able to upload a backport. If the dirauths are
running stable, they would be delayed 10 days upgrading while this
happens.

security updates for backports happen faster?

(optional) create bug in Ubuntu to get the package distributed in Ubuntu

too

If it's in unstable you'll find it just appears in Ubuntu.

didn't know this.

It's not, in my

experience, that quick to get updates to go in. This will be especially
apparent with such a niche package.

Note there's also:

[ ] (optional) upload the package to deb.tpo

Alternatively we could upload it only there, but arma mentioned that only packages that are also in Debian archive go there, cause otherwise would end up unmaintained.

comment:8 in reply to:  7 ; Changed 2 years ago by irl

Replying to dkg:

irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)

I'm not sure yet.

having a debian package can help for identifying problems, for system integration, and for ease of updates.

Not all are Debian though. We should be identifying problems with good test suites.

If there's a real concern about people measuring the network who shouldn't be, then i'm not sure that the presence of software in the debian repository or not is going to stop any even mildly interested actor. If you want to ensure that only "the right" people run sbws, you could have the dedicated debian system service do some sort of verification that it is on a host that is "acceptable", and then decline to run unless the administrator overrides it, but that seems like a lot of work to put in to an antifeature in free software.

I'm thinking of people installing it by accident. Relay operators that are looking for nyx may see "Tor Bandwidth Scanner" and think that is what they're looking for.

as for the cadence of uploads to stable-backports -- packages with a passing [DEP-8 autopkgtest testsuite](https://dep-team.pages.debian.net/deps/dep8/) and no outstanding RC bugs can [migrate from unstable to testing in less time](https://lists.debian.org/debian-devel-announce/2018/05/msg00001.html), which allows for an upload to stretch-backports faster. It also means that we can rely on debian's testing infrastructure to verify basic package functionality on minimal systems. taking advantage of that continuous integration infrastructure seems like a good idea regardless of the package migration times.

Ah ok, that's a new thing I did not know about. (: I'll withdraw that objection.

Replying to juga:

Creating the structure of a Debian package and building a policy-compliant
package is a great idea and would make deployment easier, as long the the
dirauths are running Debian which at least dannenberg and maatuska are
not.

the majority are running Debian.

Do we plan to mandate that dirauths run Debian if they are to have bandwidth scanners?

i share this concern seems months when i first thought to do this. But:

  1. in theory anyone could have been running torflow (just a bit harder

to install) for 6 years

I'm not thinking so much about people running it maliciously, but accidentally and then forgetting about it. This could become a cumulative problem over time.

You could just do this anyway, and have the dirauths fetch the code from
here? This would work for all platforms then.

agree, the ticket is created. Still upstream releases would miss
important system configuration/dependencies.

I don't understand this.

New packages need to close an ITP. AFAICT that discussion is important
to actually decided whether it makes sense to have the package in Debian.

This only requires that the ITP is filed. This is done.

security updates for backports happen faster?

I'm not sure about that. See dkg's point above though.

Note there's also:

[ ] (optional) upload the package to deb.tpo

Alternatively we could upload it only there, but arma mentioned that only packages that are also in Debian archive go there, cause otherwise would end up unmaintained.

We can even just create a simple APT repository in any web server. As an example I do this here.

comment:9 in reply to:  8 Changed 2 years ago by juga

Replying to juga:

[...]

Do we plan to mandate that dirauths run Debian if they are to have
bandwidth scanners?

No
[...]

I'm not thinking so much about people running it maliciously, but
accidentally and then forgetting about it. This could become a cumulative
problem over time.

what about having big warnings in the package description and command help?
[...]

This only requires that the ITP is filed. This is done.

ok, removing that

[...]

We can even just create a simple APT repository in any web server. As an
example I do this here.

That would not solve the problem of ensuring it is maintained, nor having the Debian keyrings.

comment:10 Changed 2 years ago by juga

Description: modified (diff)

comment:11 Changed 2 years ago by atagar

irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)

I'm pretty convinced this *is* a bad idea but we've already discussed this multiple times on irc. If pastly would like to vend sbws through the debian repos then this is up to him.

comment:12 in reply to:  11 ; Changed 2 years ago by juga

Cc: juga stefani@… arma mail@… micah@… adejoode@… added; juga@… removed

Replying to atagar:

irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)

I'm pretty convinced this *is* a bad idea but we've already discussed this multiple times on irc. If pastly would like to vend sbws through the debian repos then this is up to him.

pastly wrote to the dir-auth ml, which is our user target and i thought/think it was a good idea. Unfortunately, IRC discussions missed the context of the private discussions with had with dirauths. They would not install something from pypi nor git cause of known security issues, plus i add, it requires further system configuration.

i'm adding dirauths that expressed an opinion as CC here. Please dirauths, let me know if i should remove you.

We need dirauths to start running sbws. Afaik, the way to install sbws is blocking them right now, and we don't seem to reach consensus out of dir-auth ml, so, all i can do right now is try to get it at list on deb.tpo, and then see...?

comment:13 in reply to:  12 ; Changed 2 years ago by teor

Replying to juga:

Replying to atagar:

irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)

I'm pretty convinced this *is* a bad idea but we've already discussed this multiple times on irc. If pastly would like to vend sbws through the debian repos then this is up to him.

irl, thank you for your feedback.

We need to make a Debian package to get sbws deployed on some directory authorities. Deployment is a top priority. If it doesn't happen, the project is blocked. We will consider the issues you've raised, but as far as I can tell, none of them are blockers.

pastly wrote to the dir-auth ml, which is our user target and i thought/think it was a good idea. Unfortunately, IRC discussions missed the context of the private discussions with had with dirauths. They would not install something from pypi nor git cause of known security issues, plus i add, it requires further system configuration.

i'm adding dirauths that expressed an opinion as CC here. Please dirauths, let me know if i should remove you.

We need dirauths to start running sbws. Afaik, the way to install sbws is blocking them right now, and we don't seem to reach consensus out of dir-auth ml, so, all i can do right now is try to get it at list on deb.tpo, and then see...?

Juga, sometimes we need to make decisions, even if we can't reach consensus.

Here are my suggestions for moving forward:

  • keep going with the Debian package process
  • prioritise issues raised by people who can block the debian package from being created
  • prioritise issues raised by directory authority operators

comment:14 in reply to:  13 ; Changed 2 years ago by irl

Replying to teor:

We need to make a Debian package to get sbws deployed on some directory authorities. Deployment is a top priority. If it doesn't happen, the project is blocked. We will consider the issues you've raised, but as far as I can tell, none of them are blockers.

Is it really a requirement that the package be in the official Debian archives? If so, then OK. Having checked Debian Policy I see that I must have imagined the requirement that a package would be generally useful (as opposed to only useful in very specific cases, such as the dirauths).

I would agree though with Philipp Kern, that the source and binary packages should be in the tor- namespace to make the purpose of the package clearer to users.

comment:15 in reply to:  14 Changed 2 years ago by juga

Replying to irl:

I would agree though with Philipp Kern, that the source and binary packages should be in the tor- namespace to make the purpose of the package clearer to users.

For reference: https://bugs.debian.org/cgi-bin/bugreport.cgi?att=0;bug=903977;msg=44

So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?.
I think that just changing the name for the Debian package would be enough. I don't think though that would prevent non-dirauths from running it.
What needs to happen to get that approved by Tor Project?, i guess i should open a ticket if we go for it?.

comment:16 in reply to:  14 ; Changed 2 years ago by pastly

Replying to irl:

Replying to teor:

We need to make a Debian package to get sbws deployed on some directory authorities. [snip]

Is it really a requirement that the package be in the official Debian archives? [snip]

I've been told many times that in order to get a deb onto deb.torproject.org that the package must at least exist in Debian's repos. I don't want to do more than required in order to get sbws on deb.tpo. Some dirauths are willing to install sbws from deb.tpo, and AIUI we can have a more rapid release cycle (if required) when primarily pushing to deb.tpo.

Replying to juga:

So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?

It's fine with me if the package is called tor-sbws instead of sbws.

comment:17 in reply to:  16 ; Changed 2 years ago by irl

Replying to pastly:

Replying to irl:

Replying to teor:

We need to make a Debian package to get sbws deployed on some directory authorities. [snip]

Is it really a requirement that the package be in the official Debian archives? [snip]

I've been told many times that in order to get a deb onto deb.torproject.org that the package must at least exist in Debian's repos. I don't want to do more than required in order to get sbws on deb.tpo. Some dirauths are willing to install sbws from deb.tpo, and AIUI we can have a more rapid release cycle (if required) when primarily pushing to deb.tpo.

This is not a real requirement. This is an artificial restriction to ensure that deb.tpo packages are well maintained. In the case of sbws I think there is a strong argument for an exception.

Replying to juga:

So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?

It's fine with me if the package is called tor-sbws instead of sbws.

The upstream name doesn't need to be changed to change the name of the Debian package. There are plenty of examples in the python section where pysomething is packaged as python-something to have consistent namespace.

comment:18 in reply to:  17 ; Changed 2 years ago by teor

Replying to irl:

Replying to pastly:

Replying to irl:

Replying to teor:

We need to make a Debian package to get sbws deployed on some directory authorities. [snip]

Is it really a requirement that the package be in the official Debian archives? [snip]

I've been told many times that in order to get a deb onto deb.torproject.org that the package must at least exist in Debian's repos. I don't want to do more than required in order to get sbws on deb.tpo. Some dirauths are willing to install sbws from deb.tpo, and AIUI we can have a more rapid release cycle (if required) when primarily pushing to deb.tpo.

This is not a real requirement. This is an artificial restriction to ensure that deb.tpo packages are well maintained. In the case of sbws I think there is a strong argument for an exception.

Is there a wiki documenting this requirement?
Who can decide to make an exception?

Replying to juga:

So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?

It's fine with me if the package is called tor-sbws instead of sbws.

The upstream name doesn't need to be changed to change the name of the Debian package. There are plenty of examples in the python section where pysomething is packaged as python-something to have consistent namespace.

Please email tor-project@… about the proposed use of the trademark, and CC frontdesk@….

comment:19 in reply to:  18 Changed 2 years ago by irl

Replying to teor:

This is not a real requirement. This is an artificial restriction to ensure that deb.tpo packages are well maintained. In the case of sbws I think there is a strong argument for an exception.

Is there a wiki documenting this requirement?

I do not know of any such wiki page. There probably should be one.

Who can decide to make an exception?

irl@perdulce:~$ getent group tordeb
tordeb:x:1540:irl,lunar,weasel

Replying to juga:

So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?

It's fine with me if the package is called tor-sbws instead of sbws.

The upstream name doesn't need to be changed to change the name of the Debian package. There are plenty of examples in the python section where pysomething is packaged as python-something to have consistent namespace.

Please email tor-project@… about the proposed use of the trademark, and CC frontdesk@….

I would still suggest it's a good idea to rename the package even if it's only hosted on deb.tpo.

When I followed up on the ITP with an extended description for the binary package, I realised that this is not only useful to directory authorities but may also be useful to those running test networks for research. I think on that basis this package would be suitable for inclusion in Debian as it's not limited to only the dirauths in its usefulness.

comment:20 in reply to:  18 Changed 2 years ago by juga

Replying to teor:
[...]

Please email tor-project@… about the proposed use of the trademark, and CC frontdesk@….

Done: https://lists.torproject.org/pipermail/tor-project/2018-July/001912.html

comment:21 Changed 2 years ago by juga

Description: modified (diff)

comment:22 Changed 2 years ago by juga

Description: modified (diff)
Note: See TracTickets for help on using tickets.