We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
Trac: Description: We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
create Debian ITP [1]
try to get a team to collectively maintain the package so that it does not depend on a single person
get steam to release new version, currently blocked by #26412 (closed)
We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
Trac: Description: We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
(optionall) provide sbws releases via dist.tpo (#XXX) so that watch does not use Github
get the source reviewed
get the package uploaded to Debian archive (sid)
create debian backport to current stable (stretch)
get backport reviewed
get backport uploaded to Debian archive
(optional) upload the package to deb.tpo
(optional) create bug in Ubuntu to get the package distributed in Ubuntu too
to
We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
Trac: Description: We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
(optionall) provide sbws releases via dist.tpo (#XXX) so that watch does not use Github
get the source reviewed
get the package uploaded to Debian archive (sid)
create debian backport to current stable (stretch)
get backport reviewed
get backport uploaded to Debian archive
(optional) upload the package to deb.tpo
(optional) create bug in Ubuntu to get the package distributed in Ubuntu too
Add ticket number for having releases in dist.tpo
to
We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
Instead of modifying the description, you can open child tickets, and close them when they're done.
Then the "child tickets" table in this ticket will automatically show their statuses.
I'm really not convinced it is a good idea to do this.
Creating the structure of a Debian package and building a policy-compliant package is a great idea and would make deployment easier, as long the the dirauths are running Debian which at least dannenberg and maatuska are not.
My main objection to this though is that the package is not really useful outside of the directory authorities, not all of which run Debian and not all of which might even run sbws. It could be though that many people decide they want to measure the network themselves just for fun, and generate useless load.
(optionall) provide sbws releases via dist.tpo (#26849 (moved)) so that watch does not use Github
You could just do this anyway, and have the dirauths fetch the code from here? This would work for all platforms then.
Assuming that you do go ahead with this anyway and there's something I'm missing:
get ITP accepted
I have no idea what this means. If you file an ITP bug it means you intend to package it. No one accepts or rejects these although discussion may happen on the bug.
create debian backport to current stable (stretch)
Note that it would take 10 days after the upload to unstable is accepted (sometimes more) to be able to upload a backport. If the dirauths are running stable, they would be delayed 10 days upgrading while this happens.
(optional) create bug in Ubuntu to get the package distributed in Ubuntu too
If it's in unstable you'll find it just appears in Ubuntu. It's not, in my experience, that quick to get updates to go in. This will be especially apparent with such a niche package.
irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)
having a debian package can help for identifying problems, for system integration, and for ease of updates.
If there's a real concern about people measuring the network who shouldn't be, then i'm not sure that the presence of software in the debian repository or not is going to stop any even mildly interested actor. If you want to ensure that only "the right" people run sbws, you could have the dedicated debian system service do some sort of verification that it is on a host that is "acceptable", and then decline to run unless the administrator overrides it, but that seems like a lot of work to put in to an antifeature in free software.
as for the cadence of uploads to stable-backports -- packages with a passing DEP-8 autopkgtest testsuite and no outstanding RC bugs can migrate from unstable to testing in less time, which allows for an upload to stretch-backports faster. It also means that we can rely on debian's testing infrastructure to verify basic package functionality on minimal systems. taking advantage of that continuous integration infrastructure seems like a good idea regardless of the package migration times.
what cadence do we expect bandwidth measurement upgrades to take anyway? do current measurement sites deploy them in hours instead of days today?
I'm really not convinced it is a good idea to do this.
i'm not totally convinced either, but asked around for some time and we decided to go for it.
Creating the structure of a Debian package and building a policy-compliant
package is a great idea and would make deployment easier, as long the the
dirauths are running Debian which at least dannenberg and maatuska are
not.
the majority are running Debian.
My main objection to this though is that the package is not really useful
outside of the directory authorities, not all of which run Debian and not
all of which might even run sbws. It could be though that many people
decide they want to measure the network themselves just for fun, and
generate useless load.
i share this concern seems months when i first thought to do this. But:
in theory anyone could have been running torflow (just a bit harder
to install) for 6 years
in theory anyone can run sbws now (not hard to install)
some dirauths would refuse to install something from
unsafe/unverified sources
(optionall) provide sbws releases via dist.tpo (#26849 (moved)) so that watch
does not use Github
You could just do this anyway, and have the dirauths fetch the code from
here? This would work for all platforms then.
agree, the ticket is created. Still upstream releases would miss
important system configuration/dependencies.
Assuming that you do go ahead with this anyway and there's something I'm
missing:
get ITP accepted
I have no idea what this means. If you file an ITP bug it means you intend
to package it. No one accepts or rejects these although discussion may
happen on the bug.
New packages need to close an ITP. AFAICT that discussion is important
to actually decided whether it makes sense to have the package in Debian.
create debian backport to current stable (stretch)
Note that it would take 10 days after the upload to unstable is accepted
(sometimes more) to be able to upload a backport. If the dirauths are
running stable, they would be delayed 10 days upgrading while this
happens.
security updates for backports happen faster?
(optional) create bug in Ubuntu to get the package distributed in Ubuntu
too
If it's in unstable you'll find it just appears in Ubuntu.
didn't know this.
It's not, in my
experience, that quick to get updates to go in. This will be especially
apparent with such a niche package.
Note there's also:
[ ] (optional) upload the package to deb.tpo
Alternatively we could upload it only there, but arma mentioned that only packages that are also in Debian archive go there, cause otherwise would end up unmaintained.
irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)
I'm not sure yet.
having a debian package can help for identifying problems, for system integration, and for ease of updates.
Not all are Debian though. We should be identifying problems with good test suites.
If there's a real concern about people measuring the network who shouldn't be, then i'm not sure that the presence of software in the debian repository or not is going to stop any even mildly interested actor. If you want to ensure that only "the right" people run sbws, you could have the dedicated debian system service do some sort of verification that it is on a host that is "acceptable", and then decline to run unless the administrator overrides it, but that seems like a lot of work to put in to an antifeature in free software.
I'm thinking of people installing it by accident. Relay operators that are looking for nyx may see "Tor Bandwidth Scanner" and think that is what they're looking for.
as for the cadence of uploads to stable-backports -- packages with a passing DEP-8 autopkgtest testsuite and no outstanding RC bugs can migrate from unstable to testing in less time, which allows for an upload to stretch-backports faster. It also means that we can rely on debian's testing infrastructure to verify basic package functionality on minimal systems. taking advantage of that continuous integration infrastructure seems like a good idea regardless of the package migration times.
Ah ok, that's a new thing I did not know about. (: I'll withdraw that objection.
Creating the structure of a Debian package and building a policy-compliant
package is a great idea and would make deployment easier, as long the the
dirauths are running Debian which at least dannenberg and maatuska are
not.
the majority are running Debian.
Do we plan to mandate that dirauths run Debian if they are to have bandwidth scanners?
i share this concern seems months when i first thought to do this. But:
in theory anyone could have been running torflow (just a bit harder
to install) for 6 years
I'm not thinking so much about people running it maliciously, but accidentally and then forgetting about it. This could become a cumulative problem over time.
You could just do this anyway, and have the dirauths fetch the code from
here? This would work for all platforms then.
agree, the ticket is created. Still upstream releases would miss
important system configuration/dependencies.
I don't understand this.
New packages need to close an ITP. AFAICT that discussion is important
to actually decided whether it makes sense to have the package in Debian.
This only requires that the ITP is filed. This is done.
security updates for backports happen faster?
I'm not sure about that. See dkg's point above though.
Note there's also:
[ ] (optional) upload the package to deb.tpo
Alternatively we could upload it only there, but arma mentioned that only packages that are also in Debian archive go there, cause otherwise would end up unmaintained.
Do we plan to mandate that dirauths run Debian if they are to have
bandwidth scanners?
No
[...]
I'm not thinking so much about people running it maliciously, but
accidentally and then forgetting about it. This could become a cumulative
problem over time.
what about having big warnings in the package description and command help?
[...]
This only requires that the ITP is filed. This is done.
ok, removing that
Trac: Description: We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
(optionall) provide sbws releases via dist.tpo (#XXX) so that watch does not use Github
get the source reviewed
get the package uploaded to Debian archive (sid)
create debian backport to current stable (stretch)
get backport reviewed
get backport uploaded to Debian archive
(optional) upload the package to deb.tpo
(optional) create bug in Ubuntu to get the package distributed in Ubuntu too
Add ticket number for having releases in dist.tpo
Add link to team mail
to
We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)
I'm pretty convinced this is a bad idea but we've already discussed this multiple times on irc. If pastly would like to vend sbws through the debian repos then this is up to him.
irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)
I'm pretty convinced this is a bad idea but we've already discussed this multiple times on irc. If pastly would like to vend sbws through the debian repos then this is up to him.
pastly wrote to the dir-auth ml, which is our user target and i thought/think it was a good idea. Unfortunately, IRC discussions missed the context of the private discussions with had with dirauths. They would not install something from pypi nor git cause of known security issues, plus i add, it requires further system configuration.
i'm adding dirauths that expressed an opinion as CC here. Please dirauths, let me know if i should remove you.
We need dirauths to start running sbws. Afaik, the way to install sbws is blocking them right now, and we don't seem to reach consensus out of dir-auth ml, so, all i can do right now is try to get it at list on deb.tpo, and then see...?
irl, i can't tell if you're "really not convinced it's a good idea" or if you're "really convinced it's not a good idea". :)
I'm pretty convinced this is a bad idea but we've already discussed this multiple times on irc. If pastly would like to vend sbws through the debian repos then this is up to him.
irl, thank you for your feedback.
We need to make a Debian package to get sbws deployed on some directory authorities. Deployment is a top priority. If it doesn't happen, the project is blocked. We will consider the issues you've raised, but as far as I can tell, none of them are blockers.
pastly wrote to the dir-auth ml, which is our user target and i thought/think it was a good idea. Unfortunately, IRC discussions missed the context of the private discussions with had with dirauths. They would not install something from pypi nor git cause of known security issues, plus i add, it requires further system configuration.
i'm adding dirauths that expressed an opinion as CC here. Please dirauths, let me know if i should remove you.
We need dirauths to start running sbws. Afaik, the way to install sbws is blocking them right now, and we don't seem to reach consensus out of dir-auth ml, so, all i can do right now is try to get it at list on deb.tpo, and then see...?
Juga, sometimes we need to make decisions, even if we can't reach consensus.
Here are my suggestions for moving forward:
keep going with the Debian package process
prioritise issues raised by people who can block the debian package from being created
prioritise issues raised by directory authority operators
We need to make a Debian package to get sbws deployed on some directory authorities. Deployment is a top priority. If it doesn't happen, the project is blocked. We will consider the issues you've raised, but as far as I can tell, none of them are blockers.
Is it really a requirement that the package be in the official Debian archives? If so, then OK. Having checked Debian Policy I see that I must have imagined the requirement that a package would be generally useful (as opposed to only useful in very specific cases, such as the dirauths).
I would agree though with Philipp Kern, that the source and binary packages should be in the tor- namespace to make the purpose of the package clearer to users.
So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?.
I think that just changing the name for the Debian package would be enough. I don't think though that would prevent non-dirauths from running it.
What needs to happen to get that approved by Tor Project?, i guess i should open a ticket if we go for it?.
We need to make a Debian package to get sbws deployed on some directory authorities. [snip]
Is it really a requirement that the package be in the official Debian archives? [snip]
I've been told many times that in order to get a deb onto deb.torproject.org that the package must at least exist in Debian's repos. I don't want to do more than required in order to get sbws on deb.tpo. Some dirauths are willing to install sbws from deb.tpo, and AIUI we can have a more rapid release cycle (if required) when primarily pushing to deb.tpo.
We need to make a Debian package to get sbws deployed on some directory authorities. [snip]
Is it really a requirement that the package be in the official Debian archives? [snip]
I've been told many times that in order to get a deb onto deb.torproject.org that the package must at least exist in Debian's repos. I don't want to do more than required in order to get sbws on deb.tpo. Some dirauths are willing to install sbws from deb.tpo, and AIUI we can have a more rapid release cycle (if required) when primarily pushing to deb.tpo.
This is not a real requirement. This is an artificial restriction to ensure that deb.tpo packages are well maintained. In the case of sbws I think there is a strong argument for an exception.
So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?
It's fine with me if the package is called tor-sbws instead of sbws.
The upstream name doesn't need to be changed to change the name of the Debian package. There are plenty of examples in the python section where pysomething is packaged as python-something to have consistent namespace.
We need to make a Debian package to get sbws deployed on some directory authorities. [snip]
Is it really a requirement that the package be in the official Debian archives? [snip]
I've been told many times that in order to get a deb onto deb.torproject.org that the package must at least exist in Debian's repos. I don't want to do more than required in order to get sbws on deb.tpo. Some dirauths are willing to install sbws from deb.tpo, and AIUI we can have a more rapid release cycle (if required) when primarily pushing to deb.tpo.
This is not a real requirement. This is an artificial restriction to ensure that deb.tpo packages are well maintained. In the case of sbws I think there is a strong argument for an exception.
Is there a wiki documenting this requirement?
Who can decide to make an exception?
So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?
It's fine with me if the package is called tor-sbws instead of sbws.
The upstream name doesn't need to be changed to change the name of the Debian package. There are plenty of examples in the python section where pysomething is packaged as python-something to have consistent namespace.
This is not a real requirement. This is an artificial restriction to ensure that deb.tpo packages are well maintained. In the case of sbws I think there is a strong argument for an exception.
Is there a wiki documenting this requirement?
I do not know of any such wiki page. There probably should be one.
Who can decide to make an exception?
irl@perdulce:~$ getent group tordebtordeb:x:1540:irl,lunar,weasel
So you suggest to call sbws "tor-sbws" both for the upstream and the Debian package?. Pastly, would you agree with that name change?
It's fine with me if the package is called tor-sbws instead of sbws.
The upstream name doesn't need to be changed to change the name of the Debian package. There are plenty of examples in the python section where pysomething is packaged as python-something to have consistent namespace.
I would still suggest it's a good idea to rename the package even if it's only hosted on deb.tpo.
When I followed up on the ITP with an extended description for the binary package, I realised that this is not only useful to directory authorities but may also be useful to those running test networks for research. I think on that basis this package would be suitable for inclusion in Debian as it's not limited to only the dirauths in its usefulness.
Trac: Description: We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
(optionall) provide sbws releases via dist.tpo (#XXX) so that watch does not use Github
get the source reviewed
get the package uploaded to Debian archive (sid)
create debian backport to current stable (stretch)
get backport reviewed
get backport uploaded to Debian archive
(optional) upload the package to deb.tpo
(optional) create bug in Ubuntu to get the package distributed in Ubuntu too
Add ticket number for having releases in dist.tpo
Add link to team mail
to
We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
Trac: Description: We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
(optionall) provide sbws releases via dist.tpo (#XXX) so that watch does not use Github
get the source reviewed
get the package uploaded to Debian archive (sid)
create debian backport to current stable (stretch)
get backport reviewed
get backport uploaded to Debian archive
(optional) upload the package to deb.tpo
(optional) create bug in Ubuntu to get the package distributed in Ubuntu too
Add ticket number for having releases in dist.tpo
Add link to team mail
Number of ticket blocking this
to
We agreed that to easy bandwidth authorities to run sbws we should have a Debian package.
This is also part of #17275 (moved).
This ticket follows the issue created in github [0]
Tasks to have a Debian package:
create a proper Python package (solved by GHXX, GHXX, GHXX)
create metadata for the Python package, useful for source and binary distributions (solved by GHXX)
When I followed up on the ITP with an extended description for the binary package, I realised that this is not only useful to directory authorities but may also be useful to those running test networks for research. I think on that basis this package would be suitable for inclusion in Debian as it's not limited to only the dirauths in its usefulness.
If sbws is put into Debian, its likely that directory authorities are going to end up installing a newer version of the package from the deb.tpo package repositories. This is what we do now for core tor because the versions available in stable repositories are often too out of date for what the network requires. The ability to have a more rapid release cycle has been cited as a reason to have things on deb.tpo.
If the above are all true, then putting sbws into Debian itself will only target those people who may want to use sbws for research. I'm uncertain if a version of sbws that is in the stable repository for several years will be useful for researchers, especially if there is an idea that there might be a 'rapid release cycle'. I'd think that researchers would be interested in the latest version of the code, and not a version that is 'stable' (read: several years old). They can get that via the git repository, or via deb.tpo.
All of the above makes me think that the best way forward for sbws, at least for the near term, is to get it packaged first for deb.tpo, and then evaluate after that is done and has been there for a while if it makes sense to put it into Debian itself. Because it will already be a package, the hardest part will already be done.
Also, if there are library dependencies that aren't in Debian, those should be updated/packaged and put into Debian proper!
Regarding the name, I think the idea of putting a "tor-" prefix on the package name doesn't make too much sense to me, I have not been convinced that this is necessary, or even desirable. Especially when the short/long description can be made to make it obvious what this is for.
After the comments about tor- namespace by micah here, ulrike in the ITP, weasel in #tor-project channel, and irl in the ITP and here:
we are not going to use tor- for the Debian package
the Debian package description has been updated to clarify the scanner is for Tor and intented to be run be run only by dirauths or researchers in test networks.
Also, a new policy page has been created regarding deb.tpo [0], and it's required to have a package in Debian archive to have it also in deb.tpo.
irl is already reviewing the package, and would help to get it into Debian archive.
Thanks all of you for your comments and suggestions.
The package is in unstable, version 1.0 has been uploaded (https://buildd.debian.org/status/package.php?p=sbws).
Currently is not possible to backport it to stable, even if python3-stem has been backported, because python3-stem depends on python3-distutils, which is not backported.
Trac: Resolution: N/Ato implemented Status: assigned to closed