Opened 12 months ago

Last modified 8 weeks ago

#27008 assigned task

Remove ooniprobe and dependencies

Reported by: irl Owned by: hellais
Priority: High Milestone:
Component: Internal Services/Service - deb.tpo Version:
Severity: Normal Keywords:
Cc: hellais, weasel Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

It is not time efficient to maintain the ooniprobe package in a way that is Debian policy compliant. The packages on deb.tpo have fallen out of date and are of limited utility. As such, I proposed that we remove them, and I will do this if there are no objections. This would include the following source packages:

  • klein
  • ooniprobe
  • python-certifi
  • python-ipaddress
  • txtorcon

All binaries built from these packages would be removed.

I had originally included txtorcon in deb.tpo as a dependency for ooniprobe in older suites. I believe that users can get later versions easily enough from stable-backports on Debian.

Child Tickets

Change History (19)

comment:1 Changed 12 months ago by irl

Owner: changed from weasel to irl
Status: newaccepted

comment:2 Changed 12 months ago by hellais

Can we do this in such a way that users who are currently using the deb.torproject.org debian repo and upgrade the ooniprobe package will then start using the new version of ooniprobe (or are migrated onto another repository)?

comment:3 Changed 12 months ago by irl

Technically yes. I would prefer to just set up deb.ooni.io and get people installing the fpm packages from there. I think there are greater benefits in onboarding larger numbers of new users than trying to hack our way into updating existing users without their explicit buy-in.

comment:4 in reply to:  3 Changed 12 months ago by hellais

Replying to irl:

Technically yes. I would prefer to just set up deb.ooni.io and get people installing the fpm packages from there. I think there are greater benefits in onboarding larger numbers of new users than trying to hack our way into updating existing users without their explicit buy-in.

What happens to users that are currently using deb.torproject.org and try to upgrade ooniprobe?

If upgrading to another repository is not an option, how about we publish the fpm package also to deb.torproject.org?

What I am worried about is being getting stuck with a broken installation and there not being an easy upgrade path that leaves their system in a clean state.

Users have already done an "explicit buy-in" to installing and using ooniprobe if they have added the deb.tpo repository and apt-get installed ooniprobe.

comment:5 Changed 12 months ago by irl

If users were to add the new deb.ooni.io repository, these packages would have a higher version number. We would ensure that those packages can allow safe upgrade from the packages that were on deb.torproject.org.

I feel that automatically changing user's APT sources is a little too sneaky. Automatically adding new GPG keys to secure APT that effectively give another group root access to the machine could well be considered a breach of trust by some users.

https://wiki.debian.org/piuparts is designed to test package upgrades don't break user's machines.

comment:6 Changed 12 months ago by hellais

My preference for this would be that the ooniprobe debian package on deb.torproject.org is unbricked and we keep it available also on deb.torproject.org.

Once that happens we can also offer an install option with bintray.org (for circumvention reasons). We don't currently have any plans of running our own deb.ooni.io package repository.

comment:7 Changed 12 months ago by irl

The package that is on deb.tpo does work, it's just that it's not easily updated to the latest version. How about if I just upload a new revision of that version with a NEWS file to ask users to add the new bintray repository to their APT sources? This at least gives them a chance to upload and then later we can remove these packages, say after a month or two.

They would see the news file when they apt upgrade.

comment:8 Changed 12 months ago by hellais

The package that is on deb.tpo does work, it's just that it's not easily updated to the latest version

The current deb.tpo package does not work. Here is a list of outstanding problems with it:
https://github.com/thetorproject/ooni-probe/issues?q=is%3Aopen+is%3Aissue+label%3Adebian

How about if I just upload a new revision of that version with a NEWS file to ask users to add the new bintray repository to their APT sources? This at least gives them a chance to upload and then later we can remove these packages, say after a month or two.

I would prefer we first come up with a working package and do thorough testing of it, before we make any changes to the current package.

The package that is currently hosted on bintray is not tested at all and we currently don't have enough resources to support testing it and working on it.

comment:9 in reply to:  8 Changed 12 months ago by irl

Replying to hellais:

The current deb.tpo package does not work. Here is a list of outstanding problems with it:
https://github.com/thetorproject/ooni-probe/issues?q=is%3Aopen+is%3Aissue+label%3Adebian

Oh wow. I mean, it works the way that I am using it, which is running as a system process on Debian stable and testing systems. I guess others use it in lots of different ways.

I would prefer we first come up with a working package and do thorough testing of it, before we make any changes to the current package.

Ok.

The package that is currently hosted on bintray is not tested at all and we currently don't have enough resources to support testing it and working on it.

):

Would it help if we were to focus only on Debian or only on Ubuntu at first to get that package going. The fpm package is where I see people installing ooniprobe from on Debian/Ubuntu systems for at least the next few years. If there is no resource at all, perhaps we can do a call for volunteers?

comment:10 Changed 12 months ago by hellais

Would it help if we were to focus only on Debian or only on Ubuntu at first to get that package going. The fpm package is where I see people installing ooniprobe from on Debian/Ubuntu systems for at least the next few years. If there is no resource at all, perhaps we can do a call for volunteers?

I think at this point the biggest issue is that somebody is interested in having OONI Probe working on linux and takes on the task of figuring out how to make it work there.

I personally cannot commit to doing this, because:

  1. I am not a linux user
  1. I already have too many other things on my plate to have time to become a linux user

Maybe doing a call for volunteers is a reasonable way to go.

comment:11 Changed 12 months ago by irl

An ideal volunteer:

  • cares about OONI
  • is a Debian or Ubuntu developer
  • has experience with Python
  • has experience with nodejs

I guess also it would be good if this person can set up some CI infrastructure to perform automated testing of the packages. There are so many Ubuntu suites to support that it quickly becomes a lot of work.

comment:12 Changed 11 months ago by irl

As the packages that are currently on deb.tpo are broken, it is probably best if they are removed. This will avoid people installing packages from here and thinking they work, instead of looking for an alternative installation method that is better supported.

This wouldn't prohibit us from reintroducing ooniprobe packages here later if working packages become available, and wouldn't cause any effect for existing users of these packages.

comment:13 Changed 11 months ago by irl

Resolution: fixed
Status: acceptedclosed

This is now done.

Version numbers before removal were:

 sid|main|amd64: ooniprobe 2.2.0-allinone+1
 sid|main|i386: ooniprobe 2.2.0-allinone+1
 sid|main|armel: ooniprobe 2.1.0-2
 sid|main|armhf: ooniprobe 2.1.0-2
 sid|main|source: ooniprobe 2.2.0-allinone+1
 stretch|main|amd64: ooniprobe 2.2.0-allinone+1~tpo9+1
 stretch|main|i386: ooniprobe 2.2.0-allinone+1~tpo9+1
 stretch|main|armel: ooniprobe 2.1.0-2~tpo9+1
 stretch|main|armhf: ooniprobe 2.1.0-2~tpo9+1
 stretch|main|source: ooniprobe 2.2.0-allinone+1~tpo9+1
 jessie|main|amd64: ooniprobe 2.2.0-allinone+1~tpo8+2
 jessie|main|i386: ooniprobe 2.2.0-allinone+1~tpo8+2
 jessie|main|armel: ooniprobe 2.0.2-1~tpo8+1
 jessie|main|armhf: ooniprobe 2.0.2-1~tpo8+1
 jessie|main|source: ooniprobe 2.2.0-allinone+1~tpo8+2
 trusty|main|amd64: ooniprobe 2.0.2-1~bpo+ubuntu14.04+1
 trusty|main|i386: ooniprobe 2.0.2-1~bpo+ubuntu14.04+1
 trusty|main|armhf: ooniprobe 2.0.2-1~bpo+ubuntu14.04+1
 trusty|main|source: ooniprobe 2.0.2-1~bpo+ubuntu14.04+1
 xenial|main|amd64: ooniprobe 2.1.0-1~tpo+ubuntu16.04+1
 xenial|main|i386: ooniprobe 2.1.0-1~tpo+ubuntu16.04+1
 xenial|main|armhf: ooniprobe 2.1.0-1~tpo+ubuntu16.04+1
 xenial|main|source: ooniprobe 2.1.0-1~tpo+ubuntu16.04+1

We would need to have any future packages be greater than these version numbers to ensure smooth upgrades.

This only prevents new installations from deb.tpo, and doesn't affect existing users.

comment:14 Changed 11 months ago by hellais

Resolution: fixed
Severity: NormalCritical
Status: closedreopened

@irl I thought we had agreed to not make any changes to the packages (or removing them until) we had a clear path forward.

We currently advertise deb.torproject.org as the preferred installation method for OONI Probe on linux system to use the deb.torproject.org repository.

Can you please revert this change ASAP.

Before removing the packages we need to:

  1. Come up with a path forward for the new OONI package as discussed above
  2. Implement a new package for it
  3. Update the docs on the OONI website mentioning the new installation method
  4. Write instructions on how users can migrate to the new package

comment:15 Changed 11 months ago by hellais

Priority: MediumVery High

comment:16 Changed 11 months ago by hellais

Priority: Very HighMedium
Severity: CriticalNormal

Ian has restored access to it.

Let's for the time being not make any other changes to the OONI Probe debian packages, until there is a clear path forward.

Perhaps it's useful to set aside some time at the tor dev meeting in Mexico to touch base on this.

comment:17 Changed 2 months ago by irl

Priority: MediumHigh

These instructions are going to stop working as new releases of Debian/Ubuntu mean that the packages are not built for the current versions, even though we do not remove the packages they are going to end up falling off anyway.

comment:18 Changed 2 months ago by irl

Owner: changed from irl to hellais
Status: reopenedassigned

I am not in a position to move this forward, it needs to come from OONI.

comment:19 Changed 8 weeks ago by arma

Per the notes in https://lists.torproject.org/pipermail/tor-project/2019-May/002331.html the proposed next steps are:

"Plan: we limp along with ooniprobe in deb.tpo for now, because there aren't any better options for users, and then when ooniprobe 3.0 is released -- by Nov 2019 is the current schedule -- we remove ooniprobe from deb.tpo, point users to the new one and inform existing users how to upgrade."

Note: See TracTickets for help on using tickets.