Opened 5 months ago

Last modified 2 days ago

#31130 needs_information defect

Use Debian 10 for our Android container images

Reported by: gk Owned by: sisbell
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-rbm, TorBrowserTeam201912
Cc: boklm, sisbell, tbb-team Actual Points:
Parent ID: #31127 Points: 0.5
Reviewer: gk Sponsor:

Description

We should switch to Debian 10 for our Android builds.

Child Tickets

Change History (57)

comment:1 Changed 2 months ago by gk

Cc: sisbell added

#31924 is a duplicate.

comment:2 Changed 2 months ago by boklm

If we need to install the openjdk-8 package from testing, then we should have some pinning configuration (see ticket:31564#comment:60) to only install this package from testing. We should also be using https://snapshot.debian.org/ for the testing repository to avoid issues when packages are removed or updated.

And as it seems we'll need this pinning configuration for building most components, we should probably do it in var/pre_pkginst in rbm.conf.

comment:3 Changed 7 weeks ago by sysrqb

Points: 0.5

comment:4 Changed 6 weeks ago by sisbell

Status: newneeds_review

I kept in stretch for desktop builds, while adding in buster for android.

https://github.com/sisbell/tor-browser-build/commit/8a39f6de0ea833f63ae00e5d3a6f2e2503a6572a

The above commit works fine for openjdk-8. When I attempted to add pinning I would get some dependency errors and I didn't see any advantage to making this more complicated.

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 openjdk-8-jdk : Depends: openjdk-8-jre (= 8u232-b09-1) but it is not going to be installed
                 Depends: openjdk-8-jdk-headless (= 8u232-b09-1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Also trying to use snapshot with pinning resulted in

Starting build: Mon Oct 21 23:03:49 2019
Hit:1 http://deb.debian.org/debian buster InRelease
Get:2 http://deb.debian.org/debian buster/main Translation-en [5967 kB]
Ign:3 https://snapshot.debian.org sid InRelease
Err:4 https://snapshot.debian.org sid Release
  Certificate verification failed: The certificate is NOT trusted. The certificate issuer is unknown.  Could not handshake: Error in the certificate verification. [IP: 185.17.185.185 443]
Reading package lists...
W: https://snapshot.debian.org/dists/sid/InRelease: No system certificates available. Try installing ca-certificates.
W: https://snapshot.debian.org/dists/sid/Release: No system certificates available. Try installing ca-certificates.
E: The repository 'https://snapshot.debian.org sid Release' does not have a Release file.

comment:5 in reply to:  4 ; Changed 6 weeks ago by boklm

Status: needs_reviewneeds_revision

Replying to sisbell:

I kept in stretch for desktop builds, while adding in buster for android.

https://github.com/sisbell/tor-browser-build/commit/8a39f6de0ea833f63ae00e5d3a6f2e2503a6572a

I don't think we need to have two different Ubuntu versions to generate the images. We could just use Ubuntu 19.10 to generate all images.

The above commit works fine for openjdk-8. When I attempted to add pinning I would get some dependency errors and I didn't see any advantage to making this more complicated.

No pinning means that we are not using Debian buster, but Debian testing, which is not what we want (and not a good idea for reproducible builds as this distribution is changing a lot all the time). From the error it looks like openjdk-8-jre and openjdk-8-jdk-headless need to be added to the pinning configuration too.

I also think we should not do this configuration in projects/debootstrap-image/config, but only in the containers that we use for android builds, for example by doing it in var/pre_pkginst in rbm.conf.

Also trying to use snapshot with pinning resulted in

Starting build: Mon Oct 21 23:03:49 2019
Hit:1 http://deb.debian.org/debian buster InRelease
Get:2 http://deb.debian.org/debian buster/main Translation-en [5967 kB]
Ign:3 https://snapshot.debian.org sid InRelease
Err:4 https://snapshot.debian.org sid Release
  Certificate verification failed: The certificate is NOT trusted. The certificate issuer is unknown.  Could not handshake: Error in the certificate verification. [IP: 185.17.185.185 443]
Reading package lists...
W: https://snapshot.debian.org/dists/sid/InRelease: No system certificates available. Try installing ca-certificates.
W: https://snapshot.debian.org/dists/sid/Release: No system certificates available. Try installing ca-certificates.
E: The repository 'https://snapshot.debian.org sid Release' does not have a Release file.

Did you try installing ca-certificates as suggested in the error message?

comment:6 in reply to:  5 ; Changed 6 weeks ago by sisbell

Replying to boklm:

Replying to sisbell:

I kept in stretch for desktop builds, while adding in buster for android.

https://github.com/sisbell/tor-browser-build/commit/8a39f6de0ea833f63ae00e5d3a6f2e2503a6572a

I don't think we need to have two different Ubuntu versions to generate the images. We could just use Ubuntu 19.10 to generate all images.

Ok I can change this.

The above commit works fine for openjdk-8. When I attempted to add pinning I would get some dependency errors and I didn't see any advantage to making this more complicated.

No pinning means that we are not using Debian buster, but Debian testing, which is not what we want (and not a good idea for reproducible builds as this distribution is changing a lot all the time). From the error it looks like openjdk-8-jre and openjdk-8-jdk-headless need to be added to the pinning configuration too.

I had tried pinning the openjdk-8-jre and openjdk-8-headless but got the same result, they were missing.

I also think we should not do this configuration in projects/debootstrap-image/config, but only in the containers that we use for android builds, for example by doing it in var/pre_pkginst in rbm.conf.

I can move the apt sources change to rbm.conf. Assuming I don't modify the pre script in the debootstrap config for the android specific build, how is the # Bug 29158 going to be handled? Is this related to stretch or to the ubuntu version? Can it be removed now?

Also trying to use snapshot with pinning resulted in

Starting build: Mon Oct 21 23:03:49 2019
Hit:1 http://deb.debian.org/debian buster InRelease
Get:2 http://deb.debian.org/debian buster/main Translation-en [5967 kB]
Ign:3 https://snapshot.debian.org sid InRelease
Err:4 https://snapshot.debian.org sid Release
  Certificate verification failed: The certificate is NOT trusted. The certificate issuer is unknown.  Could not handshake: Error in the certificate verification. [IP: 185.17.185.185 443]
Reading package lists...
W: https://snapshot.debian.org/dists/sid/InRelease: No system certificates available. Try installing ca-certificates.
W: https://snapshot.debian.org/dists/sid/Release: No system certificates available. Try installing ca-certificates.
E: The repository 'https://snapshot.debian.org sid Release' does not have a Release file.

Did you try installing ca-certificates as suggested in the error message?

comment:7 in reply to:  5 ; Changed 6 weeks ago by sisbell

Replying to boklm:

Replying to sisbell:

I kept in stretch for desktop builds, while adding in buster for android.

https://github.com/sisbell/tor-browser-build/commit/8a39f6de0ea833f63ae00e5d3a6f2e2503a6572a

I don't think we need to have two different Ubuntu versions to generate the images. We could just use Ubuntu 19.10 to generate all images.

The above commit works fine for openjdk-8. When I attempted to add pinning I would get some dependency errors and I didn't see any advantage to making this more complicated.

No pinning means that we are not using Debian buster, but Debian testing, which is not what we want (and not a good idea for reproducible builds as this distribution is changing a lot all the time). From the error it looks like openjdk-8-jre and openjdk-8-jdk-headless need to be added to the pinning configuration too.

I do see that without pinning we still use debian stable buster for a bunch of libraries (the first 54). I haven't been able to find a specification of how it does its resolving in this case (so I guess pinning is designed to force certain rules).

Get:50 http://deb.debian.org/debian buster/main amd64 libxdmcp-dev amd64 1:1.1.2-3 [42.2 kB]
Get:51 http://deb.debian.org/debian buster/main amd64 xtrans-dev all 1.3.5-1 [100 kB]
Get:52 http://deb.debian.org/debian buster/main amd64 libxcb1-dev amd64 1.13.1-2 [174 kB]
Get:53 http://deb.debian.org/debian buster/main amd64 libxt-dev amd64 1:1.1.5-1+b3 [426 kB]
Get:54 http://deb.debian.org/debian buster/main amd64 xdg-user-dirs amd64 0.17-2 [53.8 kB]
Get:55 http://ftp.us.debian.org/debian unstable/main amd64 libglib2.0-0 amd64 2.62.1-1 [1317 kB]
Get:56 http://ftp.us.debian.org/debian unstable/main amd64 libwebp6 amd64 0.6.1-2+b1 [261 kB]
....
Get:114 http://ftp.us.debian.org/debian unstable/main amd64 libx11-dev amd64 2:1.6.8-1 [822 kB]
Get:115 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jre-headless amd64 8u232-b09-1 [27.2 MB]
Get:116 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jre amd64 8u232-b09-1 [69.7 kB]
Get:117 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jdk-headless amd64 8u232-b09-1 [8183 kB]
Get:118 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jdk amd64 8u232-b09-1 [1603 kB]

comment:8 in reply to:  6 Changed 6 weeks ago by boklm

Replying to sisbell:

The above commit works fine for openjdk-8. When I attempted to add pinning I would get some dependency errors and I didn't see any advantage to making this more complicated.

No pinning means that we are not using Debian buster, but Debian testing, which is not what we want (and not a good idea for reproducible builds as this distribution is changing a lot all the time). From the error it looks like openjdk-8-jre and openjdk-8-jdk-headless need to be added to the pinning configuration too.

I had tried pinning the openjdk-8-jre and openjdk-8-headless but got the same result, they were missing.

Maybe the pinning configuration was wrong. But hard to say without seeing the actual configuration that was used, and how the openjdk package was installed.

If pinning is too complicate, an other option is to download and install the openjdk packages without using apt. In any case, adding the testing repository without pinning is not an option, as that just means we're using Debian testing and not buster.

I also think we should not do this configuration in projects/debootstrap-image/config, but only in the containers that we use for android builds, for example by doing it in var/pre_pkginst in rbm.conf.

I can move the apt sources change to rbm.conf. Assuming I don't modify the pre script in the debootstrap config for the android specific build, how is the # Bug 29158 going to be handled? Is this related to stretch or to the ubuntu version? Can it be removed now?

buster was released after DSA 4371-1, so it is not affected by this issue.

comment:9 in reply to:  7 Changed 6 weeks ago by boklm

Replying to sisbell:

Replying to boklm:

Replying to sisbell:

I kept in stretch for desktop builds, while adding in buster for android.

https://github.com/sisbell/tor-browser-build/commit/8a39f6de0ea833f63ae00e5d3a6f2e2503a6572a

I don't think we need to have two different Ubuntu versions to generate the images. We could just use Ubuntu 19.10 to generate all images.

The above commit works fine for openjdk-8. When I attempted to add pinning I would get some dependency errors and I didn't see any advantage to making this more complicated.

No pinning means that we are not using Debian buster, but Debian testing, which is not what we want (and not a good idea for reproducible builds as this distribution is changing a lot all the time). From the error it looks like openjdk-8-jre and openjdk-8-jdk-headless need to be added to the pinning configuration too.

I do see that without pinning we still use debian stable buster for a bunch of libraries (the first 54). I haven't been able to find a specification of how it does its resolving in this case (so I guess pinning is designed to force certain rules).

Get:50 http://deb.debian.org/debian buster/main amd64 libxdmcp-dev amd64 1:1.1.2-3 [42.2 kB]
Get:51 http://deb.debian.org/debian buster/main amd64 xtrans-dev all 1.3.5-1 [100 kB]
Get:52 http://deb.debian.org/debian buster/main amd64 libxcb1-dev amd64 1.13.1-2 [174 kB]
Get:53 http://deb.debian.org/debian buster/main amd64 libxt-dev amd64 1:1.1.5-1+b3 [426 kB]
Get:54 http://deb.debian.org/debian buster/main amd64 xdg-user-dirs amd64 0.17-2 [53.8 kB]
Get:55 http://ftp.us.debian.org/debian unstable/main amd64 libglib2.0-0 amd64 2.62.1-1 [1317 kB]
Get:56 http://ftp.us.debian.org/debian unstable/main amd64 libwebp6 amd64 0.6.1-2+b1 [261 kB]
....
Get:114 http://ftp.us.debian.org/debian unstable/main amd64 libx11-dev amd64 2:1.6.8-1 [822 kB]
Get:115 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jre-headless amd64 8u232-b09-1 [27.2 MB]
Get:116 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jre amd64 8u232-b09-1 [69.7 kB]
Get:117 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jdk-headless amd64 8u232-b09-1 [8183 kB]
Get:118 http://ftp.us.debian.org/debian unstable/main amd64 openjdk-8-jdk amd64 8u232-b09-1 [1603 kB]

Those packages are exactly the same version in buster and in unstable.

comment:10 Changed 6 weeks ago by sisbell

I try to install ca-certificates but still get the same error regarding Certificate failed. I'll need to investigate more. Its installing from the following location

Get:3 http://archive.ubuntu.com/ubuntu eoan/main amd64 ca-certificates all 20190110 [146 kB]

comment:11 in reply to:  10 ; Changed 6 weeks ago by boklm

Replying to sisbell:

I try to install ca-certificates but still get the same error regarding Certificate failed. I'll need to investigate more. Its installing from the following location

Get:3 http://archive.ubuntu.com/ubuntu eoan/main amd64 ca-certificates all 20190110 [146 kB]

It needs to be installed in the debian buster container as this is where the apt using snapshot.debian.org is run, not the ubuntu one which we only use to run debootstrap.

comment:12 in reply to:  11 ; Changed 6 weeks ago by sisbell

Replying to boklm:

Replying to sisbell:

I try to install ca-certificates but still get the same error regarding Certificate failed. I'll need to investigate more. Its installing from the following location

Get:3 http://archive.ubuntu.com/ubuntu eoan/main amd64 ca-certificates all 20190110 [146 kB]

It needs to be installed in the debian buster container as this is where the apt using snapshot.debian.org is run, not the ubuntu one which we only use to run debootstrap.

Ok so the installation of ca-certificates works from container image. I'm using a specific version of the snapshot, which I think we would want to do to keep the packages pegged to a single snapshot.

https://snapshot.debian.org/archive/debian/20191022T222828Z/

Apt pinning also is working.

I still get missing dependencies like below so I'm assuming I need to specify the entire dependency chain (and versions) here to get openjdk-8 installed? Or is there a simpler way?

The following packages have unmet dependencies:
 openjdk-8-jre-headless : Depends: libc6 (>= 2.29) but 2.28-10 is to be installed
E: Unable to correct problems, you have held broken packages.

comment:13 in reply to:  12 Changed 6 weeks ago by boklm

Replying to sisbell:

I still get missing dependencies like below so I'm assuming I need to specify the entire dependency chain (and versions) here to get openjdk-8 installed? Or is there a simpler way?

The following packages have unmet dependencies:
 openjdk-8-jre-headless : Depends: libc6 (>= 2.29) but 2.28-10 is to be installed
E: Unable to correct problems, you have held broken packages.

It looks like openjdk-8-jre-headless version 8u222-b10-1 from September still depends on libc6 2.28, so we could use that version instead to avoid updating the libc.

comment:14 Changed 6 weeks ago by sisbell

When I try to apply an older snapshot, it says expired. So I'm wondering if specifying a specific snapshot release is really the the way to go (and if it isn't ,the snapshots and dependencies will change).

Running hooks in /etc/ca-certificates/update.d...
done.
Hit:1 http://deb.debian.org/debian buster InRelease
Get:2 http://deb.debian.org/debian buster/main Translation-en [5967 kB]
Get:3 https://snapshot.debian.org/archive/debian/20190930T204642Z unstable InRelease [139 kB]
Reading package lists...
E: Release file for https://snapshot.debian.org/archive/debian/20190930T204642Z/dists/unstable/InRelease is expired (invalid since 15d 22h 9min 17s). Updates for this repository will not be applied.

My pinning specifies release. I'll look into if this is an issue with snapshots /InRelease expiration

Package: openjdk-8-jdk
Pin: origin snapshot.debian.org
Pin: release a=unstable
Pin-Priority: 600

comment:15 Changed 6 weeks ago by boklm

apt can be configured with Acquire::Check-Valid-Until "false" to ignore this expiration.

But in this case, it's probably better to not use apt to install openjdk, and instead manually download the 3 openjdk packages we need, and install them with dpkg.

comment:16 Changed 6 weeks ago by sisbell

I can get the following installed correctly in debootstrap-image config. I can tell this from the logs. But java doesn't show up in the container image when building gradle projects.

     apt-get install -y -q ca-certificates-java java-common libcups2 liblcms2-2 libjpeg8 libfontconfig1 libnss3 libfreetype6 libpcsclite1 libx11-6 libxext6 libxi6 libxrender1 libxtst6 libglib2.0-0 libxrandr2 libxinerama1 libgl1-mesa-glx libgtk2.0-0 libatk-wrapper-java-jni libgif7 libpulse0 isc-dhcp-common
     dpkg -i ./openjdk-8-jre-headless.deb
     dpkg -i ./openjdk-8-jdk-headless.deb
     dpkg -i ./openjdk-8-jre.deb
     dpkg -i ./openjdk-8-jdk.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jre-headless_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jre-headless.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jre_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jre.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jdk_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jdk.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jdk-headless_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jdk-headless.deb

I tried moving everything above to container-image config but the build fails. I'm not sure why there would be a difference.

E: Unable to locate package libjpeg8

comment:17 in reply to:  16 Changed 6 weeks ago by boklm

Replying to sisbell:

I can get the following installed correctly in debootstrap-image config. I can tell this from the logs. But java doesn't show up in the container image when building gradle projects.

     apt-get install -y -q ca-certificates-java java-common libcups2 liblcms2-2 libjpeg8 libfontconfig1 libnss3 libfreetype6 libpcsclite1 libx11-6 libxext6 libxi6 libxrender1 libxtst6 libglib2.0-0 libxrandr2 libxinerama1 libgl1-mesa-glx libgtk2.0-0 libatk-wrapper-java-jni libgif7 libpulse0 isc-dhcp-common
     dpkg -i ./openjdk-8-jre-headless.deb
     dpkg -i ./openjdk-8-jdk-headless.deb
     dpkg -i ./openjdk-8-jre.deb
     dpkg -i ./openjdk-8-jdk.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jre-headless_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jre-headless.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jre_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jre.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jdk_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jdk.deb
  - URL: http://archive.ubuntu.com/ubuntu/pool/universe/o/openjdk-8/openjdk-8-jdk-headless_8u232-b09-0ubuntu1_amd64.deb
    filename: openjdk-8-jdk-headless.deb

Why are you using Ubuntu packages? We want to use Debian buster, so Ubuntu package is not what we should use.

And the Ubuntu container is only used to run debootstrap to create the Debian tarballs, so we don't need openjdk there.

I tried moving everything above to container-image config but the build fails. I'm not sure why there would be a difference.

E: Unable to locate package libjpeg8

Difficult to say without seeing actual patches (I have no idea where you added those lines in the debootstrap-image and container-image configs), but maybe you installed the packages in the Ubuntu container in the first case, and in the buster container in the second.

But anyway, as I said in comment:2 and comment:5 we should not do this in debootstrap-image or container-image, but in rbm.conf in var/pre_pkginst (or maybe var/post_pkginst). And as we don't want to copy the URLs in the input_files of all projects, we should download them in var/pre_pkginst (or var/post_pkginst) using wget (followed by a sha256sum verification).

comment:18 Changed 6 weeks ago by sisbell

I'm clear on the approach now. I'll specify the debian packages, use wget for downloads and add install scripts to rbm.conf. If there are any issues with this all working, I'll check in changes for further suggestions.

comment:19 Changed 6 weeks ago by sisbell

I'm still cleaning up the full build. I also need to add the checksums but the following works perfectly within the rbm.conf

      post_pkginst: |
          OPENJDK_URL=https://snapshot.debian.org/archive/debian/20191028T162808Z/pool/main/o/openjdk-8
          JDK_VERSION=8u222-b10-1~deb9u1_amd64
          apt-get install -y -q wget ca-certificates-java java-common libcups2 liblcms2-2 libjpeg62-turbo libfontconfig1 libnss3 libfreetype6 libpcsclite1 libx11-6 libglib2.0-0 libxrandr2 libxinerama1 libgl1 libgl1-mesa-glx libgtk2.0-0 libatk-wrapper-java-jni libgif7 libpulse0
          wget $OPENJDK_URL/openjdk-8-jdk_$JDK_VERSION.deb
          wget $OPENJDK_URL/openjdk-8-jdk-headless_$JDK_VERSION.deb
          wget $OPENJDK_URL/openjdk-8-jre-headless_$JDK_VERSION.deb     
          wget $OPENJDK_URL/openjdk-8-jre_$JDK_VERSION.deb
          dpkg -i ./openjdk-8-jre-headless_$JDK_VERSION.deb
          dpkg -i ./openjdk-8-jdk-headless_$JDK_VERSION.deb
          dpkg -i ./openjdk-8-jre_$JDK_VERSION.deb
          dpkg -i ./openjdk-8-jdk_$JDK_VERSION.deb

comment:20 Changed 5 weeks ago by boklm

8u222-b10-1~deb9u1 is the version that was built for stretch. Is there any reason to use that one rather than 8u222-b10-1 from Unstable?

Or if we decide to use the package from stretch, then we might as well use the latest version of the package (as it shouldn't require a new glibc version in this case), which is 8u232-b09-1~deb9u1.

comment:21 in reply to:  20 ; Changed 5 weeks ago by sisbell

Replying to boklm:

8u222-b10-1~deb9u1 is the version that was built for stretch. Is there any reason to use that one rather than 8u222-b10-1 from Unstable?

I'm going from your comment:2 which said I should be using snapshot, rather than unstable. This 222 version is the one in snapshot. Should I switch back to unstable?

Or if we decide to use the package from stretch, then we might as well use the latest version of the package (as it shouldn't require a new glibc version in this case), which is 8u232-b09-1~deb9u1.

So should we go with stretch from snapshot or the unstable version?

comment:22 in reply to:  21 Changed 5 weeks ago by boklm

Replying to sisbell:

Replying to boklm:

8u222-b10-1~deb9u1 is the version that was built for stretch. Is there any reason to use that one rather than 8u222-b10-1 from Unstable?

I'm going from your comment:2 which said I should be using snapshot, rather than unstable. This 222 version is the one in snapshot. Should I switch back to unstable?

I did not say snapshot rather than unstable, which does not make any sense since snapshot contains packages from all suites, including unstable. The packages in unstable are changing frequently, so using snapshot.debian.org allows us to use the packages that were available on a specific date.

If we want to use the package from unstable, the latest version that does not require a new glibc seems to be 8u222-b10-1, and we can download it from snapshot.debian.org:
https://snapshot.debian.org/archive/debian/20190930T145141Z/pool/main/o/openjdk-8/openjdk-8-jdk_8u222-b10-1_amd64.deb

It seems using the package from stretch is an other option. And in this case we could use the latest version of the package (which is 8u232-b09-1~deb9u1 currently) as it doesn't require a new glibc:
https://deb.debian.org/debian/pool/main/o/openjdk-8/openjdk-8-jdk_8u232-b09-1~deb9u1_amd64.deb

Or if we decide to use the package from stretch, then we might as well use the latest version of the package (as it shouldn't require a new glibc version in this case), which is 8u232-b09-1~deb9u1.

So should we go with stretch from snapshot or the unstable version?

The package from stretch might be better since it allows us to use a version with the latest security fixes. However I don't know if this package works correctly on buster, so this needs to be tested.

comment:23 Changed 5 weeks ago by sisbell

I tried the stretch version but got the following when building the firefox project (although tor-android-service and tor-onion-proxy-library built fine). I'll collect some more logs on the NPE and then I'll try out the buster version next.

 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryEventEnums.h' '../../../dist/include/mozilla'
 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryHistogramEnums.h' '../../../dist/include/mozilla'
 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryProcessEnums.h' '../../../dist/include/mozilla'
 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryScalarEnums.h' '../../../dist/include/mozilla'
 0:11.78 make[4]: Leaving directory '/var/tmp/build/firefox-e14b7b74e92c/obj-arm-linux-androideabi/toolkit/components/telemetry'
 0:18.14 FAILURE: Build failed with an exception.
 0:18.14 * What went wrong:
 0:18.14 A problem occurred configuring project ':app'.
 0:18.14 > java.lang.NullPointerException (no error message)
 0:18.14 * Try:
 0:18.14 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
 0:18.14 * Get more help at https://help.gradle.org
 0:18.14 Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
 0:18.14 Use '--warning-mode all' to show the individual deprecation warnings.
 0:18.14 See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
 0:18.14 BUILD FAILED in 13s
 

comment:24 in reply to:  23 Changed 5 weeks ago by sisbell

Once I set JAVA_HOME env, it clears up the NPE error. So everything looks good with building with stretch openjdK, up through firefox project.

Replying to sisbell:

I tried the stretch version but got the following when building the firefox project (although tor-android-service and tor-onion-proxy-library built fine). I'll collect some more logs on the NPE and then I'll try out the buster version next.

 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryEventEnums.h' '../../../dist/include/mozilla'
 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryHistogramEnums.h' '../../../dist/include/mozilla'
 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryProcessEnums.h' '../../../dist/include/mozilla'
 0:11.78 ../../../config/nsinstall -R -m 644 'TelemetryScalarEnums.h' '../../../dist/include/mozilla'
 0:11.78 make[4]: Leaving directory '/var/tmp/build/firefox-e14b7b74e92c/obj-arm-linux-androideabi/toolkit/components/telemetry'
 0:18.14 FAILURE: Build failed with an exception.
 0:18.14 * What went wrong:
 0:18.14 A problem occurred configuring project ':app'.
 0:18.14 > java.lang.NullPointerException (no error message)
 0:18.14 * Try:
 0:18.14 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
 0:18.14 * Get more help at https://help.gradle.org
 0:18.14 Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
 0:18.14 Use '--warning-mode all' to show the individual deprecation warnings.
 0:18.14 See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
 0:18.14 BUILD FAILED in 13s
 

comment:25 Changed 5 weeks ago by sisbell

Keywords: TorBrowserTeam201911R added
Status: needs_revisionneeds_review

I have the latest.

https://github.com/sisbell/tor-browser-build/commit/608f2c9623235f574518f9351c28ea4a8c833101

This builds through firefox but fails with https-everywhere stage. I'll fix https-everywhere project to handle buster. I also still need to get the sha checks but thought I would put it out for review of what I have so I can go ahead and make any changes and to make sure this is the approach everyone agrees on.

Using directory /home/shane/github/tor-browser-4/out/firefox/firefox-e14b7b74e92c-android-armv7-b6443a
Building project https-everywhere - https-everywhere-2019.6.27-03f7dd.xpi
Tag 2019.6.27 is signed with key 1073E74EB38BD6D19476CBF8EA9DBF9FB761A677
Created /home/shane/github/tor-browser-4/tmp/rbm-dEXsD/https-everywhere-2019.6.27.tar.gz
Building project container-image - container-image_stretch-amd64-4c35a5822622.tar.gz
Building project debootstrap-image - container-image_stretch-amd64-2.tar.gz
Using file /home/shane/github/tor-browser-4/out/debootstrap-image/container-image_ubuntu-base-19.10-base-amd64.tar.gz
Using file /home/shane/github/tor-browser-4/out/debootstrap-image/apt_1.6.6ubuntu0.1_amd64.deb
Using file /home/shane/github/tor-browser-4/out/debootstrap-image/libapt-pkg5.0_1.6.6ubuntu0.1_amd64.deb
Build log: /home/shane/github/tor-browser-4/logs/debootstrap-image.log
----
Starting build: Mon Nov  4 06:41:08 2019
dpkg: warning: downgrading apt from 1.9.4 to 1.6.6ubuntu0.1
(Reading database ... 4098 files and directories currently installed.)
Preparing to unpack ./apt_1.6.6ubuntu0.1_amd64.deb ...
Unpacking apt (1.6.6ubuntu0.1) over (1.9.4) ...
Selecting previously unselected package libapt-pkg5.0:amd64.
Preparing to unpack .../libapt-pkg5.0_1.6.6ubuntu0.1_amd64.deb ...
Unpacking libapt-pkg5.0:amd64 (1.6.6ubuntu0.1) ...
dpkg: dependency problems prevent configuration of libapt-pkg5.0:amd64:
 dpkg (1.19.7ubuntu2) breaks libapt-pkg5.0 (<< 1.7~b) and is installed.
  Version of libapt-pkg5.0:amd64 to be configured is 1.6.6ubuntu0.1.

dpkg: error processing package libapt-pkg5.0:amd64 (--install):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of apt:
 apt depends on libapt-pkg5.0 (>= 1.6.6ubuntu0.1); however:
  Package libapt-pkg5.0:amd64 is not configured yet.

dpkg: error processing package apt (--install):
 dependency problems - leaving unconfigured
Processing triggers for libc-bin (2.30-0ubuntu2) ...
Errors were encountered while processing:
 libapt-pkg5.0:amd64
 apt

comment:26 in reply to:  25 Changed 5 weeks ago by boklm

Replying to sisbell:

This builds through firefox but fails with https-everywhere stage. I'll fix https-everywhere project to handle buster.

I think the update of http-everywhere to use buster is unrelated to this ticket. So it can be done in a separate ticket.

comment:27 in reply to:  25 Changed 5 weeks ago by boklm

Replying to sisbell:

I have the latest.

https://github.com/sisbell/tor-browser-build/commit/608f2c9623235f574518f9351c28ea4a8c833101

This builds through firefox but fails with https-everywhere stage. I'll fix https-everywhere project to handle buster. I also still need to get the sha checks but thought I would put it out for review of what I have so I can go ahead and make any changes and to make sure this is the approach everyone agrees on.

Using directory /home/shane/github/tor-browser-4/out/firefox/firefox-e14b7b74e92c-android-armv7-b6443a
Building project https-everywhere - https-everywhere-2019.6.27-03f7dd.xpi
Tag 2019.6.27 is signed with key 1073E74EB38BD6D19476CBF8EA9DBF9FB761A677
Created /home/shane/github/tor-browser-4/tmp/rbm-dEXsD/https-everywhere-2019.6.27.tar.gz
Building project container-image - container-image_stretch-amd64-4c35a5822622.tar.gz
Building project debootstrap-image - container-image_stretch-amd64-2.tar.gz
Using file /home/shane/github/tor-browser-4/out/debootstrap-image/container-image_ubuntu-base-19.10-base-amd64.tar.gz
Using file /home/shane/github/tor-browser-4/out/debootstrap-image/apt_1.6.6ubuntu0.1_amd64.deb
Using file /home/shane/github/tor-browser-4/out/debootstrap-image/libapt-pkg5.0_1.6.6ubuntu0.1_amd64.deb
Build log: /home/shane/github/tor-browser-4/logs/debootstrap-image.log
----
Starting build: Mon Nov  4 06:41:08 2019
dpkg: warning: downgrading apt from 1.9.4 to 1.6.6ubuntu0.1
(Reading database ... 4098 files and directories currently installed.)
Preparing to unpack ./apt_1.6.6ubuntu0.1_amd64.deb ...
Unpacking apt (1.6.6ubuntu0.1) over (1.9.4) ...
Selecting previously unselected package libapt-pkg5.0:amd64.
Preparing to unpack .../libapt-pkg5.0_1.6.6ubuntu0.1_amd64.deb ...
Unpacking libapt-pkg5.0:amd64 (1.6.6ubuntu0.1) ...
dpkg: dependency problems prevent configuration of libapt-pkg5.0:amd64:
 dpkg (1.19.7ubuntu2) breaks libapt-pkg5.0 (<< 1.7~b) and is installed.
  Version of libapt-pkg5.0:amd64 to be configured is 1.6.6ubuntu0.1.

dpkg: error processing package libapt-pkg5.0:amd64 (--install):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of apt:
 apt depends on libapt-pkg5.0 (>= 1.6.6ubuntu0.1); however:
  Package libapt-pkg5.0:amd64 is not configured yet.

dpkg: error processing package apt (--install):
 dependency problems - leaving unconfigured
Processing triggers for libc-bin (2.30-0ubuntu2) ...
Errors were encountered while processing:
 libapt-pkg5.0:amd64
 apt

I think you get this error when building any debootstrap image that is not buster. In this case https-everywhere is using stretch.

The issue is that in projects/debootstrap-image/config you are trying to install some downloaded apt packages, which doesn't work since those packages are for an Ubuntu release older than the one we now use. Since we're using Ubuntu 19.10 now, there is no need to install specific apt packages.

comment:28 Changed 4 weeks ago by gk

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Some comments:

1) This ticket is about using Debian 10 for our Android containers and #31992 is about removing our apktool workaround. Yet, it seems you are essentially mixing both issues in one commit saying it fixes #31110 which is confusing. Could we have two commits here one for #31110 (that is upgrading to buster) and one (on top of it) for #31992? Ideally, the build would still work if each commit would be used for building.

2) Please remove the whitespace at the end of

+          JDK_VERSION=8u232-b09-1~deb9u1_amd64

comment:29 Changed 4 weeks ago by boklm

An other comment:
In var/pre_pkginst you are setting the JAVA_HOME environment variable. Is that useful? The only things that come after that are packages installation.

comment:30 Changed 4 weeks ago by sisbell

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201911 removed
Status: needs_revisionneeds_review

I cleaned up the configs. I added an install_jdk flag to prevent openjdk install in https-everywhere project. Otherwise the build would break on installation.

https://github.com/sisbell/tor-browser-build/commit/1291e758e91784d739c9051873dc07073811df16

It looks very close, I'm still investigating why I get the following error in tor-browser project when using apksigner.

  inflating: tmp/removed-files       
Error: Unable to access jarfile /usr/share/apksigner/apksigner.jar

comment:31 in reply to:  30 Changed 4 weeks ago by gk

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

I cleaned up the configs. I added an install_jdk flag to prevent openjdk install in https-everywhere project. Otherwise the build would break on installation.

https://github.com/sisbell/tor-browser-build/commit/1291e758e91784d739c9051873dc07073811df16

It looks very close, I'm still investigating why I get the following error in tor-browser project when using apksigner.

  inflating: tmp/removed-files       
Error: Unable to access jarfile /usr/share/apksigner/apksigner.jar

Once you fixed that bit, please add the ticket back into needs_review. And please address *all* comments before putting a ticket into needs_review (That does not necessarily mean you need to fix all concerns as they might not be valid, but they should be addressed).

Last edited 4 weeks ago by gk (previous) (diff)

comment:32 in reply to:  30 Changed 4 weeks ago by boklm

Replying to sisbell:

I cleaned up the configs. I added an install_jdk flag to prevent openjdk install in https-everywhere project. Otherwise the build would break on installation.

I think you don't need an install_jdk flag. You can just define var/pre_pkginst to empty in http-everywhere.

comment:33 in reply to:  30 ; Changed 4 weeks ago by boklm

Replying to sisbell:

It looks very close, I'm still investigating why I get the following error in tor-browser project when using apksigner.

  inflating: tmp/removed-files       
Error: Unable to access jarfile /usr/share/apksigner/apksigner.jar

The path to apksigner in buster is /usr/lib/android-sdk/build-tools/debian/apksigner.jar.

comment:34 in reply to:  33 Changed 4 weeks ago by sisbell

Replying to boklm:

Replying to sisbell:

It looks very close, I'm still investigating why I get the following error in tor-browser project when using apksigner.

  inflating: tmp/removed-files       
Error: Unable to access jarfile /usr/share/apksigner/apksigner.jar

The path to apksigner in buster is /usr/lib/android-sdk/build-tools/debian/apksigner.jar.

So I think removing the install_jdk and getting the apksigner path updated should do the trick to get everything building. I'll get this building today and test out.

comment:35 Changed 4 weeks ago by sisbell

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201911 removed
Status: needs_revisionneeds_review

Fixed

  • added correct apksigner path for debian
  • removed use of install_jdk flag (override pre_pkginst)

https://github.com/sisbell/tor-browser-build/commit/da928c8cc046ad71e718ddd1eb2c4690af8097dd

comment:36 in reply to:  35 Changed 4 weeks ago by boklm

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

Fixed

  • added correct apksigner path for debian
  • removed use of install_jdk flag (override pre_pkginst)

https://github.com/sisbell/tor-browser-build/commit/da928c8cc046ad71e718ddd1eb2c4690af8097dd

in var/pre_pkginst:

          sha256sum 04f35746d85ea065a06a79a0ffb95825bd591a1b3a0f7cc641d06e7d263bbef5 openjdk-8-jdk_$JDK_VERSION.deb | sha256sum -c
          sha256sum 92b4f8fb77d793a86e0b03b3b0750592b40a26a5d75956d10dd984a7b3aad4c9 openjdk-8-jdk-headless_$JDK_VERSION.deb | sha256sum -c
          sha256sum 84bf52b6cce20ead08b0d5b9fd9b81b4aa3da385ca951b313fe11d5cb1aa4d17 openjdk-8-jre-headless_$JDK_VERSION.deb | sha256sum -c
          sha256sum 75057205b56791edbc4dd0358cb56efd7d20ce0cb730d6ecba38e37267d72b9b openjdk-8-jre_$JDK_VERSION.deb | sha256sum -c

This is wrong, we could replace the checksum in those lines by anything, and it will never fail. Running sha245sum checksum some-file | sha256sum -c will complain that the file checksum does not exist, but it is basically equivalent to running sha245sum some-file | sha256sum -c which is equivalent to doing nothing.

comment:37 Changed 3 weeks ago by sisbell

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201911 removed
Status: needs_revisionneeds_review

I needed to echo the checksum, so made this change and verified that an incorrect checksum fails the build.

https://github.com/sisbell/tor-browser-build/commit/9107a6849ec8f7c7e3073a125e93750bf3eb529e

comment:38 in reply to:  37 Changed 3 weeks ago by boklm

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

I needed to echo the checksum, so made this change and verified that an incorrect checksum fails the build.

https://github.com/sisbell/tor-browser-build/commit/9107a6849ec8f7c7e3073a125e93750bf3eb529e

This is still not good:

+          sha256sum 04f35746d85ea065a06a79a0ffb95825bd591a1b3a0f7cc641d06e7d263bbef5 openjdk-8-jdk_$JDK_VERSION.deb | sha256sum -c
+          sha256sum 92b4f8fb77d793a86e0b03b3b0750592b40a26a5d75956d10dd984a7b3aad4c9 openjdk-8-jdk-headless_$JDK_VERSION.deb | sha256sum -c
+          sha256sum 84bf52b6cce20ead08b0d5b9fd9b81b4aa3da385ca951b313fe11d5cb1aa4d17 openjdk-8-jre-headless_$JDK_VERSION.deb | sha256sum -c
+          sha256sum 75057205b56791edbc4dd0358cb56efd7d20ce0cb730d6ecba38e37267d72b9b openjdk-8-jre_$JDK_VERSION.deb | sha256sum -c

comment:39 Changed 3 weeks ago by sisbell

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201911 removed
Status: needs_revisionneeds_review

Sorry about that, I made a mistake on the check in

Try this one

https://github.com/sisbell/tor-browser-build/commit/dac5088c383692aa22617d7d1e628e4a348abab5


a bad checksum will look like

2019-11-13 18:05:53 (3.98 MB/s) - 'openjdk-8-jre_8u232-b09-1~deb9u1_amd64.deb' saved [69506/69506]

openjdk-8-jdk_8u232-b09-1~deb9u1_amd64.deb: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match

comment:40 Changed 3 weeks ago by gk

How much of OpenJDK do we really need? That is, do we really need jdk, jdk-headless, jre, and jre-headless? If not, please remove the parts that are not needed (together with dependencies not needed).

comment:41 in reply to:  40 ; Changed 3 weeks ago by sisbell

Yes, we need these components. Removing them results in

dpkg: dependency problems prevent configuration of openjdk-8-jdk:amd64:
 openjdk-8-jdk:amd64 depends on openjdk-8-jdk-headless (= 8u232-b09-1~deb9u1); however:
  Package openjdk-8-jdk-headless is not installed.

dpkg: error processing package openjdk-8-jdk:amd64 (--install):
 dependency problems - leaving unconfigured

Similar errors occur if removing the JRE packages.

Replying to gk:

How much of OpenJDK do we really need? That is, do we really need jdk, jdk-headless, jre, and jre-headless? If not, please remove the parts that are not needed (together with dependencies not needed).

comment:42 in reply to:  41 Changed 2 weeks ago by gk

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

Yes, we need these components. Removing them results in

dpkg: dependency problems prevent configuration of openjdk-8-jdk:amd64:
 openjdk-8-jdk:amd64 depends on openjdk-8-jdk-headless (= 8u232-b09-1~deb9u1); however:
  Package openjdk-8-jdk-headless is not installed.

dpkg: error processing package openjdk-8-jdk:amd64 (--install):
 dependency problems - leaving unconfigured

Similar errors occur if removing the JRE packages.

I wonder why then just using the headless versions and not the other ones are working for me. Could you please fix that up? Please make sure as well that we really need all the packages you want to install with apt-get install in the pre_pkginst step.

Replying to gk:

How much of OpenJDK do we really need? That is, do we really need jdk, jdk-headless, jre, and jre-headless? If not, please remove the parts that are not needed (together with dependencies not needed).

comment:43 Changed 2 weeks ago by sysrqb

Cc: tbb-team added
Owner: changed from tbb-team to sisbell
Status: needs_revisionassigned

comment:44 Changed 2 weeks ago by gk

Cc: sisbell removed

comment:45 Changed 2 weeks ago by sysrqb

Cc: sisbell added
Status: assignedneeds_revision

comment:46 Changed 2 weeks ago by sisbell

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201911 removed
Status: needs_revisionneeds_review

I just included the headless jre-headless/jdk-headless and dependencies in this version. I verified a complete build.

https://github.com/sisbell/tor-browser-build/commit/f370600beb2b3fb5ff722f5a737874074fb60397

comment:47 in reply to:  46 Changed 2 weeks ago by gk

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

I just included the headless jre-headless/jdk-headless and dependencies in this version. I verified a complete build.

https://github.com/sisbell/tor-browser-build/commit/f370600beb2b3fb5ff722f5a737874074fb60397

Thanks! If you look at the dependencies you'll see that libxtst6 already depends on libx11-6, libxext6, and libxi6. So, there is no need to add them explicitly (we don't add all the other implicit dependencies either).

comment:48 Changed 9 days ago by sisbell

Status: needs_revisionneeds_review

Looks like we just need one additional dependency installed for the JDK headless installation. I verified a full build.

https://github.com/sisbell/tor-browser-build/commit/005b6651ba42737c7e1bd177f01286294355c02f

comment:49 Changed 9 days ago by sisbell

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201911 removed

comment:50 in reply to:  48 ; Changed 9 days ago by gk

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201911R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

Looks like we just need one additional dependency installed for the JDK headless installation. I verified a full build.

https://github.com/sisbell/tor-browser-build/commit/005b6651ba42737c7e1bd177f01286294355c02f

Okay, we are close. Looking over the patch again I see you are pointing to https://deb.debian.org/debian/pool/main/o/openjdk-8 for the openjdk packages. However, that seems to be brittle to me as it's easily conceivable that our version is getting superseded by newer updates in the couple of weeks, essentially getting unavailable. Better is using snapshot.debian.org (e.g. https://snapshot.debian.org/archive/debian/20191031T212011Z/pool/main/o/openjdk-8/). boklm: anything else that should get fixed?

Last edited 9 days ago by gk (previous) (diff)

comment:51 in reply to:  50 ; Changed 9 days ago by boklm

Replying to gk:

Replying to sisbell:

Looks like we just need one additional dependency installed for the JDK headless installation. I verified a full build.

https://github.com/sisbell/tor-browser-build/commit/005b6651ba42737c7e1bd177f01286294355c02f

Okay, we are close. Looking over the patch again I see you are pointing to https://deb.debian.org/debian/pool/main/o/openjdk-8 for the openjdk packages. However, that seems to be brittle to me as it's easily conceivable that our version is getting superseded by newer updates in the couple of weeks, essentially getting unavailable. Better is using snapshot.debian.org (e.g. https://snapshot.debian.org/archive/debian/20191031T212011Z/pool/main/o/openjdk-8/).

As it's an update for a stable distribution, I think previous versions of the packages are not removed from mirrors (until the whole suite is removed, which I think will not be before 2022 for stretch). So I think using https://deb.debian.org/ is fine (but using snapshot.debian.org is fine too).

boklm: anything else that should get fixed?

I did not try to do a build yet, but the patch looks good to me.

I think we should also check that the creation of wheezy-amd64 and stretch-amd64 images in debootstrap-image is still working correctly with the updated Ubuntu version.

comment:52 in reply to:  51 ; Changed 8 days ago by gk

Replying to boklm:

Replying to gk:

Replying to sisbell:

Looks like we just need one additional dependency installed for the JDK headless installation. I verified a full build.

https://github.com/sisbell/tor-browser-build/commit/005b6651ba42737c7e1bd177f01286294355c02f

Okay, we are close. Looking over the patch again I see you are pointing to https://deb.debian.org/debian/pool/main/o/openjdk-8 for the openjdk packages. However, that seems to be brittle to me as it's easily conceivable that our version is getting superseded by newer updates in the couple of weeks, essentially getting unavailable. Better is using snapshot.debian.org (e.g. https://snapshot.debian.org/archive/debian/20191031T212011Z/pool/main/o/openjdk-8/).

As it's an update for a stable distribution, I think previous versions of the packages are not removed from mirrors (until the whole suite is removed, which I think will not be before 2022 for stretch). So I think using https://deb.debian.org/ is fine (but using snapshot.debian.org is fine too).

I think you are wrong here. Have a look at

https://snapshot.debian.org/archive/debian/20191001T024204Z/pool/main/o/openjdk-8/ and
https://snapshot.debian.org/archive/debian/20191031T212011Z/pool/main/o/openjdk-8/

in the former you still find 8u232-b04 binaries but not the latter right now there are 8u232-b09-1~deb9u1 ones *instead* of the former which the stable archive mirrors but it seems that's not guaranteed to hold. That is there seems to be no reason to assume that 8u232-b09-1~deb9u1 could not get replaced in the future by a successor, which is why I think we should go the snapshot route to be on the safe side.

boklm: anything else that should get fixed?

I did not try to do a build yet, but the patch looks good to me.

I think we should also check that the creation of wheezy-amd64 and stretch-amd64 images in debootstrap-image is still working correctly with the updated Ubuntu version.

Yes, please.

comment:53 in reply to:  52 Changed 8 days ago by boklm

Replying to gk:

Replying to boklm:

Replying to gk:

Replying to sisbell:

Looks like we just need one additional dependency installed for the JDK headless installation. I verified a full build.

https://github.com/sisbell/tor-browser-build/commit/005b6651ba42737c7e1bd177f01286294355c02f

Okay, we are close. Looking over the patch again I see you are pointing to https://deb.debian.org/debian/pool/main/o/openjdk-8 for the openjdk packages. However, that seems to be brittle to me as it's easily conceivable that our version is getting superseded by newer updates in the couple of weeks, essentially getting unavailable. Better is using snapshot.debian.org (e.g. https://snapshot.debian.org/archive/debian/20191031T212011Z/pool/main/o/openjdk-8/).

As it's an update for a stable distribution, I think previous versions of the packages are not removed from mirrors (until the whole suite is removed, which I think will not be before 2022 for stretch). So I think using https://deb.debian.org/ is fine (but using snapshot.debian.org is fine too).

I think you are wrong here. Have a look at

https://snapshot.debian.org/archive/debian/20191001T024204Z/pool/main/o/openjdk-8/ and
https://snapshot.debian.org/archive/debian/20191031T212011Z/pool/main/o/openjdk-8/

in the former you still find 8u232-b04 binaries but not the latter right now there are 8u232-b09-1~deb9u1 ones *instead* of the former which the stable archive mirrors but it seems that's not guaranteed to hold. That is there seems to be no reason to assume that 8u232-b09-1~deb9u1 could not get replaced in the future by a successor, which is why I think we should go the snapshot route to be on the safe side.

I think 8u232-b04 was removed because it never was in the stable/oldstable suite, but only in experimental:
https://tracker.debian.org/news/1062184/accepted-openjdk-8-8u232-b04-1-source-into-experimental/

Packages from unstable, experimental and testing are removed from mirrors when a new version of the package is available, however it seemed to me this was not the case for stable or oldstable, but after looking at it more closely it seems it is not always the case. So using snapshot sounds good.

comment:54 Changed 4 days ago by sisbell

Keywords: TorBrowserTeam201912R added; TorBrowserTeam201911 removed
Status: needs_revisionneeds_review

I'm using the latest snapshot. From my understanding of the previous discussions, 8u232-b09-1~deb9u1 version will not be removed from snapshot since it has also been published to stable (if I missed this let me know).

The commit uses snapshot. I verified build for Android.

https://github.com/sisbell/tor-browser-build/commit/26be3f16054b7d9b1c4c6cf0332d90f92511fc3e

comment:55 Changed 4 days ago by gk

What about boklm's comment:51 (last two lines)?

comment:56 Changed 4 days ago by pili

Reviewer: gk

gk to review these tickets

comment:57 Changed 2 days ago by gk

Keywords: TorBrowserTeam201912 added; TorBrowserTeam201912R removed
Status: needs_reviewneeds_information

Taking this off or our review radar until the tests in comment:51 are done.

Note: See TracTickets for help on using tickets.