Opened 5 months ago

Last modified 2 weeks ago

#31650 needs_review enhancement

pre-crunch and pre-strip PNG in tor-service-android to make it reproducible

Reported by: eighthave Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-mobile, TorBrowserTeam202001R
Cc: sysrqb, sisbell, gk, n8fr8 Actual Points:
Parent ID: #31042 Points: 1
Reviewer: sysrqb Sponsor:

Description

PNG crunching is not a deterministic process, especially the way aapt does it. This makes the builds not reproducible. The easy solution to this is to pre-crunch the PNGs and commit them to git. This also uses exiftool to strip any metadata from the PNG. This then disables running the crunch as part of the gradle build process.

https://medium.com/@duhroach/smaller-pngs-and-android-s-aapt-tool-4ce38a24019d

I have submitted this upstream to Orbot:
https://github.com/guardianproject/orbot/pull/252

And it is here in my tor-android-service-fork:
https://gitlab.com/eighthave/tor-android-service/commits/png-pre-compress

Once it is merged, someone will need to do:

$ sudo apt install exiftool zopfli
$ cd /path/to/tor-android-service
$ ./tools/png-pre-compress

Then commit all the changed PNGs.

Child Tickets

Attachments (1)

0001-manually-use-aapt-singleCrunch-for-reproducible-buil.patch (1.6 KB) - added by eighthave 5 months ago.

Download all attachments as: .zip

Change History (15)

comment:1 Changed 5 months ago by eighthave

Component: ApplicationsApplications/Tor Browser
Owner: set to tbb-team
Status: newneeds_review

comment:2 Changed 5 months ago by gk

Keywords: TorBrowserTeam201909R added

comment:3 Changed 4 months ago by pili

Keywords: TorBrowserTeam201910R added; TorBrowserTeam201909R removed

We're now in October, moving September outstanding reviews to October

comment:4 Changed 3 months ago by sysrqb

I wonder why we were able to create deterministic builds with pngs without this patch. Maybe they are deterministic if the environments are identical (such as in tor-browser-build)?

comment:5 Changed 3 months ago by eighthave

Maybe Mozilla is already pre-crunching?

comment:6 Changed 3 months ago by eighthave

Oh right, but Orbot is included. It seems in some cases they can be reproducibly crunched. Pre-crunching and disabling the automatic crunching ensures that there are no surprises.

comment:7 in reply to:  6 Changed 3 months ago by sisbell

Looks good to me. I can see the pngs updated after running the script. I think just adding a small readme to the repo to explain what the script does would be helpful.

Replying to eighthave:

Oh right, but Orbot is included. It seems in some cases they can be reproducibly crunched. Pre-crunching and disabling the automatic crunching ensures that there are no surprises.

comment:8 Changed 3 months ago by pili

Keywords: TorBrowserTeam201911 added

Moving tickets to November 2019

comment:9 Changed 3 months ago by pili

Points: 1

comment:10 Changed 3 months ago by gk

Keywords: TorBrowserTeam201911R added; TorBrowserTeam201910R removed

There is no way to do reviews in October 2019 anymore.

comment:11 Changed 3 months ago by gk

Keywords: TorBrowserTeam201911 removed

No need for duplicate keyword.

comment:12 Changed 7 weeks ago by gk

Keywords: TorBrowserTeam201912R added; TorBrowserTeam201911R removed

We are in December now.

comment:13 Changed 7 weeks ago by pili

Reviewer: sysrqb

sysrqb to review

comment:14 Changed 2 weeks ago by sysrqb

Keywords: TorBrowserTeam202001R added; TorBrowserTeam201912R removed
Note: See TracTickets for help on using tickets.