Opened 6 years ago

Closed 4 years ago

#8283 closed enhancement (implemented)

track hashes for all TBB dep source files

Reported by: ioerror Owned by: ioerror
Priority: Medium Milestone:
Component: Applications/Tor bundles/installation Version:
Severity: Keywords:
Cc: erinn, mikeperry, trams, sebastian Actual Points:
Parent ID: #8288 Points:
Reviewer: Sponsor:

Description

I think that to increase the ability for people to build TBB, we should track the expected hashes in git. I also think that during the source-dance` target, we should ensure that before we unpack anything that we have verified the downloaded files have the expected hashes.

Child Tickets

Attachments (1)

versions.mk.patch (2.7 KB) - added by ioerror 6 years ago.
patch to add hashes to tbb make process

Download all attachments as: .zip

Change History (17)

comment:1 Changed 6 years ago by ioerror

Cc: erinn mikeperry trams sebastian added

comment:2 Changed 6 years ago by ioerror

I have found the following sha256 hashes for the following versions:

027acbbafd682644ef44cbeea3d886498e7cac9d3af769a90c9beebc3fdc61d1  firefox-17.0.3esr.source.tar.bz2
22a530a8a5ba1cb9c080cba033206b17dacd21437762155c6d30ee6469f574f5  libevent-2.0.21-stable.tar.gz
bd6bab0534d75ceec08be3ddc08d57096013b829f81cd39ebd468f2c7d10d1bc  libpng-1.5.14.tar.bz2
6e0ed147e9be4b9f89862b5e2597d355427e977a69c8dfb6e15c04530d3bedb3  obfsproxy-0.1.4.tar.gz
2982b2e9697a857b336c5c1b1b7b463747e5c1d560f25f6ace95365791b1efd1  openssl-1.0.0k.tar.gz
ef851a36aa41b4ad7a3e4c96ca27eaed2a629a6d2fa06c20f072118caed87ae8  qt-everywhere-opensource-src-4.8.1.tar.gz
bb2d6f1136f33e11d37e6e34184143bf191e59501613daf33ae3d6f78f3176a0  tor-0.2.3.25.tar.gz
c4008e7e7781dddf4a8670a435da6496dc9309dbdbc6125ac6d2cc871bdc1be7  vidalia-0.2.21.tar.gz
fa9c9c8638efb8cb8ef5e4dd5453e455751e1c530b1595eed466e1be9b7e26c5  zlib-1.2.7.tar.gz

Do these versions and hashes match the expected values for everyone?

comment:3 Changed 6 years ago by ioerror

Here is a patch that integrates the hashes into version.mk:

diff --git a/build-scripts/versions.mk b/build-scripts/versions.mk
index 304ed63..d43ac48 100644
--- a/build-scripts/versions.mk
+++ b/build-scripts/versions.mk
@@ -3,14 +3,22 @@
 RELEASE_VER=2.3.25
 
 ZLIB_VER=1.2.7
+ZLIB_HASH=fa9c9c8638efb8cb8ef5e4dd5453e455751e1c530b1595eed466e1be9b7e26c5
 OPENSSL_VER=1.0.0k
-LIBPNG_VER=1.5.13
+OPENSSL_HASH=2982b2e9697a857b336c5c1b1b7b463747e5c1d560f25f6ace95365791b1efd1
+LIBPNG_VER=1.5.14
+LIBPNG_HASH=bd6bab0534d75ceec08be3ddc08d57096013b829f81cd39ebd468f2c7d10d1bc
 QT_VER=4.8.1
+QT_HASH=ef851a36aa41b4ad7a3e4c96ca27eaed2a629a6d2fa06c20f072118caed87ae8
 VIDALIA_VER=0.2.21
+VIDALIA_HASH=c4008e7e7781dddf4a8670a435da6496dc9309dbdbc6125ac6d2cc871bdc1be7
 LIBEVENT_VER=2.0.21-stable
+LIBEVENT_HASH=22a530a8a5ba1cb9c080cba033206b17dacd21437762155c6d30ee6469f574f5
 TOR_VER=0.2.3.25
+TOR_HASH=bb2d6f1136f33e11d37e6e34184143bf191e59501613daf33ae3d6f78f3176a0
 PIDGIN_VER=2.6.4
 FIREFOX_VER=17.0.3esr
+FIREFOX_HASH=027acbbafd682644ef44cbeea3d886498e7cac9d3af769a90c9beebc3fdc61d1
 MOZBUILD_VER=1.5.1
 TORBUTTON_VER=1.5.0
 NOSCRIPT_VER=2.6.4.4

comment:4 Changed 6 years ago by ioerror

I think I downloaded all of the xpi files and source code (except the pidgin stuff) releases:

027acbbafd682644ef44cbeea3d886498e7cac9d3af769a90c9beebc3fdc61d1  firefox-17.0.3esr.source.tar.bz2
a26656d7abd196cfdcb959b267c0bae292e2edb6f575835b43ee2b4fd00eef81  httpseverywhere.xpi
1dc4d01496c8b6d4db1ad011f96cd1631609a39f7803fda8ac6957ba8907edbe  langpack_ar.xpi
e03f49c663e59d089716ab9f80ceefd0cca764c6a5c7ff3bcb10d26f45abdd98  langpack_de.xpi
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  langpack_en-US.xpi
a2aaed349b69759903443c1872178371f1b833258576f2adb771ac07e003a0a0  langpack_es-ES.xpi
13b19e41f9643086290f189588b1761807541a6360af44f6839e7601846f3a33  langpack_fa.xpi
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  langpack_fit.xpi
56509ad22a6e5af3d3d44735f617ffd1041bae5e8b0083d890c710c9bd8e8df5  langpack_fr.xpi
420f30da2da62b9ddabe9b62bac72fb7c87c6bb98775ede068c82e1698a496b1  langpack_it.xpi
c9313970e264b558a8ec60e1f81e5d4c33cf1bd4461300fdd6db782690619fff  langpack_ko.xpi
b4bef29025bd1f9a0d8ac49d7a3ffe6b5f53588690c6f2fc2e75caa5aed54fe5  langpack_nl.xpi
a8f148183322ef1e401ed9dfb292dadd14cac93de7d490c931906597fda4c85d  langpack_pl.xpi
edcfe131c4f2d93678e0887f074e905f263c0b97302bc77c3b433f4a6ad37a2f  langpack_pt-PT.xpi
9922d6ab08a8e2a22837863c833c66d1c0ca3441030bab416cb8e4d6f13e0430  langpack_ru.xpi
a44337350fcd840414b7dbac29b64783e42334b35aa614f3347db48deefc6863  langpack_vi.xpi
0ccc9439303856cdb03cce340801085265dde6c3e43d139d009f0684697f86c4  langpack_zh-CN.xpi
22a530a8a5ba1cb9c080cba033206b17dacd21437762155c6d30ee6469f574f5  libevent-2.0.21-stable.tar.gz
bd6bab0534d75ceec08be3ddc08d57096013b829f81cd39ebd468f2c7d10d1bc  libpng-1.5.14.tar.bz2
7af142eeb270f26dc71c64fbe3882a950be35a568609466a4ab4c705b4ca9c71  noscript.xpi
6e0ed147e9be4b9f89862b5e2597d355427e977a69c8dfb6e15c04530d3bedb3  obfsproxy-0.1.4.tar.gz
2982b2e9697a857b336c5c1b1b7b463747e5c1d560f25f6ace95365791b1efd1  openssl-1.0.0k.tar.gz
ef851a36aa41b4ad7a3e4c96ca27eaed2a629a6d2fa06c20f072118caed87ae8  qt-everywhere-opensource-src-4.8.1.tar.gz
bb2d6f1136f33e11d37e6e34184143bf191e59501613daf33ae3d6f78f3176a0  tor-0.2.3.25.tar.gz
ddd98b04cf1f69629b6ded80296bad4b3984d44b6bea085fde3b4f2d68eb2e7b  torbutton.xpi
c4008e7e7781dddf4a8670a435da6496dc9309dbdbc6125ac6d2cc871bdc1be7  vidalia-0.2.21.tar.gz
fa9c9c8638efb8cb8ef5e4dd5453e455751e1c530b1595eed466e1be9b7e26c5  zlib-1.2.7.tar.gz

Changed 6 years ago by ioerror

Attachment: versions.mk.patch added

patch to add hashes to tbb make process

comment:5 Changed 6 years ago by ioerror

Status: newneeds_review

Ok, I've put up a patch for the versions.mk file - it only adds data - it does not change the workflow or use the hashes.

Does this seem reasonable?

comment:6 Changed 6 years ago by arma

So when some component changes, somebody edits the versions.mk file by hand?

That isn't going to work for people who don't like text editors, or when lots of things change. Maybe there should be a script to take a bunch of files and generate that file-with-hashes out of them? Then when there's a new thing, you fetch it, rerun the script, and commit the new file.

And if the versions.mk file does other stuff too, then it may be wisest to keep the auto-generated stuff as a separate file.

I guess it looks like versions.mk had versions in it before, not just hashes, so the problem I describe here might not be a new problem.

comment:7 Changed 6 years ago by arma

(Adding expected hashes when the build scripts fetches a bunch of unauthenticated stuff from the Internet is a great plan.)

comment:8 Changed 6 years ago by ioerror

Parent ID: #8288

comment:9 in reply to:  6 Changed 6 years ago by ioerror

Replying to arma:

So when some component changes, somebody edits the versions.mk file by hand?

That isn't going to work for people who don't like text editors, or when lots of things change. Maybe there should be a script to take a bunch of files and generate that file-with-hashes out of them? Then when there's a new thing, you fetch it, rerun the script, and commit the new file.

The build process is not for people who cannot use a text editor or build software in a terminal.

And if the versions.mk file does other stuff too, then it may be wisest to keep the auto-generated stuff as a separate file.

Currently, it is populated with manual data - eg: version numbers and urls.

I guess it looks like versions.mk had versions in it before, not just hashes, so the problem I describe here might not be a new problem.

No, I don't think so. If we update the version, we should ensure we also either verify the download (and add the hash of the file at the same time) or we should probably not update the version.

comment:10 in reply to:  7 Changed 6 years ago by ioerror

Replying to arma:

(Adding expected hashes when the build scripts fetches a bunch of unauthenticated stuff from the Internet is a great plan.)

OK - glad to hear it. Shall I prepare a patch that checks the hash of the downloaded file against the hash in the Makefile? I'll wait until we hear from Erinn or Mike or both before I spend time implementing such a change.

comment:11 Changed 6 years ago by ioerror

I've found another hash that was worth mentioning for the alpha bundle:

11192135d8ed094139febc44add034e78a3bd26ade0d0ae513273a2c6654fbfb  addon-352704-latest.xpi

This is pdfjs.xpi by another name, I suppose.

comment:12 Changed 6 years ago by ioerror

I've opened #8289 to address actually using these hashes. This bug should just be about all of the hashes we expect, know and love and not about using the hash for anything.

comment:13 Changed 6 years ago by ioerror

Of course I forgot Tor's alpha release:

2f4122f7dc88ccd319a4677e46ae57e3ac157fd1da45d77e4e4d53f9e81953eb  tor-0.2.4.10-alpha.tar.gz

comment:14 Changed 6 years ago by mikeperry

Status: needs_reviewneeds_revision

I think we want to wait to merge #8289 when we merge this, so we're sure everything is working.

I also think we should only patch the alpha makefiles for now (or at least first?), though. I don't want to risk destabilizing the stable makefiles for this just yet. Of course I also realize that our alpha makefile changes are a pain to backport to the stable makefiles. We need some kind of solution for that (perhaps just copying the alpha makefiles to the stable makefiles at some point?).

comment:15 Changed 6 years ago by ioerror

I'd like to add it before we directly use the hashes for anything.

comment:16 Changed 4 years ago by Sebastian

Resolution: implemented
Status: needs_revisionclosed

This seems to be no longer required with reproducible builds

Note: See TracTickets for help on using tickets.