Opened 5 years ago

Closed 5 years ago

#13857 closed enhancement (fixed)

Create update related information per channel

Reported by: gk Owned by: boklm
Priority: Medium Milestone:
Component: Applications/Tor bundles/installation Version:
Severity: Keywords: TorBrowserTeam201501R, tbb-4.5-alpha-3, MikePerry201501R
Cc: boklm, brade, mcs Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We are currently creating update related information for all channels even if we just bump the version in a particular channel. Instead we should try to create only the update information (incremental .mar files; XML file updates...) needed for the particular channel: if we release a new stable version there should be no need to have an alpha build lying around for creating incremental stable .mar files. Nor should there be a need for updating alpha related XML files in this case.

Child Tickets

Attachments (3)

0001-Bug-13587-Updates-per-channel.patch (2.2 KB) - added by gk 5 years ago.
0001-Bug-13857-Updates-per-channel.patch (2.2 KB) - added by gk 5 years ago.
Now with the correct bug number
0001-release-process-generate-update-responses.patch (1.2 KB) - added by boklm 5 years ago.
release process: generate update responses before uploading them

Download all attachments as: .zip

Change History (26)

comment:1 Changed 5 years ago by boklm

Owner: changed from erinn to boklm
Status: newassigned

Ok. I will be looking at this.

Currently, all the xml files are stored in the same directory, with a single .htaccess file for all channels redirecting to the corect xml file. The changes needed in the update_responses script will be the following:

  • creating a separate directory containing xml files and an .htaccess for each channel
  • adding a command line flag to select which channel to update / generate incremental mars for

comment:2 Changed 5 years ago by mcs

Cc: brade mcs added

comment:3 Changed 5 years ago by mcs

How will this change affect the browser's update client? Do you plan to use an additional .htaccess file to redirect update URL requests to the correct location based on channel?

comment:4 in reply to:  3 ; Changed 5 years ago by boklm

Replying to mcs:

How will this change affect the browser's update client? Do you plan to use an additional .htaccess file to redirect update URL requests to the correct location based on channel?

This should not affect the browser's update client. The content returned by all URLs should be the same as before.

Instead of having one .htaccess that redirect based on channel, OS, version and language, we would have one separate .htaccess for each channel, in a sub-directory, which redirects based on OS, version and language. The matching of the channel would be done with the directory name.

comment:5 in reply to:  4 Changed 5 years ago by mcs

Replying to boklm:

This should not affect the browser's update client. The content returned by all URLs should be the same as before.

Instead of having one .htaccess that redirect based on channel, OS, version and language, we would have one separate .htaccess for each channel, in a sub-directory, which redirects based on OS, version and language. The matching of the channel would be done with the directory name.

Thanks for the clarification. I should have looked at our app.update.url format before I posted my question; for some reason I forgot that %CHANNEL%/ (with trailing /) was in there in such a convenient way.

comment:6 Changed 5 years ago by boklm

I made a first version of some patches in branch bug13857-v1 on my user repository:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=bug13857-v1

It is based on branch brade/bug13379-03 as it changes the same parts, so will probably need to wait for bug 13379 to be merged.

comment:7 Changed 5 years ago by boklm

Keywords: TorBrowserTeam201412 added

comment:8 in reply to:  6 Changed 5 years ago by mcs

Replying to boklm:

I made a first version of some patches in branch bug13857-v1 on my user repository:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=bug13857-v1

It is based on branch brade/bug13379-03 as it changes the same parts, so will probably need to wait for bug 13379 to be merged.

Kathy and I reviewed the changes and they look OK.

comment:9 in reply to:  6 ; Changed 5 years ago by gk

Replying to boklm:

I made a first version of some patches in branch bug13857-v1 on my user repository:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=bug13857-v1

Did you test that everything still works if we replace the current mechanism (I don't have the means to do that easily myself)?

comment:10 in reply to:  9 Changed 5 years ago by boklm

I created a bug13857-v2 branch which contains the same patches rebased on current master:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=bug13857-v2

It also updates the README to list the new makefile rules.

To make it easier to test those changes I also added a 'check_update_responses_deployement' command:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug13857-v2&id=ca6ca42b8a78f5dd6b0cad7db0f8416e3a419332

This loads a few URLs and check that it returns the expected version, includes incremental mars, or returns no update if we already have the latest version.

It requires installing 2 perl modules: XML::LibXML and LWP (libxml-libxml-perl and libwww-perl packages on Debian/Ubuntu).

After generating the xml responses using 'make update_responses update_responses-alpha' and copying the htdocs directory to a test web server, I used the 'check_update_responses_deployement' to check it. The only errors it found are caused by the osx32 -> osx64 migration:

  • on the release channel, errors on the Darwin_x86_64-gcc3 URLs, which is normal since we don't have yet a release for this architecture on this channel
  • on the alpha channel, no incremental mars on Darwin_x86-gcc3, which is normal since we don't generate the Darwin_x86-gcc3 -> Darwin_x86_64-gcc3 incrementals

So it seems that things are working as expected.

comment:11 Changed 5 years ago by mikeperry

Keywords: TorBrowserTeam201501 added; TorBrowserTeam201412 removed

comment:12 Changed 5 years ago by mikeperry

Keywords: TorBrowserTeam201501R added; TorBrowserTeam201501 removed
Status: assignedneeds_review

comment:13 Changed 5 years ago by gk

Okay, I merged this with commit 5401e06508eae0aaa8f6357303b006dd833ba8e5 into master.

comment:14 Changed 5 years ago by gk

Attached is the respective release process patch.

comment:15 in reply to:  14 Changed 5 years ago by boklm

Replying to gk:

Attached is the respective release process patch.

I think we need to remove the trailing '/' in the source argument of the rsync command, so that the $TORBROWSER_UPDATE_CHANNEL directory is created on the server.

So the rsync command should be:
torsocks rsync -av htdocs/$TORBROWSER_UPDATE_CHANNEL staticiforme.torproject.org:/srv/dist-master.torproject.org/htdocs/torbrowser/update_2/

comment:16 Changed 5 years ago by gk

Bleh... Updated the patch.

Changed 5 years ago by gk

Now with the correct bug number

comment:17 Changed 5 years ago by mikeperry

Keywords: 4.5-alpha-3 added

comment:18 Changed 5 years ago by mikeperry

Keywords: tbb-4.5-alpha-3 added

comment:19 Changed 5 years ago by mikeperry

Keywords: 4.5-alpha-3 removed

comment:20 in reply to:  16 ; Changed 5 years ago by boklm

Replying to gk:

Bleh... Updated the patch.

The updated patch looks good.

While updating the release process, I have an other small patch that I'm attaching: since #13379, running "make incrementals" does not generate the new update responses, so we need to run "make update_responses" before uploading them.

Changed 5 years ago by boklm

release process: generate update responses before uploading them

comment:21 Changed 5 years ago by mikeperry

Keywords: MikePerry201501R added

comment:22 in reply to:  20 Changed 5 years ago by gk

Replying to boklm:

Replying to gk:

Bleh... Updated the patch.

The updated patch looks good.

While updating the release process, I have an other small patch that I'm attaching: since #13379, running "make incrementals" does not generate the new update responses, so we need to run "make update_responses" before uploading them.

Good catch. You are missing a cd ../../ or something, right?

comment:23 Changed 5 years ago by mikeperry

Resolution: fixed
Status: needs_reviewclosed

Ok, I merged these release process updates and also fixed the ../../ path issue gk pointed out.

Note: See TracTickets for help on using tickets.