Skip to content
Snippets Groups Projects
New look: Off

Update Orfox HTTPS-E Add-on

    • Closed (moved) Issue created by Matthew Finkel @sysrqb

    Orfox bundles NoScript and HTTPS-Everywhere as distribution extensions. The HTTPS-E version is old. Orfox contains NoScript Anywhere (NSA) 3.5a15, but it seems like this is still the latest version. We'll start using NoScript when we start using the WebExtensions version.

    Orfox uses HTTPS-Everywhere 5.2.20, so (at a minumum) we should bump this to 5.2.21 [0] (as gk mentioned in #19675 (moved) [1]). We should also test the WebExtensions version and decide if we should simply upgrade to version 2018.3.13 (like Desktop).

    I think we should continue using the Orfox distribution, and we can move to a better solution during the rebase (and when we move to rbm).

    Orfox includes the tor-browser-settings [2] add-on, too, but that hasn't changed.

    [0] https://addons.mozilla.org/en-US/firefox/addon/https-everywhere/versions/?page=1#version-5.2.21 [1] https://trac.torproject.org/projects/tor/ticket/19675#comment:20 [2] https://git.synz.io/synzvato/tor-browser-settings.git

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • Matthew Finkel added TorBrowserTeam201805R component::applications/tor browser owner::sysrqb priority::very high resolution::fixed severity::normal status::closed tbb-mobile type::enhancement labels

      added TorBrowserTeam201805R component::applications/tor browser owner::sysrqb priority::very high resolution::fixed severity::normal status::closed tbb-mobile type::enhancement labels

    • Georg Koppen
      Georg Koppen @gk ·

      Trac:
      Keywords: N/A deleted, TorBrowserTeam201803 added

    • Georg Koppen
      Georg Koppen @gk ·

      Moving our tickets to April.

      Trac:
      Keywords: TorBrowserTeam201803 deleted, TorBrowserTeam201804 added

    • Georg Koppen
      Georg Koppen @gk ·

      Trac:
      Priority: High to Very High

    • S
      Matthew Finkel @sysrqb ·
      Author

      Unfortunately this is not working as expected.

      In addition to #25659 (moved), during testing I am seeing https-everywhere successfully load, but then it doesn't rewrite any URLs. I also tested using an older version of Orfox (based on 52.2.0), and it bundles https-everywhere 5.2.20 (tagged on 05 Jul, 2017). This combination of https-everywhere+Orfox works as expected, and the URLs are rewritten. If I leave Orfox open for some time, https-everywhere automatically updates to version 2018.4.11 (webextension). After restarting the app, I see the add-on loads, but it doesn't perform any URL rewrites. I see some messages in the log about an error processing the add-on's manifest, but it seems like the extension is loaded successfully.

      I/Gecko   (24006): 1523904137200        addons.xpi      DEBUG   getModTime: Recursive scan of https-everywhere-eff@eff.org
I/Gecko   (24006): 1523904137200        DeferredSave.extensions.json    DEBUG   Starting timer
I/Gecko   (24006): 1523904137500        DeferredSave.extensions.json    DEBUG   Starting write
I/Gecko   (24006): 1523904137700        addons.webextension.https-everywhere-eff@eff.org        WARN    Loading extension 'https-everywhere-eff@eff.org': Reading manifest: Error processing permissions.3: Unknown permission "tabs"
I/Gecko   (24006): 1523904137700        addons.webextension.https-everywhere-eff@eff.org        WARN    Loading extension 'https-everywhere-eff@eff.org': Reading manifest: Error processing browser_action: An unexpected property was found in the WebExtension manifest.
I/Gecko   (24006): 1523904137700        addons.webextension.https-everywhere-eff@eff.org        WARN    Loading extension 'https-everywhere-eff@eff.org': Reading manifest: Error processing devtools_page: An unexpected property was found in the WebExtension manifest.
I/Gecko   (24006): 1523904137700        DeferredSave.extensions.json    DEBUG   Write succeeded

      I see these same messages when I bundle https-everywhere 2018.03.13.

      Next, I'll try bundling https-e 5.2.20 with the newest Orfox version, because I'm curious if that reliably loads and if it rewrites URLs. If we can't get https-everywhere loading or working reliably, I'd rather remove it from Orfox in the next release. I don't want to give the false impression https-everywhere is loaded and working - when it isn't working in reality.

      Trac:
      Owner: tbb-team to sysrqb
      Status: new to assigned

    • S
      Matthew Finkel @sysrqb ·
      Author

      For reference, I am specifically testing whether loading http://deb.torproject.org is rewritten to https://deb.torproject.org. This does not happen automatically in Firefox because it is not forced on the server-side, but it does happen in Tor Browser. I also tested in Tor Browser with https-everywhere disabled, and the request is not redirected.

      Also, I see similar (but not the same) load errors occur on the desktop, too.

      1523974564000	addons.webextension.{73a6fe31-595d-460b-a920-fcc0f8843232}	WARN	Loading extension '{73a6fe31-595d-460b-a920-fcc0f8843232}': Reading manifest: Error processing permissions.1: Unknown permission "privacy"
1523974564000	addons.webextension.{73a6fe31-595d-460b-a920-fcc0f8843232}	WARN	Loading extension '{73a6fe31-595d-460b-a920-fcc0f8843232}': Reading manifest: Error processing permissions.4: Unknown permission "unlimitedStorage"
Apr 17 14:16:04.000 [notice] Bootstrapped 85%: Finishing handshake with first hop
1523974564400	addons.webextension.https-everywhere-eff@eff.org	WARN	Loading extension 'https-everywhere-eff@eff.org': Reading manifest: Error processing devtools_page: An unexpected property was found in the WebExtension manifest.

      Both desktop and Android use the same extensions code for parsing and loading, so I'm not sure why there is a difference.

    • Georg Koppen
      Georg Koppen @gk ·

      Does HTTPS-E work for you in a vanilla Firefox for Mobile?

      Trac:
      Cc: N/A to gk

    • S
      Matthew Finkel @sysrqb ·
      Author

      Replying to gk:

      Does HTTPS-E work for you in a vanilla Firefox for Mobile?

      Yes, 2018.4.11 works with Fennec 59.0.2. I don't have a vanilla Fennec 52.7.3 already built, so I'll need to build that and test this.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Replying to sysrqb:

      Replying to gk:

      Does HTTPS-E work for you in a vanilla Firefox for Mobile?

      Yes, 2018.4.11 works with Fennec 59.0.2. I don't have a vanilla Fennec 52.7.3 already built, so I'll need to build that and test this.

      It doesn't work with a fresh Fennec 52.7.3. I installed https-everywhere 2018.4.11 from AMO, and it does not rewrite the URL. I'll focus on testing Orfox with https-everywhere 5.2.21. Maybe we can ship it with an updated ruleset.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Oh, and note-to-self. We should disable auto-update for this extension, else Orfox will download the new version and it won't rewrite any URLs.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Process for creating new https-everywhere xpi:

      1. Checkout tag 2018.4.11
      2. Copy src/chrome/content/rules into another directory outside working directory (cp -r src/chrome/content/rules ../)
      3. Checkout tag 5.2.21
      4. Edit src/install.rdf - clear updateURL and updateKey [0] (this prevents auto-updating)
      5. Delete directory src/META-INF/ (rm -r src/META-INF)
      6. Delete all current rules (rm -r src/chrome/content/rules/)
      7. Copy new rules into working directory (cp -r ../rules src/chrome/content/)
      8. ./makexpi.sh 9 sha256sum pkg/https-everywhere-5.2.21~4c7803208b-dirty-eff.xpi is
      7a33c13dbd80fd881b1508fca6dc10fca787f8eb4da754104321537240ffb866  pkg/https-everywhere-5.2.21~4c7803208b-dirty-eff.xpi
      1. Copy pkg/https-everywhere-5.2.21~4c7803208b-dirty-eff.xpi into tor-browser/mobile/android/orfox/distribution/assets/distribution/extensions/https-everywhere-eff@eff.org.xpi

      [0] Diff from step (4)

      diff --git a/src/install.rdf b/src/install.rdf
index fe321f48b0..6fa8ece9c7 100644
--- a/src/install.rdf
+++ b/src/install.rdf
@@ -15,8 +15,8 @@
         <em:optionsURL>chrome://https-everywhere/content/observatory-preferences.xul</em:optionsURL>
         <em:iconURL>chrome://https-everywhere/skin/icon-active-48.png</em:iconURL>
         <em:unpack>false</em:unpack>
-        <em:updateURL>https://www.eff.org/files/https-everywhere-eff-update-2048.rdf</em:updateURL> <!-- 2015-08-14: New update URL to go with new id (https-everywhere-eff@ef.org) -->
-        <em:updateKey>MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA6MR8W/galdxnpGqBsYbqOzQb2eyW15YFjDDEMI0ZOzt8f504obNs920lDnpPD2/KqgsfjOgw2K7xWDJIj/18xUvWPk3LDkrnokNiRkA3KOx3W6fHycKL+zID7zy+xZYBuh2fLyQtWV1VGQ45iNRp9+Zo7rH86cdfgkdnWTlNSHyTLW9NbXvyv/E12bppPcEvgCTAQXgnDVJ0/sqmeiijn9tTFh03aM+R2V/21h8aTraAS24qiPCz6gkmYGC8yr6mglcnNoYbsLNYZ69zF1XHcXPduCPdPdfLlzVlKK1/U7hkA28eG3BIAMh6uJYBRJTpiGgaGdPd7YekUB8S6cy+CQIDAQAB</em:updateKey>
+        <em:updateURL></em:updateURL> <!-- 2015-08-14: New update URL to go with new id (https-everywhere-eff@ef.org) Deleted for Orfox -->
+        <em:updateKey></em:updateKey> <!-- Deleted for Orfox -->
         <!-- Firefox -->
         <em:targetApplication>
             <Description>
    • S
      Matthew Finkel @sysrqb ·
      Author

      After #25980 (moved) is merged, I have a branch ready for this.

      The top commit on branch bug25980+bug25603 is the update, the branch is based on the fix for #25980 (moved).

      I wish we had a better way of providing the source for the bundled https-everywhere xpi in Orfox. I can create a public repo somewhere. The steps documented in comment:10 are exactly how I created this xpi. The most important step is deleting the updateURL (but leaving the em:updateURL tags). We must delete the manifest because the signature is no longer valid. I know, this is sad.

      ​https://git.torproject.org/user/sysrqb/tor-browser.git

      Trac:
      Status: assigned to needs_review
      Cc: gk to gk, igt0

    • Georg Koppen
      Georg Koppen @gk ·

      Trac:
      Keywords: TorBrowserTeam201804 deleted, TorBrowserTeam201805R added

    • Georg Koppen
      Georg Koppen @gk ·

      Replying to sysrqb:

      After #25980 (moved) is merged, I have a branch ready for this.

      The top commit on branch bug25980+bug25603 is the update, the branch is based on the fix for #25980 (moved).

      I wish we had a better way of providing the source for the bundled https-everywhere xpi in Orfox. I can create a public repo somewhere. The steps documented in comment:10 are exactly how I created this xpi. The most important step is deleting the updateURL (but leaving the em:updateURL tags). We must delete the manifest because the signature is no longer valid. I know, this is sad.

      I think we should not spend our time fixing this problem as this is supposed to go away with transitioning to Tor Browser for Android.

      The patch looks good. I cherry-picked it onto tor-browser-52.7.3esr-8.0-1 (commit e3effb933616512282e0047d56ed0087cce702a6).

      Trac:
      Status: needs_review to closed
      Resolution: N/A to fixed

    • Trac closed

      closed

    • Trac moved to tpo/applications/tor-browser#25603 (closed)

      moved to tpo/applications/tor-browser#25603 (closed)

    Please register or sign in to reply