Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#23258 closed defect (fixed)

HTTPS Everywhere is not working when noscript is enabled globally

Reported by: Dbryrtfbcbhgf Owned by: tbb-team
Priority: High Milestone:
Component: Applications/Tor Browser Version:
Severity: Major Keywords: TorBrowserTeam201708R
Cc: legind Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

HTTPS Everywhere auto updated and now there is a new icon in the toolbar, clicking on the new icon shows a blank popup with 2 checkboxes.
TorBrowser 7.0.4
Https everywhere 2017.8.15

Child Tickets

Attachments (3)

bug.png (198.1 KB) - added by Dbryrtfbcbhgf 2 years ago.
Image showing bug
23258-7.5a4.png (16.6 KB) - added by dcf 2 years ago.
bug.2.png (51.8 KB) - added by Dbryrtfbcbhgf 2 years ago.

Download all attachments as: .zip

Change History (45)

comment:1 Changed 2 years ago by Dbryrtfbcbhgf

HTTPS everywhere changelog 2017.8.15

  • Incorporating numerous fixes for WebExtensions to work within Firefox
  • Firefox: Creating Embedded WebExtension wrapper to migrate legacy settings
  • Removing legacy XPCOM codebase, unifying Firefox and Chrome codebase
  • Ruleset updates

https://www.eff.org/files/Changelog.txt

Changed 2 years ago by Dbryrtfbcbhgf

Attachment: bug.png added

Image showing bug

Changed 2 years ago by dcf

Attachment: 23258-7.5a4.png added

comment:2 Changed 2 years ago by dcf

I guess I have the same on GNU/Linux with 7.5a4. I don't remember if the icon was on the toolbar before or not. In my case, the panel isn't blank, unlike Dbryrtfbcbhgf's bug.png.


comment:3 Changed 2 years ago by gk

Component: Applications/Tor BrowserHTTPS Everywhere/EFF-HTTPS Everywhere
Owner: changed from tbb-team to jsha
Status: newneeds_information

The new icon should be as expected but the blank one not. legind: Do you track this bug somewhere?

comment:4 Changed 2 years ago by gk

Owner: changed from jsha to legind
Status: needs_informationassigned

comment:5 Changed 2 years ago by gk

Status: assignedneeds_information

comment:6 in reply to:  2 Changed 2 years ago by cypherpunks

Replying to dcf:

I guess I have the same on GNU/Linux with 7.5a4. I don't remember if the icon was on the toolbar before or not. In my case, the panel isn't blank, unlike Dbryrtfbcbhgf's bug.png.

This only happens when the security level is set to high or medium, only the low level shows the panel correctly.

Last edited 2 years ago by cypherpunks (previous) (diff)

comment:7 Changed 2 years ago by cypherpunks

Additionally HTTPS Everywhere has stopped redirecting pages to HTTPS according to the rules. For example, http://imgur.com/ used to redirect to HTTPS, now it does not.

comment:8 in reply to:  7 Changed 2 years ago by Lolint

Replying to cypherpunks:

Additionally HTTPS Everywhere has stopped redirecting pages to HTTPS according to the rules. For example, http://imgur.com/ used to redirect to HTTPS, now it does not.

I confirm, even at Low security setting redirects don't happen.

Also rulesets aren't shown when clicking on the HTTPS Everywhere icon, so we can't disable some, unlike with other versions.

Last edited 2 years ago by Lolint (previous) (diff)

comment:9 Changed 2 years ago by mcs

Cc: tbb-team added

comment:10 in reply to:  7 Changed 2 years ago by boklm

Replying to cypherpunks:

Additionally HTTPS Everywhere has stopped redirecting pages to HTTPS according to the rules. For example, http://imgur.com/ used to redirect to HTTPS, now it does not.

I can reproduce this problem (and the blank popup too) when security level is set to High or Medium. It is however still working when using the Low security level (after restarting the browser when changing the level).

comment:11 Changed 2 years ago by boklm

It seems the problem is related to noscript. Setting noscript.global to false and restarting the browser is enough to cause HTTPS Everywhere to stop working.

comment:12 Changed 2 years ago by boklm

Summary: HTTPS Everywhere auto updated and now there is a new icon in the toolbarHTTPS Everywhere is not working when noscript is enabled globally

comment:13 Changed 2 years ago by cypherpunks

Once again the developers messed the "stable" series without testing, even when they had been warned https://blog.torproject.org/comment/270389#comment-270389. This is unprofessional.

comment:14 in reply to:  2 Changed 2 years ago by dcf

Replying to dcf:

I guess I have the same on GNU/Linux with 7.5a4. I don't remember if the icon was on the toolbar before or not. In my case, the panel isn't blank, unlike Dbryrtfbcbhgf's bug.png.

comment:6, comment:7, comment:10 are correct as far as a I can tell. Yesterday I was on the Low security setting and the panel was not blank. Today I am on the Medium security level and the panel is blank. And typing "web.archive.org" into the URL bar does not automatically take me to https://web.archive.org/.

comment:15 Changed 2 years ago by gk

Cc: legind added; tbb-team removed
Component: HTTPS Everywhere/EFF-HTTPS EverywhereApplications/Tor Browser
Keywords: TorBrowserTeam201708 added
Owner: changed from legind to tbb-team
Status: needs_informationassigned

I guess what we can and maybe even should do is whitelist moz-extension: (see #21270). That solves the problem for me. However, it causes a startup delay of several seconds on my Linux debug build. So, hrm.

comment:16 in reply to:  15 ; Changed 2 years ago by gk

Replying to gk:

I guess what we can and maybe even should do is whitelist moz-extension: (see #21270). That solves the problem for me. However, it causes a startup delay of several seconds on my Linux debug build. So, hrm.

So, the startup delay is not related to the whitelisting but to the new HTTPS-E itself. :/

comment:17 Changed 2 years ago by yawning

comment:18 Changed 2 years ago by legind

@cypherpunks again, your comments are extremely unhelpful.

This is related to https://github.com/EFForg/https-everywhere/issues/11948#issuecomment-323147599.

comment:19 in reply to:  16 Changed 2 years ago by yawning

Replying to gk:

Replying to gk:

I guess what we can and maybe even should do is whitelist moz-extension: (see #21270). That solves the problem for me. However, it causes a startup delay of several seconds on my Linux debug build. So, hrm.

So, the startup delay is not related to the whitelisting but to the new HTTPS-E itself. :/

That's unfortunate especially for people that use amnesiac setups like Tails or the experimental sandbox option. What is HTTPSE doing during the startup delay? Is it something that can be accelerated by pregenerating things during the build process?

As far as I am aware, the plan is to disable automated updates (that aren't done via the Tor Browser update process, as in, via a MAR), so the delay could be eliminated...

comment:20 Changed 2 years ago by legind

I've identified the issue with cookies being disabled completely and have a fix for it, but this doesn't seem to solve the problem for Tor Browser on medium or high security settings, and I'm getting no useful output from the browser console.

comment:21 Changed 2 years ago by legind

Testing between Firefox 52 (ESR) and Firefox 55.

When JavaScript is disabled in 55, the extension behaves properly upon browser load.
When JavaScript is disabled in 52 ESR, the extension does not redirect URLs and the popup is loaded with blank fields.

So this seems to be a problem Firefox fixed between the two versions. I wonder if we can get this backported to ESR. I'm still not sure what the *exact* problem is, since Firefox does not throw any errors in the debugger.

comment:22 in reply to:  21 Changed 2 years ago by mcs

Replying to legind:

So this seems to be a problem Firefox fixed between the two versions. I wonder if we can get this backported to ESR. I'm still not sure what the *exact* problem is, since Firefox does not throw any errors in the debugger.

I do not have time to investigate further right now, but a quick bugzilla search turned up this possibility:

"Disabling JavaScript breaks Web Extensions"
https://bugzilla.mozilla.org/show_bug.cgi?id=1329731

comment:23 Changed 2 years ago by legind

Thanks @mcs, I'm guessing that's the issue. Can we make a new Tor Browser release, applying this patch? This is not something I can fix in HTTPS Everywhere.

comment:24 Changed 2 years ago by legind

I've just made a new release - use HTTPS Everywhere 2017.8.17 for new builds.

comment:25 Changed 2 years ago by legind

If backporting the fix in https://bugzilla.mozilla.org/show_bug.cgi?id=1329731 into Tor Browser isn't viable, you could keep the current build version 5.2.21 and disable the XPCOM update channel, which should be achievable via

sed -i 's/https:\/\/www.eff.org\/files\/https-everywhere-eff-update-2048.rddata:text\/plain,/g' src/install.rdf

Note that this will not get ruleset updates, since rulesets are updated with the extension itself.

This is a less than ideal solution since the upgrade to WebExtensions will have to be made in any case, and users who do not upgrade to the Embedded WebExtension (current HTTPS Everywhere) first will miss an important settings migration step in addition to having stale rulesets. Let's try to have Mozilla fix this in ESR.

Last edited 2 years ago by legind (previous) (diff)

comment:26 Changed 2 years ago by legind

For the latest rulesets you could add another line to the build process, either after or before removing the update channel:

git tag -v 2017.8.17 && git checkout 2017.8.17 src/chrome/content/rules/

The rulesets are backwards compatible with the XPCOM version of the extension.

Last edited 2 years ago by legind (previous) (diff)

comment:27 Changed 2 years ago by boklm

I think we can take this patch in the next release if that fix the issue. However the next planned release is on September 26, which is a long time. So we have to see if we want to make a release to fix that before, or if there is an other workaround we could use to avoid a new release.

Changed 2 years ago by Dbryrtfbcbhgf

Attachment: bug.2.png added

comment:28 Changed 2 years ago by Dbryrtfbcbhgf

The preference button on ad-on->extension->click on the blue more button next to https everywhere, is missing and shows a browse button instead. Here is a photo showing the bug. bug.2.png

comment:29 in reply to:  16 ; Changed 2 years ago by cypherpunks

Replying to gk:

So, the startup delay is not related to the whitelisting but to the new HTTPS-E itself. :/

Or maybe you were using Medium or High security settings and there's no JIT whitelist for webextensions just like it was the case with pdfjs?

comment:30 in reply to:  18 Changed 2 years ago by cypherpunks

Replying to legind:
If you really want to fix this, you can make a new release and put refreshed 5.2.21 in it, then postpone updates from EFF for TBB channels like it was before (ticket:10394#comment:7 if it's still available), and then make a new release with the latest updates for all clients which unaffected.

Last edited 2 years ago by cypherpunks (previous) (diff)

comment:31 in reply to:  29 Changed 2 years ago by gk

Replying to cypherpunks:

Replying to gk:

So, the startup delay is not related to the whitelisting but to the new HTTPS-E itself. :/

Or maybe you were using Medium or High security settings and there's no JIT whitelist for webextensions just like it was the case with pdfjs?

No, that's on level "low". Actually, getting NoScript to work on higher security slider settings shows the same effect. Thus, that's very likely due to the new HTTPS-Everywhere.

comment:32 Changed 2 years ago by boklm

#23290 is a duplicate.

comment:33 Changed 2 years ago by legind

With NoScript, much of the code is still in the XPCOM wrapper. That's why it continues to work.

On all pure WebExtensions I've tested in ESR with JavaScript disabled, I've seen the same result. I've tested with uBlock Origin, Disconnect, Ghostery, Privacy Badger. All the same: no functionality, unresponsive.

As I've mentioned the way to fix this is to pick one of the following:

  1. Backport the Firefox 54 fix to Tor Browser
  2. Wait for Firefox to backport the fix, which given their responsiveness lately I doubt will be very effective
  3. Make a new TB release with 5.2.21, removing the update channel and using the new rulesets in 2017.8.19.

Option 3 is out of my hands, but I've provided build instructions above. Of course I'm willing to help if you run into any issues.

comment:34 in reply to:  33 Changed 2 years ago by gk

Replying to legind:

With NoScript, much of the code is still in the XPCOM wrapper. That's why it continues to work.

On all pure WebExtensions I've tested in ESR with JavaScript disabled, I've seen the same result. I've tested with uBlock Origin, Disconnect, Ghostery, Privacy Badger. All the same: no functionality, unresponsive.

As I've mentioned the way to fix this is to pick one of the following:

  1. Backport the Firefox 54 fix to Tor Browser
  2. Wait for Firefox to backport the fix, which given their responsiveness lately I doubt will be very effective
  3. Make a new TB release with 5.2.21, removing the update channel and using the new rulesets in 2017.8.19.

Option 3 is out of my hands, but I've provided build instructions above. Of course I'm willing to help if you run into any issues.

Thanks. My current plan is to backport the patch and, after some testing, get a new stable release out (probably next week).

comment:35 Changed 2 years ago by gk

It turns out that the Firefox fix is not working for us as NoScript is blocking things differently. A Tor Browser with the fix for https://bugzilla.mozilla.org/show_bug.cgi?id=1329731 _and_ NoScript disabled works if javascript.enabled is set to false (contrary to a vanilla Firefox 52 ESR). However, once NoScript is enabled we are still stuck.

So I guess we go with an option 4 which I mentioned in comment:15.

comment:36 Changed 2 years ago by gk

Keywords: TorBrowserTeam201708R added; TorBrowserTeam201708 removed
Status: assignedneeds_review

comment:37 in reply to:  36 Changed 2 years ago by cypherpunks

Replying to gk:

Shipping HTTPS-Everywhere based on the new WebExtensions model broke its
functionality on higher security levels as our JavaScript blocking does
not whitelist moz-extension: which is needed.

Shipping NoScript as our JavaScript blocking on higher security levels does not whitelist moz-extension: which is needed and broke functionality of HTTPS-Everywhere based on the new WebExtensions model.

It turns out that vanilla Firefox 52 ESR is affected as well but differently:
Backorting https://bugzilla.mozilla.org/show_bug.cgi?id=1329731 does not help
us as NoScript is regulating JavaScript related things.

Remove as irrelevant.

comment:38 Changed 2 years ago by yawning

Closed #23321 as a duplicate.

comment:39 in reply to:  36 Changed 2 years ago by arthuredelstein

Status: needs_reviewmerge_ready

Replying to gk:

See: https://oniongit.eu/gk/tor-browser-bundle/merge_requests/4 for a patch up for review.

This patch looks good to me.

comment:40 Changed 2 years ago by gk

Resolution: fixed
Status: merge_readyclosed

Thanks. Applied to master and maint-7.0 (commit 32dc8cef283f739bdc045918adbd448c9d78a746 and 35ebe6d5e7f1e33f5cb1b67f0a4022814fd72a15).

cypherpunks: I wanted to keep the Mozilla related part of the commit message as it explains why we did not take that road. Thus, I kept the first part as well.

comment:41 in reply to:  16 Changed 2 years ago by cypherpunks

Replying to gk:

So, the startup delay is not related to the whitelisting but to the new HTTPS-E itself. :/

Confirmed. It creates

│   ├──27.41 MB (19.21%) -- https-everywhere-eff@eff.org
│   │  ├──27.38 MB (19.19%) ++ window-objects/top(moz-extension://06272878-3037-4aa2-bbd3-e0effb78113c/_generated_background_page.html, id=14)

at startup.

comment:42 in reply to:  29 Changed 2 years ago by cypherpunks

Replying to gk:

So, the startup delay is not related to the whitelisting but to the new HTTPS-E itself. :/

They have a couple ideas to resolve a bit that issue, you may want to follow https://github.com/EFForg/https-everywhere/issues/12232

Note: See TracTickets for help on using tickets.