Opened 5 years ago

Closed 4 years ago

#13053 closed task (fixed)

Write regression tests for new NoScript options

Reported by: mikeperry Owned by: boklm
Priority: Medium Milestone:
Component: Applications/Quality Assurance and Testing Version:
Severity: Keywords: tbb-testcase, tbb-4.5-alpha
Cc: gk Actual Points:
Parent ID: #9387 Points:
Reviewer: Sponsor:

Description

Giorgio recently introduced three NoScript options just for us:
noscript.cascadePermissions, noscript.restrictSubdocScripting, and noscript.globalHttpsWhitelist.

We intend to use these prefs to make it easier for people to use the security slider. The first two cause sub-scripts to be allowed on top-level sites for which the user allows scripts and blocked on top-level urls where scripting is blocked, and the third pref should allow HTTPS sub-scripts to run if and only if the url bar is also HTTPS.

Because we're the only people widely using these prefs, we should write regression tests to ensure this functionality does not break in future NoScript releases.

I am most concerned about the globalHTTPSWhitelist option, as I've already noticed some bugs. The cases we should test include:

  1. Do <script> elements that source https urls get blocked from http url bars, no matter what (even if those domains are in the NoScript whitelist)
  2. Does the same happen for iframes?
  3. Is the converse true? If we have an https:// url bar, do script elements to http:// urls for the same domain end up blocked?
  4. And for iframes as well?

Child Tickets

Change History (28)

comment:1 Changed 5 years ago by mikeperry

Keywords: TorBrowserTeam201410 added

comment:2 Changed 5 years ago by mikeperry

Keywords: TorBrowserTeam201411 added; TorBrowserTeam201410 removed

comment:3 Changed 5 years ago by gk

Cc: gk added

comment:4 Changed 4 years ago by mikeperry

Keywords: TorBrowserTeam201412 added; TorBrowserTeam201411 removed

comment:5 Changed 4 years ago by mikeperry

Keywords: TorBrowserTeam201501 added; TorBrowserTeam201412 removed

comment:6 Changed 4 years ago by mikeperry

Keywords: TorBrowserTeam201501 removed

comment:7 Changed 4 years ago by gk

Keywords: tbb-4.5-alpha added

comment:8 Changed 4 years ago by boklm

I started adding some pages to test the globalHTTPSWhitelist option.

http and https pages that source an http url:
(1) https://test-data.tbb.torproject.org/noscript/http_src.html
(2) http://test-data.tbb.torproject.org/noscript/http_src.html

http and https pages that source an https url:
(3) http://test-data.tbb.torproject.org/noscript/https_src.html
(4) https://test-data.tbb.torproject.org/noscript/https_src.html

An http iframe in http and https pages:
(5) http://test-data.tbb.torproject.org/noscript/http_iframe.html
(6) https://test-data.tbb.torproject.org/noscript/http_iframe.html

An https iframe in http and https pages:
(7) http://test-data.tbb.torproject.org/noscript/https_iframe.html
(8) https://test-data.tbb.torproject.org/noscript/https_iframe.html

When noscript.global is set to false, noscript.cascadePermissions to false and noscript.globalHttpsWhitelist to true, this seems to work as expected: 1, 2, 3, 5, 6, 7 are blocked, but 4 and 8 are not.

However, if noscript.cascadePermissions is set to true, then 1 and 6 are no longer blocked: an https page can source http URLs. I don't think it's the expected behavior.

comment:9 Changed 4 years ago by gk

I think I've found the problem. The cascadePermissions setting is basically overriding the globalHttpsWhitelist one. I'll write Giorgio an email.

comment:10 Changed 4 years ago by ma1

So we want cascadePermission to be effective only if the top site has been explicitly whitelisted, rather than allowed to run JavaScript by other implicit policies (such as globalHttpsWhitelist), correct?

comment:11 Changed 4 years ago by mikeperry

ma1 - yes, that's correct. We want cascading permissions to apply only to explicitly whitelisted top sites.

comment:12 Changed 4 years ago by ma1

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

comment:13 in reply to:  12 ; Changed 4 years ago by gk

Replying to ma1:

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

That's better but the iframe test is still failing: If we have globalHTTPSWhitelist set and we don't have whitelisted top sites then it should not be possible that scripts loaded via HTTP in an iframe get exeucted.

comment:14 in reply to:  13 ; Changed 4 years ago by ma1

Replying to gk:

Replying to ma1:

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

That's better but the iframe test is still failing: If we have globalHTTPSWhitelist set and we don't have whitelisted top sites then it should not be possible that scripts loaded via HTTP in an iframe get exeucted.

Checking and fixing it, thanks.
Out of curiosity, which Gecko version are you testing against, which does not block mixed content by default (I had to disable mixed content blocking in order to make your test case work)?

comment:15 in reply to:  14 Changed 4 years ago by gk

Replying to ma1:

Replying to gk:

Replying to ma1:

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

That's better but the iframe test is still failing: If we have globalHTTPSWhitelist set and we don't have whitelisted top sites then it should not be possible that scripts loaded via HTTP in an iframe get exeucted.

Checking and fixing it, thanks.
Out of curiosity, which Gecko version are you testing against, which does not block mixed content by default (I had to disable mixed content blocking in order to make your test case work)?

I am just testing with the Tor Browser which is based on ESR 31 but has mixed content blocking disabled until Mozilla has a saner implementation at least.

comment:16 Changed 4 years ago by ma1

Please check 2.6.9.17rc2 from ​https://noscript.net/getit#devel thank you.

comment:17 in reply to:  16 ; Changed 4 years ago by gk

Replying to ma1:

Please check 2.6.9.17rc2 from ​https://noscript.net/getit#devel thank you.

Looks good now, thanks.

boklm: Could you please covert the tests you wrote in a way they fit into our automated testing infrastructure (Mozmill tests should work fine given a proper TLS certificate, e.g.)?

comment:18 in reply to:  17 ; Changed 4 years ago by boklm

Replying to gk:

boklm: Could you please covert the tests you wrote in a way they fit into our automated testing infrastructure (Mozmill tests should work fine given a proper TLS certificate, e.g.)?

I added a mozmill test checking URLs 1, 2, 3, 4:
https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/tree/mozmill-tests/tbb-tests/noscript.js

I still need to add checks for 5, 6, 7, 8.

To avoid the self-signed certificate warning, I added a cert_override.txt file:
https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/tree/data/cert_override.txt

comment:19 Changed 4 years ago by gk

boklm: Could you add a test for #15225 as well, please?

comment:20 in reply to:  18 Changed 4 years ago by boklm

Replying to boklm:

I still need to add checks for 5, 6, 7, 8.

Tests for 5, 6, 7, 8 added.

comment:21 in reply to:  19 Changed 4 years ago by boklm

Replying to gk:

boklm: Could you add a test for #15225 as well, please?

I opened #15298 to get a new hostname to be able to test this.

comment:22 Changed 4 years ago by gk

I get a

ERROR | Test Failure | {
  "exception": {
    "message": "f.getNode(...) is null", 
    "lineNumber": 42, 
    "name": "TypeError", 
    "fileName": "resource://mozmill/modules/frame.js -> file:///path/to/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js"
  }
}

I wonder how this is working for you?

And why do you need an additional hostname to test #15225? I mean, is a hostname and an IP address not enough?

comment:23 Changed 4 years ago by gk

Status: newneeds_information

comment:24 in reply to:  22 ; Changed 4 years ago by boklm

The error "f.getNode(...) is null" probably means that javascript was not executed in the page when it should have. I will update the error to make that more clear.

Which version or Tor Browser and noscript did you use for your test ?

If I test a 4.5a4 bundle I get two errors about "http src in https page" and "http iframe in https page".

If I test a 4.5a4 bundle in which I manually installed noscript 2.6.9.18rc3, I get no error.

comment:25 in reply to:  22 Changed 4 years ago by boklm

Replying to gk:

And why do you need an additional hostname to test #15225? I mean, is a hostname and an IP address not enough?

Ok, I have done that with this URL:
https://test-data.tbb.torproject.org/noscript/alternate_https_src.html
and added it to the mozmill test.

The test fails with noscript 2.6.9.17rc2 but work with noscript 2.6.9.18rc3 (as expected).

comment:26 in reply to:  24 ; Changed 4 years ago by gk

Replying to boklm:

The error "f.getNode(...) is null" probably means that javascript was not executed in the page when it should have. I will update the error to make that more clear.

Which version or Tor Browser and noscript did you use for your test ?

4.5a3.

If I test a 4.5a4 bundle I get two errors about "http src in https page" and "http iframe in https page".

I am on commit 7c9eb3c52b009de4625cacd7a4a832b56fc919c3. If I take a default en-US bundle and start the test with ./tbb-testsuite --enable-tests=noscript tor-browser-linux64-4.5a4_en-US.tar.xz I get:

ERROR | Test Failure | {
  "fail": {
    "message": "https src in https page", 
    "fileName": "file:///path/to/your/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js", 
    "name": "testNoscript", 
    "lineNumber": 50
  }
}
ERROR | Test Failure | {
  "fail": {
    "message": "https src in https page (alternate hostname)", 
    "fileName": "file:///path/to/your/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js", 
    "name": "testNoscript", 
    "lineNumber": 60
  }
}
ERROR | Test Failure | {
  "exception": {
    "message": "frame.getNode(...) is null", 
    "lineNumber": 69, 
    "name": "TypeError", 
    "fileName": "resource://mozmill/modules/frame.js -> file:///path/to/your/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js"
  }
}

If I test a 4.5a4 bundle in which I manually installed noscript 2.6.9.18rc3, I get no error.

I get the same errors as above if I install NoScript 2.6.9.18.

Last edited 4 years ago by gk (previous) (diff)

comment:27 in reply to:  26 ; Changed 4 years ago by boklm

Replying to gk:

I am on commit 7c9eb3c52b009de4625cacd7a4a832b56fc919c3. If I take a default en-US bundle and start the test with ./tbb-testsuite --enable-tests=noscript tor-browser-linux64-4.5a4_en-US.tar.xz I get:

This test requires a working tor daemon, so you might want to add "--tor-control-port=9151 --tor-socks-port=9150" to the command line to use an existing tor daemon. Or use "--enable-tests=tor_bootstrap,noscript" to start one before the noscript test. You can also add "--no-xdummy" to display the browser (and see if it shows a connection error page).

comment:28 in reply to:  27 Changed 4 years ago by gk

Resolution: fixed
Status: needs_informationclosed

Replying to boklm:

Replying to gk:

I am on commit 7c9eb3c52b009de4625cacd7a4a832b56fc919c3. If I take a default en-US bundle and start the test with ./tbb-testsuite --enable-tests=noscript tor-browser-linux64-4.5a4_en-US.tar.xz I get:

This test requires a working tor daemon, so you might want to add "--tor-control-port=9151 --tor-socks-port=9150" to the command line to use an existing tor daemon.

Ok. But why are the other tests not failing then if the test requires a working tor daemon? Should not all tests (even outside of noscript.js) that require tor fail if it is not working in order to be sure that we have a proper test environment at all in order to do the tests we want?

Or use "--enable-tests=tor_bootstrap,noscript" to start one before the noscript test.

That is not working for me. Just the first test is running and that's it.

Anyway, this ticket seems fixed then.

Note: See TracTickets for help on using tickets.