Write regression tests for new NoScript options

Giorgio recently introduced three NoScript options just for us: noscript.cascadePermissions, noscript.restrictSubdocScripting, and noscript.globalHttpsWhitelist.

We intend to use these prefs to make it easier for people to use the security slider. The first two cause sub-scripts to be allowed on top-level sites for which the user allows scripts and blocked on top-level urls where scripting is blocked, and the third pref should allow HTTPS sub-scripts to run if and only if the url bar is also HTTPS.

Because we're the only people widely using these prefs, we should write regression tests to ensure this functionality does not break in future NoScript releases.

I am most concerned about the globalHTTPSWhitelist option, as I've already noticed some bugs. The cases we should test include:

1. Do

added component::applications/quality assurance and testing owner::boklm parent::9387 priority::medium resolution::fixed status::closed tbb-4.5-alpha tbb-testcase type::task labels

Trac:
Keywords: N/A deleted, TorBrowserTeam201410 added

Trac:
Keywords: TorBrowserTeam201410 deleted, TorBrowserTeam201411 added

Trac:
Cc: N/A to gk

Trac:
Keywords: TorBrowserTeam201411 deleted, TorBrowserTeam201412 added

Trac:
Keywords: TorBrowserTeam201412 deleted, TorBrowserTeam201501 added

Trac:
Keywords: TorBrowserTeam201501 deleted, N/A added

Trac:
Keywords: N/A deleted, tbb-4.5-alpha added

I started adding some pages to test the globalHTTPSWhitelist option.

http and https pages that source an http url: (1) https://test-data.tbb.torproject.org/noscript/http_src.html (2) http://test-data.tbb.torproject.org/noscript/http_src.html

http and https pages that source an https url: (3) http://test-data.tbb.torproject.org/noscript/https_src.html (4) https://test-data.tbb.torproject.org/noscript/https_src.html

An http iframe in http and https pages: (5) http://test-data.tbb.torproject.org/noscript/http_iframe.html (6) https://test-data.tbb.torproject.org/noscript/http_iframe.html

An https iframe in http and https pages: (7) http://test-data.tbb.torproject.org/noscript/https_iframe.html (8) https://test-data.tbb.torproject.org/noscript/https_iframe.html

When noscript.global is set to false, noscript.cascadePermissions to false and noscript.globalHttpsWhitelist to true, this seems to work as expected: 1, 2, 3, 5, 6, 7 are blocked, but 4 and 8 are not.

However, if noscript.cascadePermissions is set to true, then 1 and 6 are no longer blocked: an https page can source http URLs. I don't think it's the expected behavior.

I think I've found the problem. The cascadePermissions setting is basically overriding the globalHttpsWhitelist one. I'll write Giorgio an email.

So we want cascadePermission to be effective only if the top site has been explicitly whitelisted, rather than allowed to run JavaScript by other implicit policies (such as globalHttpsWhitelist), correct?

ma1 - yes, that's correct. We want cascading permissions to apply only to explicitly whitelisted top sites.

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

Replying to ma1:

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

That's better but the iframe test is still failing: If we have globalHTTPSWhitelist set and we don't have whitelisted top sites then it should not be possible that scripts loaded via HTTP in an iframe get exeucted.

Replying to gk:

Replying to ma1:

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

That's better but the iframe test is still failing: If we have globalHTTPSWhitelist set and we don't have whitelisted top sites then it should not be possible that scripts loaded via HTTP in an iframe get exeucted.

Checking and fixing it, thanks. Out of curiosity, which Gecko version are you testing against, which does not block mixed content by default (I had to disable mixed content blocking in order to make your test case work)?

Replying to ma1:

Replying to gk:

Replying to ma1:

Please check 2.6.9.17rc1, https://noscript.net/getit#devel

That's better but the iframe test is still failing: If we have globalHTTPSWhitelist set and we don't have whitelisted top sites then it should not be possible that scripts loaded via HTTP in an iframe get exeucted.

Checking and fixing it, thanks. Out of curiosity, which Gecko version are you testing against, which does not block mixed content by default (I had to disable mixed content blocking in order to make your test case work)?

I am just testing with the Tor Browser which is based on ESR 31 but has mixed content blocking disabled until Mozilla has a saner implementation at least.

Please check 2.6.9.17rc2 from ​https://noscript.net/getit#devel thank you.

Replying to ma1:

Please check 2.6.9.17rc2 from ​https://noscript.net/getit#devel thank you.

Looks good now, thanks.

boklm: Could you please covert the tests you wrote in a way they fit into our automated testing infrastructure (Mozmill tests should work fine given a proper TLS certificate, e.g.)?

Replying to gk:

boklm: Could you please covert the tests you wrote in a way they fit into our automated testing infrastructure (Mozmill tests should work fine given a proper TLS certificate, e.g.)?

I added a mozmill test checking URLs 1, 2, 3, 4: https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/tree/mozmill-tests/tbb-tests/noscript.js

I still need to add checks for 5, 6, 7, 8.

To avoid the self-signed certificate warning, I added a cert_override.txt file: https://gitweb.torproject.org/boklm/tor-browser-bundle-testsuite.git/tree/data/cert_override.txt

boklm: Could you add a test for #15225 (moved) as well, please?

Replying to boklm:

I still need to add checks for 5, 6, 7, 8.

Tests for 5, 6, 7, 8 added.

Replying to gk:

boklm: Could you add a test for #15225 (moved) as well, please?

I opened #15298 (moved) to get a new hostname to be able to test this.

I get a

ERROR | Test Failure | {
  "exception": {
    "message": "f.getNode(...) is null", 
    "lineNumber": 42, 
    "name": "TypeError", 
    "fileName": "resource://mozmill/modules/frame.js -> file:///path/to/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js"
  }
}

I wonder how this is working for you?

And why do you need an additional hostname to test #15225 (moved)? I mean, is a hostname and an IP address not enough?

Trac:
Status: new to needs_information

The error "f.getNode(...) is null" probably means that javascript was not executed in the page when it should have. I will update the error to make that more clear.

Which version or Tor Browser and noscript did you use for your test ?

If I test a 4.5a4 bundle I get two errors about "http src in https page" and "http iframe in https page".

If I test a 4.5a4 bundle in which I manually installed noscript 2.6.9.18rc3, I get no error.

Replying to gk:

And why do you need an additional hostname to test #15225 (moved)? I mean, is a hostname and an IP address not enough?

Ok, I have done that with this URL: https://test-data.tbb.torproject.org/noscript/alternate_https_src.html and added it to the mozmill test.

The test fails with noscript 2.6.9.17rc2 but work with noscript 2.6.9.18rc3 (as expected).

Replying to boklm:

The error "f.getNode(...) is null" probably means that javascript was not executed in the page when it should have. I will update the error to make that more clear.

Which version or Tor Browser and noscript did you use for your test ?

4.5a3.

If I test a 4.5a4 bundle I get two errors about "http src in https page" and "http iframe in https page".

I am on commit 7c9eb3c52b009de4625cacd7a4a832b56fc919c3. If I take a default en-US bundle and start the test with ./tbb-testsuite --enable-tests=noscript tor-browser-linux64-4.5a4_en-US.tar.xz I get:

ERROR | Test Failure | {
  "fail": {
    "message": "https src in https page", 
    "fileName": "file:///path/to/your/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js", 
    "name": "testNoscript", 
    "lineNumber": 50
  }
}
ERROR | Test Failure | {
  "fail": {
    "message": "https src in https page (alternate hostname)", 
    "fileName": "file:///path/to/your/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js", 
    "name": "testNoscript", 
    "lineNumber": 60
  }
}
ERROR | Test Failure | {
  "exception": {
    "message": "frame.getNode(...) is null", 
    "lineNumber": 69, 
    "name": "TypeError", 
    "fileName": "resource://mozmill/modules/frame.js -> file:///path/to/your/TBB/tor-browser-bundle-testsuite/mozmill-tests/tbb-tests/noscript.js"
  }
}

If I test a 4.5a4 bundle in which I manually installed noscript 2.6.9.18rc3, I get no error.

I get the same errors as above if I install NoScript 2.6.9.18.

Replying to gk:

I am on commit 7c9eb3c52b009de4625cacd7a4a832b56fc919c3. If I take a default en-US bundle and start the test with ./tbb-testsuite --enable-tests=noscript tor-browser-linux64-4.5a4_en-US.tar.xz I get:

This test requires a working tor daemon, so you might want to add "--tor-control-port=9151 --tor-socks-port=9150" to the command line to use an existing tor daemon. Or use "--enable-tests=tor_bootstrap,noscript" to start one before the noscript test. You can also add "--no-xdummy" to display the browser (and see if it shows a connection error page).

Replying to boklm:

Replying to gk:

I am on commit 7c9eb3c52b009de4625cacd7a4a832b56fc919c3. If I take a default en-US bundle and start the test with ./tbb-testsuite --enable-tests=noscript tor-browser-linux64-4.5a4_en-US.tar.xz I get:

This test requires a working tor daemon, so you might want to add "--tor-control-port=9151 --tor-socks-port=9150" to the command line to use an existing tor daemon.

Ok. But why are the other tests not failing then if the test requires a working tor daemon? Should not all tests (even outside of noscript.js) that require tor fail if it is not working in order to be sure that we have a proper test environment at all in order to do the tests we want?

Or use "--enable-tests=tor_bootstrap,noscript" to start one before the noscript test.

That is not working for me. Just the first test is running and that's it.

Anyway, this ticket seems fixed then.

Trac:
Status: needs_information to closed
Resolution: N/A to fixed

closed

mentioned in issue #17570 (moved)

moved to tpo/applications/tor-browser-bundle-testsuite#13053 (closed)