Skip to content
Snippets Groups Projects
New look: Off

Test from #18912 failing

    • Closed (moved) Issue created by Arthur Edelstein @arthuredelstein

    While preparing to uplift our patch for #18912 (moved), I rebased it to mozilla-central but found one of the mochitests is failing. I then ran the test in the latest tor-browser.git branch (tor-browser-52.6.0esr-8.0-2) and saw the same test failure:

    <div style="font-family: monospace; white-space: pre-wrap;">
0 INFO TEST-START | Shutdown
1 INFO Passed:  30
2 INFO Failed:  1
3 INFO Todo:    0
4 INFO Mode:    non-e10s
5 INFO SimpleTest FINISHED
The following tests failed:
56 INFO TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE - got 2153390067, expected 2153398272
SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:271:5
checkErrorCode@chrome://mochitests/content/chrome/toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul:61:3
defaultCallback@chrome://mochitests/content/chrome/toolkit/mozapps/update/tests/chrome/utils.js:429:5
</div>

    Somehow we're not getting the desired error code.

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items 0

    • No child items are currently assigned. Use child items to break down this issue into smaller parts.

    Activity

    • Arthur Edelstein added TorBrowserTeam201804R component::applications/tor browser owner::tbb-team priority::medium resolution::fixed severity::normal status::closed type::defect labels

      added TorBrowserTeam201804R component::applications/tor browser owner::tbb-team priority::medium resolution::fixed severity::normal status::closed type::defect labels

    • Mark Smith
      Mark Smith @mcs ·

      After poking around for a while with a 8.0a1-ish build of Tor Browser, Kathy and I learned that the test is failing due to #18087 (moved). When we run the test using the command below it works (i.e., the expected error code is generated):

      ./mach mochitest --setpref=security.nocertdb=false toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      Arthur, when you ran the test on mozilla-central was security.nocertdb set to false? I am not sure if the build you are using has the Tor Browser default prefs or not.

      Trac:
      Status: new to needs_information

    • A
      Arthur Edelstein @arthuredelstein ·
      Author

      Thanks for looking into this!

      I ran some tests with your patch rebased on top of mozilla-central (https://github.com/arthuredelstein/tor-browser/1436226+2). First I ran ./mach run and in the browser console I entered Services.prefs.prefHasUserValue("security.nocertdb") and got false.

      Then I ran the test as before:

      ./mach mochitest toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      and got

        TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE. If removing or changing this test, please notify tbb-dev@lists.torproject.org - got 2153394164, expected 2153398272

      Next, I tried running the test with the --setpref flag as you suggested:

      ./mach mochitest --setpref=security.nocertdb=false toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      and the result I got was:

        TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE. If removing or changing this test, please notify tbb-dev@lists.torproject.org - got 2153394164, expected 2153398272
SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:312:5
checkErrorCode@chrome://mochitests/content/chrome/toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul:61:3

      I then tried setting the flag to true:

      ./mach mochitest --setpref=security.nocertdb=true toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      and I got:

        TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE. If removing or changing this test, please notify tbb-dev@lists.torproject.org - got 2153390067, expected 2153398272
SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:312:5
checkErrorCode@chrome://mochitests/content/chrome/toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul:61:3

      In sum, I think my build does not have Tor Browser pref settings, by default it should be operating equivalent to the pref being set to false. So when security.nocertdb is default or false, I get the unexpected error code of 2153394164 and when security.nocertdb is set to true, I get the unexpected error code of 2153390067. Both error values are different from the expected error code 2153398272.

      Next I'm going to try these same set of experiments on tor-browser.git.

    • A
      Arthur Edelstein @arthuredelstein ·
      Author

      I built tor-browser.git and confirmed in about:config that security.nocertdb is set to true.

      Then I ran the same 3 tests as before:

      ./mach mochitest toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      gave

      56 INFO TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE - got 2153390067, expected 2153398272

      Then

      ./mach mochitest --setpref=security.nocertdb=false toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      resulted in all tests passing. Finally,

      ./mach mochitest --setpref=security.nocertdb=true toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul

      gave the result

      56 INFO TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/chrome/test_0790_check_certPinning_noUpdate.xul | Checking update.errorCode == MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE - got 2153390067, expected 2153398272

      So to analyze: my observation in the description tor-browser-52.6.0esr-8.0-2 was irrelevant because I had left security.nocertdb=true (Both branches give the same unexpected error code of 2153390067, but that's not very relevant.). If we look at security.nocertdb=false only, then I see that the test passes for tor-browser-52.6.0esr-8.0-2 but it fails on top of mozilla-central with a different unexpected error code of 2153394164.

    • A
      Arthur Edelstein @arthuredelstein ·
      Author

      So my question is, does it matter that we're getting a different error code? I am wondering whether any error code will do, or if we need MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE specifically. (I'm probably still naive about the design of the test.)

      I haven't figured out what this new error code, 2153394164 is intended to mean. I guess the next step for me might be to try to use gdb to track down where the old and new error codes are coming from.

    • Mark Smith
      Mark Smith @mcs ·

      Replying to arthuredelstein:

      So my question is, does it matter that we're getting a different error code? I am wondering whether any error code will do, or if we need MOZILLA_PKIX_ERROR_KEY_PINNING_FAILURE specifically. (I'm probably still naive about the design of the test.)

      It probably does not matter too much, although it is designed to look specifically for a pinning failure. Let me think about what to do.

      I haven't figured out what this new error code, 2153394164 is intended to mean. I guess the next step for me might be to try to use gdb to track down where the old and new error codes are coming from.

      Yes, although I don't know how much work that will be. Let me know if you want me to make an attempt (although I won't get to it until next week at this point). The new error code is 0x805A2FF4 which is SSL_ERROR_BAD_CERT_DOMAIN so it isn't too far off in terms of the expected error. But I think we should dive deep enough to understand what is happening.

    • Mark Smith
      Mark Smith @mcs ·

      After some time in the debugger comparing execution path of Tor Browser vs. a mozilla-central build, I found the problem: to get the correct error code we need to enable strict enforcement for pinning. I will attach a patch for the test which works for me.

    • Mark Smith
      Mark Smith @mcs ·

      Trac:
      bug25331-patch.txt

    • Mark Smith
      Mark Smith @mcs ·

      patch for test

    • A
      Arthur Edelstein @arthuredelstein ·
      Author

      Trac:
      Cc: mcs, brade to mcs, brade, arthuredelstein

    • A
      Arthur Edelstein @arthuredelstein ·
      Author

      This patch looks good to me. I have included it my nascent ESR60 branch. We could also consider including it in the ESR52 branches.

      Trac:
      Keywords: N/A deleted, TorBrowserTeam201804R added
      Status: needs_information to needs_review
      Cc: mcs, brade, arthuredelstein to mcs, brade, arthuredelstein, gk

    • Georg Koppen
      Georg Koppen @gk ·

      Fine with me. To save a round trip I just pushed the patch to tor-browser-52.7.3esr-8.0-1 (commit 292456846d8420e88b19664e36562cb4d6169776) and tor-browser-52.7.3esr-7.5-1 (commit 8592446c194092c93c854611fc7dadd1fd3f3917) mentioning that mcs and brade wrote it. Thanks!

      Trac:
      Status: needs_review to closed
      Resolution: N/A to fixed

    • Trac closed

      closed

    • Trac moved to tpo/applications/tor-browser#25331 (closed)

      moved to tpo/applications/tor-browser#25331 (closed)

    Please register or sign in to reply