https://browserleaks.com/webgl crashes tab on 64bit Tor Browser for Windows

added TorBrowserTeam201901R component::applications/tor browser owner::tbb-team priority::high resolution::fixed severity::normal status::closed tbb-crash tbb-rbm type::defect labels

Trac:
Description: On our blog one user noted that https://browserleaks.com/webgl is crashing their tab reliably using 8.5a6 in the 64bit flavor for Windows. This does not happen with the 32bit variant. And neither does this happen with vanilla Firfox 60.4.0esr.

It seems that a bare Firefox compiled our way without any Tor Browser patches is crashing as well. Thus, I wonder whether we hit a GCC bug or something that is only affecting some 64bit system. Mysterious.

to

On our blog one user noted (https://blog.torproject.org/comment/278911#comment-278911) that https://browserleaks.com/webgl is crashing their tab reliably using 8.5a6 in the 64bit flavor for Windows. This does not happen with the 32bit variant. And neither does this happen with vanilla Firfox 60.4.0esr.

It seems that a bare Firefox compiled our way without any Tor Browser patches is crashing as well. Thus, I wonder whether we hit a GCC bug or something that is only affecting some 64bit system. Mysterious.

I tried to create debug builds to help narrowing this down but that only resulted in sadness and #28875 (moved). :(

Does the vanilla 64bit build work for you, cypherpunk? (see: https://queue.taskcluster.net/v1/task/crV_nUFPRjarCLDD-mmZRQ/runs/0/artifacts/public/build/target.zip for an artifact. Unzipping the target.zip should give you a firefox folder to test).

Trac:
Status: new to needs_information
Keywords: N/A deleted, TorBrowserTeam201812 added

Replying to gk:

Does the vanilla 64bit build work for you, cypherpunk? (see: https://queue.taskcluster.net/v1/task/crV_nUFPRjarCLDD-mmZRQ/runs/0/artifacts/public/build/target.zip for an artifact. Unzipping the target.zip should give you a firefox folder to test).

That one is crashing as well. It seems the user hits

        if (!mResolvedDefaultFB) {
            gfxCriticalNote << funcName << ": Failed to create mResolvedDefaultFB.";
            return false;
        }

in dom/canvas/WebGLContext.cpp immediately before the crash.

Hey, could you test a debug build to gather more information about what is going on?

https://people.torproject.org/~gk/testbuilds/torbrowser-install-win64-tbb-nightly_debug_en-US.exe https://people.torproject.org/~gk/testbuilds/torbrowser-install-win64-tbb-nightly_debug_en-US.exe.asc

It will crash during first start (which is #28875 (moved)) but should work afterwards. It would be interesting to see if we hit any assertions before crashing.

not sure how to test it, but here is what I've got: it crashed during first start as expected; it got stuck at connect to tor phase during second and third start due to #28405 (moved) and #28614 (moved); it connected and "crashed" from webgl as expected during fourth start; nothing in debug logs; then I set e10s off and got

WARN: initializeD3DDevice(799): Failed creating Debug D3D11 device - falling back to release runtime.
Initializing context 000002b11d306db0 surface 000002b11d364500 on display 000002b11fa9e5b0
Initializing context 000002b11530efc0 surface 000002b11d362300 on display 000002b11fa9e5b0
Initializing context 000002b11c731c20 surface 000002b11d358c00 on display 000002b11fa9e5b0
JavaScript error: , line 0: uncaught exception: TagError: adsbygoogle.push() error: All ins elements in the DOM with class=adsbygoogle already have ads in them.
JavaScript error: , line 0: uncaught exception: TagError: adsbygoogle.push() error: All ins elements in the DOM with class=adsbygoogle already have ads in them.
Initializing context 000002b1140d85f0 surface 000002b115358920 on display 000002b11fa9e5b0
ERR: ensureInitialized(141): D3D compiler module not found.
[gl:000002b127ca1ee0] void mozilla::gl::GLContext::fLinkProgram(GLuint): Generated unexpected GL_OUT_OF_MEMORY error. (0x0505)
Hit MOZ_CRASH(Unexpected error with MOZ_GL_DEBUG_ABORT_ON_ERROR. (Run with MOZ_GL_DEBUG_ABORT_ON_ERROR=0 to disable)) at /var/tmp/build/firefox-8cac0295141b/gfx/gl/GLContext.cpp:3030

with usual "EXCEPTION_ACCESS_VIOLATION_READ" in xul.dll

Trac:
Username: gl

Thanks!

Replying to gl:

ERR: ensureInitialized(141): D3D compiler module not found.

I guess that's the reason for the trouble you are seeing. The relevant code around it is:

#if !defined(ANGLE_ENABLE_WINDOWS_STORE)
#if defined(ANGLE_PRELOADED_D3DCOMPILER_MODULE_NAMES)
    // Find a D3DCompiler module that had already been loaded based on a predefined list of versions.
    static const char *d3dCompilerNames[] = ANGLE_PRELOADED_D3DCOMPILER_MODULE_NAMES;

    for (size_t i = 0; i < ArraySize(d3dCompilerNames); ++i)
    {
        if (GetModuleHandleExA(0, d3dCompilerNames[i], &mD3DCompilerModule))
        {
            break;
        }
    }
#endif  // ANGLE_PRELOADED_D3DCOMPILER_MODULE_NAMES

    if (!mD3DCompilerModule)
    {
        // Load the version of the D3DCompiler DLL associated with the Direct3D version ANGLE was built with.
        mD3DCompilerModule = LoadLibrary(D3DCOMPILER_DLL);
    }

    if (!mD3DCompilerModule)
    {
        ERR() << "D3D compiler module not found.";
        return gl::OutOfMemory() << "D3D compiler module not found.";
    }

    mD3DCompileFunc = reinterpret_cast<pD3DCompile>(GetProcAddress(mD3DCompilerModule, "D3DCompile"));
    ASSERT(mD3DCompileFunc);

    mD3DDisassembleFunc = reinterpret_cast<pD3DDisassemble>(GetProcAddress(mD3DCompilerModule, "D3DDisassemble"));
    ASSERT(mD3DDisassembleFunc);

#else
    // D3D Shader compiler is linked already into this module, so the export
    // can be directly assigned.
    mD3DCompilerModule = nullptr;
    mD3DCompileFunc = reinterpret_cast<pD3DCompile>(D3DCompile);
    mD3DDisassembleFunc = reinterpret_cast<pD3DDisassemble>(D3DDisassemble);
#endif

in HLSLCompiler.cpp

We are not defining ANGLE_PRELOADED_D3DCOMPILER_MODULE_NAMES in https://dxr.mozilla.org/mozilla-esr60/source/gfx/angle/update-angle.py#257. But I wonder where D3DCOMPILER_DLL is supposed to come from in our case. From d3dcompiler.rs??

https://bugzilla.mozilla.org/show_bug.cgi?id=1460620

Trac:
Username: omg

what should I check? and why is that 64-bit-specific?

Trac:
Username: gl

Replying to gk:

We are not defining ANGLE_PRELOADED_D3DCOMPILER_MODULE_NAMES in https://dxr.mozilla.org/mozilla-esr60/source/gfx/angle/update-angle.py#257. But I wonder where D3DCOMPILER_DLL is supposed to come from in our case. From d3dcompiler.rs??

Nope: https://github.com/apitrace/dxsdk/blob/master/Include/d3dcompiler.h#L18

Since all this seems to be related to DirectX; I suppose getting some info about that might be helpful.

Start -> Run -> dxdiag will get diagnostic information. At a minimum I'd say go to the Display page and copy that info (either with a screenshot, or save the info to a text file and copy it out.)

gl: any update on this issue?

https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/direct-x/include/d3dcompiler.h#l38

Trac:
Username: omg

Thanks cypherpunk; that's probably it. I'm sending in some trials on TaskCluster to see if that solves my 8-month old error. I pinged Jacek about whether MinGW should update or we should override.

gl or anyone else who hits this problem: does it get solved by

https://people.torproject.org/~gk/testbuilds/torbrowser-install-win64-tbb-nightly_28874_en-US.exe https://people.torproject.org/~gk/testbuilds/torbrowser-install-win64-tbb-nightly_28874_en-US.exe.asc

? (You might need to restart Tor Browser due to things like #28405 (moved))

The bundle got built with tom's mingw-w64 fix, fwiw.

D3DCOMPILER_DLL define is fixed in upstream mingw-w64 now. I recall that some declarations also vary between d3dcompiler versions, so if similar problems persist, it might be good place to look.

Trac:
Username: jacek

Moving tickets to Jan 2019.

Trac:
Keywords: TorBrowserTeam201812 deleted, TorBrowserTeam201901 added

Replying to jacek:

D3DCOMPILER_DLL define is fixed in upstream mingw-w64 now. I recall that some declarations also vary between d3dcompiler versions, so if similar problems persist, it might be good place to look.

Thanks! It seems we are fine for now by just bumping the mingw-w64 commit we use (see the test results in comment:32:ticket:28823 compared to comment:28:ticket:28823). bug_28874 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/commit/?h=bug_28874&id=5e875c30e825bd726558a3b9fea63b59d2a80d15) has the respective patch for review.

Trac:
Keywords: TorBrowserTeam201901 deleted, TorBrowserTeam201901R, tbb-rbm added
Status: needs_information to needs_review

https://trac.torproject.org/projects/tor/ticket/26650#comment:5

Trac:
Username: omg

Replying to gk:

Thanks! It seems we are fine for now by just bumping the mingw-w64 commit we use (see the test results in comment:32:ticket:28823 compared to comment:28:ticket:28823). bug_28874 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/commit/?h=bug_28874&id=5e875c30e825bd726558a3b9fea63b59d2a80d15) has the respective patch for review.

This looks good to me. I cherry-picked this patch as commit 87f892495a13e878922147429e4c2b7a423f0d8e on master.

Trac:
Status: needs_review to closed
Resolution: N/A to fixed

https://browserleaks.com/webgl crashes tab on 64bit Tor Browser for Windows

Child items 0

Activity