Opened 4 years ago

Closed 4 years ago

Last modified 2 years ago

#17248 closed task (fixed)

Investigate new WebExtensions API requirements for our extensions

Reported by: gk Owned by: tbb-team
Priority: High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: TorBrowserTeam201512
Cc: mcs, brade, huseby, intrigeri, anonym, arthuredelstein, tom Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

A while ago Mozilla announced (https://blog.mozilla.org/addons/2015/08/21/the-future-of-developing-firefox-add-ons/) that the XUL/XPCOM-based extensions will be deprecated and that add-ons built with WebExtensions are the way forward.

We should investigate what this means for our extensions and report back to Mozilla in order to make sure our requirements are taken into account while designing the APIs.

Child Tickets

Change History (21)

comment:1 Changed 4 years ago by gk

Keywords: TorBrowserTeam201511 added; TorBrowserTeam201510 removed

comment:2 Changed 4 years ago by brade

Cc: mcs brade added
Severity: Normal

Mark and I made a pass through Tor Launcher looking at API requirements. I am going to post a series of comments that capture what we learned (this is the first).

Requirements for which there is no solution / no Web Extensions API (NS#):

NS1: Define and potentially override default preference values
Current Solution: defaults/preferences/prefs.js

NS2: Read, write, observe, and enumerate existing browser preferences
Current Solution: nsIPrefService/nsIPrefBranch and related APIs

NS3: Change the browser's UI locale without restart
Current Solution: Change the browser.useragent.locale preference and then send a "chrome-flush-caches" observer notification
Notes: Used in Tor Launcher after the user choose a language

NS4: Localized HTML
Current Solution: XUL with localized entities
Related bugs: https://code.google.com/p/chromium/issues/detail?id=115800

NS5: Get notified just before the browser exits
Current Solution: Observer for "quit-application-granted"
Notes: Could count open and close of tabs but this will not work on Mac OS

NS6: Display a confirm prompt with custom button labels
Current Solution: nsIPromptService.confirmEx()
Notes: Used in Tor Launcher for the "Restart Tor?" prompt

NS7: Compare application version strings
Current Solution: nsIVersionComparator
Notes: We could create our own JS implemention or use third party code such as https://github.com/linagora/mozilla-version-comparator

NS8: Check for existence, get, and set environment variables
Current Solution: nsIEnvironment

NS9: Log to stderr
Current Solution: window.dump()

NS10: TCP communication to the Tor control port
Current Solution: nsISocketTransportService.createTransport()
Notes: Tor Launcher uses non-blocking sockets as well as nsIInputStream.asyncWait(). Also, in the future we want to be able to use UNIX domain sockets.

NS11: Modal interaction before browser completely starts up
Current Solution: Modal dialogs opened from a "profile-after-change" observer
Notes: The first time Tor Browser starts up, Tor Launcher displays a configuration wizard. Every time Tor Browser starts up, a modal progress dialog is displayed while a Tor network connection is established. It is important that browser windows and menus are not shown until the startup dialogs have closed.

comment:3 Changed 4 years ago by brade

Requirements for which Web Extension APIs exist that are not yet implemented in Firefox (NI#):

NI1: Retrieve OS identifier
Current Solution: nsIXULRuntime.OS
New Solution: chrome.runtime.PlatformInfo.PlatformOs
Related Bugs: https://bugzil.la/1213473

NI2: Size a popup window to fit its content
Current Solution: window.sizeToContent() and window.resizeTo()
New Solution: document.body.offsetWidth/offsetHeight followed by windows.update() to adjust size
Related Bugs: https://bugzil.la/1213484

NI3: Copy text to the system clipboard
Current Solution: nsIClipboardHelper.copyString()
New Solution: Add this to manifest.json: permissions: [ "clipboardWrite" ] then put the text to be copied in an HTML element, select it, and then call document.execCommand("copy");
Related Bugs: https://bugzil.la/1197451

NI4: Exit the browser
Current Solution: nsIAppStartup.quit(eAttemptQuit)
New Solution: chrome.processes.terminate(0) (dev channel only)

comment:4 Changed 4 years ago by brade

Requirements for which Web Extension APIs have been implemented in Firefox (D#, aka Done#):

D1: Asynchronous communication between code modules
Current Solution: nsIObserverService/nsIObserver
New Solution: chrome.runtime.sendMessage(), chrome.runtime.onMessage()

D2: Get current UI locale (used for language prompt)
Current Solution: nsIXULChromeRegistry.getSelectedLocale("global")
New Solution: Retrieve predefined @@ui_locale message using i18n API, e.g., let locale = chrome.i18n.getMessage("@@ui_locale"); or use let language = chrome.i18n.getUILanguage(); (the latter is not yet supported in Firefox)

D3: Retrieve a localized string
Current Solution: nsIStringBundle.GetStringFromName()/formatStringFromName()
New Solution: chrome.i18n.getMessage()

comment:5 Changed 4 years ago by brade

Requirements that we would probably implement using Native Messaging once it is available in Firefox (NM#):
Related Bugs: https://bugzil.la/1190682

NM1: Start and monitor an external process
Current Solution: nsIProcess
New Solution: Native Messaging

NM2: Get PID of the browser process
Current Solution: nsIXULRuntime.processID (fallback uses js-ctypes)
New Solution: Native Messaging

NM3: Get current working directory
Current Solution: directory service "CurProcD"
New Solution: Native Messaging

NM4: Access to local files
Current Solution: nsIFile
New Solution: Native Messaging
Notes: Tor Launcher checks for existence of files, reads data from files, constructs files paths, and parses file paths.

comment:6 Changed 4 years ago by brade

Requirements for which there are other acceptable workarounds (WA#):

WA1: Identify application (e.g., Firefox vs. Thunderbird)
Current Solution: nsIXULAppInfo.ID
New Solution: Extract info from navigator.userAgent

WA2: Get application version (e.g., 42.0)
Current Solution: nsIXULAppInfo.version
New Solution: Extract info from navigator.userAgent

WA3: Find a window of a particular type
Current Solution: nsIWindowMediator.getMostRecentWindow()
New Solution: Cache the ID of special windows that we open and se chrome.windows.get() to retrieve by ID

WA4: Log to the console
Current Solution: nsIConsoleService.logStringMessage()
New Solution: console.log()

WA5: Display an error alert
Current Solution: nsIPromptService.alert()
New Solution: window.alert()

WA6: Localized date and time formatting
Current Solution: nsIScriptableDateFormat.FormatDateTime()
New Solution: Use JS APIs such as Date.toLocaleDateString()
Notes: Needed in Tor Launcher to format dates and time for log messages

WA7: Timers
Current Solution: nsITimer with TYPE_ONE_SHOT
New Solution: window.setTimeout(). Also look at chrome.alarms.*

WA8: Encode a password using SHA1
Current Solution: nsICryptoHash
New Solution: Web Crypto API: window.crypto.subtle.digest("SHA-1")

WA9: Generate random bytes
Current Solution: nsIRandomGenerator
New Solution: Web Crypto API: window.crypto.getRandomValues()
Notes: Used in Tor Launcher to generate a random control port password.
Tor Issue: Will window.crypto.getRandomValues() produce a secure enough password?

WA10: Wizard-style configuration UI
Current Solution: XUL <wizard>
New Solution: Roll our own using HTML and CSS.

WA11: Progress meter
Current Solution: XUL <progressmeter>
New Solution: HTML5 <meter>

comment:7 Changed 4 years ago by huseby

Cc: huseby added

comment:8 Changed 4 years ago by mikeperry

Keywords: TorBrowserTeam201512 added; TorBrowserTeam201511 removed

comment:9 Changed 4 years ago by gk

Resolution: fixed
Status: newclosed

This is done and Mozilla should be aware of our needs by now.

comment:10 Changed 3 years ago by intrigeri

Cc: intrigeri anonym added

comment:11 Changed 3 years ago by arthuredelstein

Cc: arthuredelstein added

comment:12 in reply to:  2 ; Changed 3 years ago by tom

Cc: tom added

Replying to brade:

NS10: TCP communication to the Tor control port
Current Solution: nsISocketTransportService.createTransport()
Notes: Tor Launcher uses non-blocking sockets as well as nsIInputStream.asyncWait(). Also, in the future we want to be able to use UNIX domain sockets.

I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.

UNfortunetly, the only currently defined type for the communication is using standard input/output, so it'd require Tor's control port to change quite drastically....

comment:13 in reply to:  12 ; Changed 3 years ago by mcs

Replying to tom:

I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.

I think we should leave tor unchanged and instead think about splitting Tor Launcher into a WebExtensions (JavaScript) part and a compiled (Go?) "shim." Communication between the two pieces would be via Native messaging (aka connectNative). Kathy and I did something similar for a Chrome extension that we wrote several years ago (a non-Tor project). It worked pretty well. Obviously we will need to define our own protocol between the JS code and the compiled code.

comment:14 in reply to:  13 ; Changed 3 years ago by gk

Replying to mcs:

Replying to tom:

I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.

I think we should leave tor unchanged and instead think about splitting Tor Launcher into a WebExtensions (JavaScript) part and a compiled (Go?) "shim." Communication between the two pieces would be via Native messaging (aka connectNative). Kathy and I did something similar for a Chrome extension that we wrote several years ago (a non-Tor project). It worked pretty well. Obviously we will need to define our own protocol between the JS code and the compiled code.

Keep in mind that we need this to work on mobile as well. While I don't worry about the JS part I am not sure about the shim. That said we are probably going to get money for Tor Browser on mobile and one of my secret hopes was to get a mobile version of Tor Launcher with it that we then can use on desktop too.

comment:15 in reply to:  14 ; Changed 3 years ago by yawning

Replying to gk:

Replying to mcs:

Replying to tom:

I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.

I think we should leave tor unchanged and instead think about splitting Tor Launcher into a WebExtensions (JavaScript) part and a compiled (Go?) "shim." Communication between the two pieces would be via Native messaging (aka connectNative). Kathy and I did something similar for a Chrome extension that we wrote several years ago (a non-Tor project). It worked pretty well. Obviously we will need to define our own protocol between the JS code and the compiled code.

Keep in mind that we need this to work on mobile as well. While I don't worry about the JS part I am not sure about the shim. That said we are probably going to get money for Tor Browser on mobile and one of my secret hopes was to get a mobile version of Tor Launcher with it that we then can use on desktop too.

My 0.02 Zimbabwean Dollar opinion on this matter is that the right thing to do from a technical standpoint is to kill tor-launcher with fire, and maintain launchers for each of the supported platforms. Having to start firefox with the privileges/capabilities that tor needs is horrific and is something that we should move away from unless we have a lot of faith in Firefox acquiring decent security/sandboxing.

nb: I picked a totally worthless currency for a reason, and don't expect this to actually be done.

comment:16 Changed 3 years ago by tom

So while I agree with Yawning to some respects for future endeavors and improvements, after talking with some Add-On/WebExtension folks I think the path forward for Tor Launcher is simple - we don't _have_ to do anything. We don't _have_ to port these to WebExtensions, as 'legacy' add-ons will live on as Privileged Add-Ons. Mozilla will require special signatures for Privileged Add-Ons, but the capability to use them will live on - possibly indefinitely but certainly for 59.

Tor Browser will need to sign our extensions as these privileged add-ons, using a key we bake into Tor Browser; but it's certainly easier than doing anything else.

comment:17 in reply to:  16 ; Changed 3 years ago by mcs

Replying to tom:

So while I agree with Yawning to some respects for future endeavors and improvements, after talking with some Add-On/WebExtension folks I think the path forward for Tor Launcher is simple - we don't _have_ to do anything. We don't _have_ to port these to WebExtensions, as 'legacy' add-ons will live on as Privileged Add-Ons. Mozilla will require special signatures for Privileged Add-Ons, but the capability to use them will live on - possibly indefinitely but certainly for 59.

Thanks for hunting down that info. Is there anything available publicly about "Privileged Add-ons", e.g., documentation or a Bugzilla bug? Does Mozilla plan to use such add-ons for their own purposes?

comment:18 in reply to:  17 Changed 3 years ago by tom

Replying to mcs:

Replying to tom:

So while I agree with Yawning to some respects for future endeavors and improvements, after talking with some Add-On/WebExtension folks I think the path forward for Tor Launcher is simple - we don't _have_ to do anything. We don't _have_ to port these to WebExtensions, as 'legacy' add-ons will live on as Privileged Add-Ons. Mozilla will require special signatures for Privileged Add-Ons, but the capability to use them will live on - possibly indefinitely but certainly for 59.

Thanks for hunting down that info. Is there anything available publicly about "Privileged Add-ons", e.g., documentation or a Bugzilla bug?

Probably not.

Does Mozilla plan to use such add-ons for their own purposes?

Yes, that's the plan. Things like Test Pilot will be privileged add-ons, and I'm certain there will be others.

comment:19 Changed 3 years ago by cypherpunks

we don't _have_ to do anything. We don't _have_ to port these to WebExtensions, as 'legacy' add-ons will live on

Except that they'll start ripping out XUL/XPCOM in its entirety, and making other backwards-incompatible changes with no worries, since only Mozilla-written code is intended to use those APIs from now on anyway. There's going to be some required porting of some kind, it might as well be to the supported stable API.

In particular, if there's any stored data the add-on uses, you definitely want to move to an 'embedded WebExtension' https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Embedded_WebExtensions architecture and use webext APIs for stored data sooner than later to ease any future migration, like if the legacy APIs used for data storage in e.g. Firefox 52 are suddenly not there in Firefox 59.

comment:20 in reply to:  15 Changed 2 years ago by cypherpunks

Replying to yawning:

Having to start firefox with the privileges/capabilities that tor needs is horrific and is something that we should move away from unless we have a lot of faith in Firefox acquiring decent security/sandboxing.

Well seems like the faith paid off ;) FF acquired Chromium-level sandboxing on all three major platforms and should be more and more restricted for the upcoming esr FF59: https://wiki.mozilla.org/Security/Sandbox https://wiki.mozilla.org/Security/Sandbox/Specifics

comment:21 Changed 2 years ago by cypherpunks

As stated by the other cypherpunk in #19, Mozilla are removing the stuff that is no longer needed when XPCOM extensions are no longer supported https://bugzilla.mozilla.org/show_bug.cgi?id=1347507

FWIW here's the discussion on WebExtensions and Testpilot (might not be up to date) https://bugzilla.mozilla.org/show_bug.cgi?id=1280233

Also porting it to WebExtension early may be a great way to catch all the bugs before it's production ready.

Last edited 2 years ago by cypherpunks (previous) (diff)
Note: See TracTickets for help on using tickets.