Investigate new WebExtensions API requirements for our extensions
A while ago Mozilla announced (https://blog.mozilla.org/addons/2015/08/21/the-future-of-developing-firefox-add-ons/) that the XUL/XPCOM-based extensions will be deprecated and that add-ons built with WebExtensions are the way forward.
We should investigate what this means for our extensions and report back to Mozilla in order to make sure our requirements are taken into account while designing the APIs.
Activity
Mark and I made a pass through Tor Launcher looking at API requirements. I am going to post a series of comments that capture what we learned (this is the first).
Requirements for which there is no solution / no Web Extensions API (NS#):
NS1: Define and potentially override default preference values Current Solution: defaults/preferences/prefs.js
NS2: Read, write, observe, and enumerate existing browser preferences Current Solution: nsIPrefService/nsIPrefBranch and related APIs
NS3: Change the browser's UI locale without restart Current Solution: Change the browser.useragent.locale preference and then send a "chrome-flush-caches" observer notification Notes: Used in Tor Launcher after the user choose a language
NS4: Localized HTML Current Solution: XUL with localized entities Related bugs: https://code.google.com/p/chromium/issues/detail?id=115800
NS5: Get notified just before the browser exits Current Solution: Observer for "quit-application-granted" Notes: Could count open and close of tabs but this will not work on Mac OS
NS6: Display a confirm prompt with custom button labels Current Solution: nsIPromptService.confirmEx() Notes: Used in Tor Launcher for the "Restart Tor?" prompt
NS7: Compare application version strings Current Solution: nsIVersionComparator Notes: We could create our own JS implemention or use third party code such as https://github.com/linagora/mozilla-version-comparator
NS8: Check for existence, get, and set environment variables Current Solution: nsIEnvironment
NS9: Log to stderr Current Solution: window.dump()
NS10: TCP communication to the Tor control port Current Solution: nsISocketTransportService.createTransport() Notes: Tor Launcher uses non-blocking sockets as well as nsIInputStream.asyncWait(). Also, in the future we want to be able to use UNIX domain sockets.
NS11: Modal interaction before browser completely starts up Current Solution: Modal dialogs opened from a "profile-after-change" observer Notes: The first time Tor Browser starts up, Tor Launcher displays a configuration wizard. Every time Tor Browser starts up, a modal progress dialog is displayed while a Tor network connection is established. It is important that browser windows and menus are not shown until the startup dialogs have closed.
Trac:
Cc: N/A to mcs, brade
Severity: N/A to Normal-
Requirements for which Web Extension APIs exist that are not yet implemented in Firefox (NI#):
NI1: Retrieve OS identifier Current Solution: nsIXULRuntime.OS New Solution: chrome.runtime.PlatformInfo.PlatformOs Related Bugs: https://bugzil.la/1213473
NI2: Size a popup window to fit its content Current Solution: window.sizeToContent() and window.resizeTo() New Solution: document.body.offsetWidth/offsetHeight followed by windows.update() to adjust size Related Bugs: https://bugzil.la/1213484
NI3: Copy text to the system clipboard Current Solution: nsIClipboardHelper.copyString() New Solution: Add this to manifest.json: permissions: [ "clipboardWrite" ] then put the text to be copied in an HTML element, select it, and then call document.execCommand("copy"); Related Bugs: https://bugzil.la/1197451
NI4: Exit the browser Current Solution: nsIAppStartup.quit(eAttemptQuit) New Solution: chrome.processes.terminate(0) (dev channel only)
-
Requirements for which Web Extension APIs have been implemented in Firefox (D#, aka Done#):
D1: Asynchronous communication between code modules Current Solution: nsIObserverService/nsIObserver New Solution: chrome.runtime.sendMessage(), chrome.runtime.onMessage()
D2: Get current UI locale (used for language prompt) Current Solution: nsIXULChromeRegistry.getSelectedLocale("global") New Solution: Retrieve predefined @@ui_locale message using i18n API, e.g., let locale = chrome.i18n.getMessage("@@ui_locale"); or use let language = chrome.i18n.getUILanguage(); (the latter is not yet supported in Firefox)
D3: Retrieve a localized string Current Solution: nsIStringBundle.GetStringFromName()/formatStringFromName() New Solution: chrome.i18n.getMessage()
-
Requirements that we would probably implement using Native Messaging once it is available in Firefox (NM#): Related Bugs: https://bugzil.la/1190682
NM1: Start and monitor an external process Current Solution: nsIProcess New Solution: Native Messaging
NM2: Get PID of the browser process Current Solution: nsIXULRuntime.processID (fallback uses js-ctypes) New Solution: Native Messaging
NM3: Get current working directory Current Solution: directory service "CurProcD" New Solution: Native Messaging
NM4: Access to local files Current Solution: nsIFile New Solution: Native Messaging Notes: Tor Launcher checks for existence of files, reads data from files, constructs files paths, and parses file paths.
-
Requirements for which there are other acceptable workarounds (WA#):
WA1: Identify application (e.g., Firefox vs. Thunderbird) Current Solution: nsIXULAppInfo.ID New Solution: Extract info from navigator.userAgent
WA2: Get application version (e.g., 42.0) Current Solution: nsIXULAppInfo.version New Solution: Extract info from navigator.userAgent
WA3: Find a window of a particular type Current Solution: nsIWindowMediator.getMostRecentWindow() New Solution: Cache the ID of special windows that we open and se chrome.windows.get() to retrieve by ID
WA4: Log to the console Current Solution: nsIConsoleService.logStringMessage() New Solution: console.log()
WA5: Display an error alert Current Solution: nsIPromptService.alert() New Solution: window.alert()
WA6: Localized date and time formatting Current Solution: nsIScriptableDateFormat.FormatDateTime() New Solution: Use JS APIs such as Date.toLocaleDateString() Notes: Needed in Tor Launcher to format dates and time for log messages
WA7: Timers Current Solution: nsITimer with TYPE_ONE_SHOT New Solution: window.setTimeout(). Also look at chrome.alarms.*
WA8: Encode a password using SHA1 Current Solution: nsICryptoHash New Solution: Web Crypto API: window.crypto.subtle.digest("SHA-1")
WA9: Generate random bytes Current Solution: nsIRandomGenerator New Solution: Web Crypto API: window.crypto.getRandomValues() Notes: Used in Tor Launcher to generate a random control port password. Tor Issue: Will window.crypto.getRandomValues() produce a secure enough password?
WA10: Wizard-style configuration UI Current Solution: XUL New Solution: Roll our own using HTML and CSS.
WA11: Progress meter Current Solution: XUL New Solution: HTML5
Replying to brade:
NS10: TCP communication to the Tor control port Current Solution: nsISocketTransportService.createTransport() Notes: Tor Launcher uses non-blocking sockets as well as nsIInputStream.asyncWait(). Also, in the future we want to be able to use UNIX domain sockets.
I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.
UNfortunetly, the only currently defined type for the communication is using standard input/output, so it'd require Tor's control port to change quite drastically....
Trac:
Cc: mcs, brade, huseby, intrigeri, anonym, arthuredelstein to mcs, brade, huseby, intrigeri, anonym, arthuredelstein, tom
Replying to tom:
I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.
I think we should leave tor unchanged and instead think about splitting Tor Launcher into a WebExtensions (JavaScript) part and a compiled (Go?) "shim." Communication between the two pieces would be via Native messaging (aka connectNative). Kathy and I did something similar for a Chrome extension that we wrote several years ago (a non-Tor project). It worked pretty well. Obviously we will need to define our own protocol between the JS code and the compiled code.
Replying to mcs:
Replying to tom:
I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.
I think we should leave tor unchanged and instead think about splitting Tor Launcher into a WebExtensions (JavaScript) part and a compiled (Go?) "shim." Communication between the two pieces would be via Native messaging (aka connectNative). Kathy and I did something similar for a Chrome extension that we wrote several years ago (a non-Tor project). It worked pretty well. Obviously we will need to define our own protocol between the JS code and the compiled code.
Keep in mind that we need this to work on mobile as well. While I don't worry about the JS part I am not sure about the shim. That said we are probably going to get money for Tor Browser on mobile and one of my secret hopes was to get a mobile version of Tor Launcher with it that we then can use on desktop too.
Replying to gk:
Replying to mcs:
Replying to tom:
I think that https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/Runtime/connectNative might be a workable solution to this one. Obviously it would require the other side (Tor's control port) to change.
I think we should leave tor unchanged and instead think about splitting Tor Launcher into a WebExtensions (JavaScript) part and a compiled (Go?) "shim." Communication between the two pieces would be via Native messaging (aka connectNative). Kathy and I did something similar for a Chrome extension that we wrote several years ago (a non-Tor project). It worked pretty well. Obviously we will need to define our own protocol between the JS code and the compiled code.
Keep in mind that we need this to work on mobile as well. While I don't worry about the JS part I am not sure about the shim. That said we are probably going to get money for Tor Browser on mobile and one of my secret hopes was to get a mobile version of Tor Launcher with it that we then can use on desktop too.
My 0.02 Zimbabwean Dollar opinion on this matter is that the right thing to do from a technical standpoint is to kill
tor-launcherwith fire, and maintain launchers for each of the supported platforms. Having to start firefox with the privileges/capabilities that tor needs is horrific and is something that we should move away from unless we have a lot of faith in Firefox acquiring decent security/sandboxing.nb: I picked a totally worthless currency for a reason, and don't expect this to actually be done.
-
So while I agree with Yawning to some respects for future endeavors and improvements, after talking with some Add-On/WebExtension folks I think the path forward for Tor Launcher is simple - we don't have to do anything. We don't have to port these to WebExtensions, as 'legacy' add-ons will live on as Privileged Add-Ons. Mozilla will require special signatures for Privileged Add-Ons, but the capability to use them will live on - possibly indefinitely but certainly for 59.
Tor Browser will need to sign our extensions as these privileged add-ons, using a key we bake into Tor Browser; but it's certainly easier than doing anything else.
-
Replying to tom:
So while I agree with Yawning to some respects for future endeavors and improvements, after talking with some Add-On/WebExtension folks I think the path forward for Tor Launcher is simple - we don't have to do anything. We don't have to port these to WebExtensions, as 'legacy' add-ons will live on as Privileged Add-Ons. Mozilla will require special signatures for Privileged Add-Ons, but the capability to use them will live on - possibly indefinitely but certainly for 59.
Thanks for hunting down that info. Is there anything available publicly about "Privileged Add-ons", e.g., documentation or a Bugzilla bug? Does Mozilla plan to use such add-ons for their own purposes?
-
Replying to mcs:
Replying to tom:
So while I agree with Yawning to some respects for future endeavors and improvements, after talking with some Add-On/WebExtension folks I think the path forward for Tor Launcher is simple - we don't have to do anything. We don't have to port these to WebExtensions, as 'legacy' add-ons will live on as Privileged Add-Ons. Mozilla will require special signatures for Privileged Add-Ons, but the capability to use them will live on - possibly indefinitely but certainly for 59.
Thanks for hunting down that info. Is there anything available publicly about "Privileged Add-ons", e.g., documentation or a Bugzilla bug?
Probably not.
Does Mozilla plan to use such add-ons for their own purposes?
Yes, that's the plan. Things like Test Pilot will be privileged add-ons, and I'm certain there will be others.
we don't have to do anything. We don't have to port these to WebExtensions, as 'legacy' add-ons will live on
Except that they'll start ripping out XUL/XPCOM in its entirety, and making other backwards-incompatible changes with no worries, since only Mozilla-written code is intended to use those APIs from now on anyway. There's going to be some required porting of some kind, it might as well be to the supported stable API.
In particular, if there's any stored data the add-on uses, you definitely want to move to an 'embedded WebExtension' https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Embedded_WebExtensions architecture and use webext APIs for stored data sooner than later to ease any future migration, like if the legacy APIs used for data storage in e.g. Firefox 52 are suddenly not there in Firefox 59.
