Opened 4 months ago

Last modified 3 hours ago

#30126 new task

Make Tor Browser on macOS compatible with Apple's notarization

Reported by: gk Owned by: tbb-team
Priority: Very High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-security, TorBrowserTeam201908
Cc: mcs, brade, boklm, dcf Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by gk)

Notarization is a technique by Apple to make apps on macOS more secure to run. There a numerous parts to this and one can find more details about that on:

https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution

Mozilla is tracking the work in:

https://bugzilla.mozilla.org/show_bug.cgi?id=1470607

and there are a bunch of large pieces that still need to get solved on their side, like enabling the Hardened Runtime and building with the 10.14 SDK.

However, at some point in the future apps won't run without that anymore and the potential changes we need to made are probably considerable. Thus, we should keep an eye on that and start thinking about which pieces of our signing infrastructure need to get adapted. Questions could be:

1) Is it still enough to sign the builds on a 10.9 machine?
2) How do we integrate sending the apps to Apple to get their blessing into our release process?
3) How does that system work with our plan to get rid of the Apple signing machine and do the signing on Linux? (see: #29815)

I don't see this being relevant for ESR 68 but it might become so during the transition to the ESR after that one (or for the regular release train in case we'll start following that one instead).

Child Tickets

TicketStatusOwnerSummaryComponent
#31465closedtbb-teamAdapt tor-browser-build projects for macOS notarizationApplications/Tor Browser

Change History (40)

comment:1 Changed 4 months ago by gk

Description: modified (diff)

comment:2 Changed 2 months ago by gk

Cc: boklm added
Keywords: TorBrowserTeam201906 added
Priority: MediumVery High

It seems the future is getting faster to us than we hoped: https://blog.torproject.org/comment/282597#comment-282597 :(

We might not have all the Firefox code pieces ready to have this properly working but we should look pretty soon into making our signing infrastructure ready.

comment:3 Changed 2 months ago by gk

comment:4 in reply to:  3 Changed 7 weeks ago by gk

Replying to gk:

FWIW: It seems the updater is one of the first victims here: https://blog.torproject.org/comment/282619#comment-282619 and https://bugzilla.mozilla.org/show_bug.cgi?id=1556733.

That particular issue got fixed on Mozilla's side and backported to all relevant branches including esr60. Thus, we are good for that issue at least.

comment:5 Changed 7 weeks ago by gk

Keywords: TorBrowserTeam201907 added; TorBrowserTeam201906 removed

Moving tickets to July

comment:6 Changed 6 weeks ago by gk

https://bugzilla.mozilla.org/show_bug.cgi?id=1563631#c5 seems relevant here:

According to the WWDC presentation[1] around 8:00, all software "signed on built after June 1st" must be Notarized.

The presentation itself is at https://developer.apple.com/videos/play/wwdc2019/701/.

comment:7 Changed 6 weeks ago by mcs

Kathy and I need to do more research, but here are some things we learned so far.

Additional resources:

Some of the requirements, as specified by Apple's documentation:

  • Link against the macOS 10.9 or later SDK (already done for Tor Browser).
  • Notarization requires Xcode 10 or later (maybe simply because we need an xcrun that supports the altool, and that first appeared in Xcode 10.0).
  • Building a new app for notarization requires macOS 10.13.6 or later & Xcode 10 (macOS 10.13.6 is required for Xcode 10.0).
  • Stapling an app requires macOS 10.12 or later (but I guess we will have macOS 10.13.16 or newer anyway).
  • Enable code-signing for all of the executables you distribute (hopefully we already do this).
  • Use a Developer ID application, kernel extension, or installer certificate for your code-signing signature (a Mac Distribution or local development certificate will not work).
  • Include a secure timestamp with your code-signing signature (which means we need to include the --timestamp option when running the codesign tool).
  • Enable the Hardened Runtime capability for your app (how do we handle entitlements?)
  • Don't include the com.apple.security.get-task-allow entitlement with the value set to any variation of true (again, how do we add entitlements during our build process — if at all?)

The following Firefox bug includes at least one patch related to entitlements, although the patches are for taskcluster and not core Firefox code: https://bugzilla.mozilla.org/show_bug.cgi?id=1471004

It was suggested that we look at how Bitcoin Core is handling notarization, but all we found so far is this open issue: https://github.com/bitcoin/bitcoin/issues/15774

comment:8 Changed 6 weeks ago by tom

If you have questions; :haik spent a considerable time investigating and deploying this for Mozilla.

comment:9 in reply to:  8 Changed 5 weeks ago by gk

Replying to tom:

If you have questions; :haik spent a considerable time investigating and deploying this for Mozilla.

Good idea! mcs/brade could you follow up on that with a particular focus on our signing requirements, so we get the remaining questions we have answered and have a clearer picture on our constraints?

Last edited 5 weeks ago by gk (previous) (diff)

comment:10 Changed 5 weeks ago by gk

Oh, I stumbled over https://searchfox.org/mozilla-esr68/source/security/mac/hardenedruntime/codesign.bash by chance which seems to have additional useful information.

comment:11 Changed 5 weeks ago by mcs

Today, Kathy and I downloaded https://dist.torproject.org/torbrowser/9.0a4/TorBrowser-9.0a4-osx64_en-US.dmg, extracted Tor Browser.app, and experimented with Apple's notarization process. Using our own "Developer ID Application" signing key, we re-signed the app bundle like this:

CERT="Developer ID Application: ..."
ENTITLEMENTS=/path/to/production.entitlements.xml
codesign -vvv --deep -o runtime --entitlements "$ENTITLEMENTS" --timestamp -f -s "$CERT" "Tor Browser.app/"

(compared to the usual Tor Browser Gatekeeper signing we added -o runtime to enable the hardened runtime, we added the Firefox 68 entitlements file, and we enabled timestamping).

Then we used Xcode 10.1's Application Loader tool to upload a .zip to Apple for notarization, like this:

rm -f tb.zip
zip -qr tb.zip "Tor Browser.app"
export PW="secret"
BUNDLEID="org.torproject.torbrowser"
xcrun altool --notarize-app -t osx -f tb.zip --primary-bundle-id "$BUNDLEID" -u ID -p @env:PW --output-format xml

Finally, we used altool to poll for the status of Apple's server-side notarization process, e.g.,

xcrun altool --notarization-info REQUESTID -u ID -p @env:PW --output-format xml

(REQUESTID is a GUID returned by the altool --notarize-app command).

The result was a "Package Invalid" error. Apple also provides an URL that points to a detailed log, and that shows a series of repeated issues (one for each of our binaries), e.g.,

"severity": "error",
"code": null,
"path": "tb.zip/Tor Browser.app/Contents/MacOS/firefox",
"message": "The binary uses an SDK older than the 10.9 SDK.",
"docUrl": null,
"architecture": "x86_64"
...

That is strange. We then used otool -l on macOS to look at the firefox binary. The output indeed indicates that the minimum macOS version and SDK version are not what we would expect:

cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.7
sdk 10.6

Kathy and I do not know enough about the cross-compile process to know where to look for a solution to this problem, but I think we need to fix this SDK issue before we can go further with notarization.

comment:13 Changed 5 weeks ago by cypherpunks

Where sdk is leaked from is still unknown: https://github.com/NixOS/nixpkgs/issues/21629

comment:14 in reply to:  11 ; Changed 4 weeks ago by gk

Replying to mcs:

[snip]

That is strange. We then used otool -l on macOS to look at the firefox binary. The output indeed indicates that the minimum macOS version and SDK version are not what we would expect:

cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.7
sdk 10.6

Kathy and I do not know enough about the cross-compile process to know where to look for a solution to this problem, but I think we need to fix this SDK issue before we can go further with notarization.

Hm, I think we don't do something differently to what Mozilla is using. So, what is the Firefox ESR 60 output here? And does that change with the patch for bug 1270217 landed in Firefox (that is with Firefox 62)? What does the firefox binary for ESR 68 say in that regard?

Meanwhile I've started a test build with the patch for bug 1270217 applied, in case that buys us something. I'll post a link to a bundle later when the build has finished.

Last edited 4 weeks ago by gk (previous) (diff)

comment:16 in reply to:  14 ; Changed 3 weeks ago by mcs

Replying to gk:

Hm, I think we don't do something differently to what Mozilla is using. So, what is the Firefox ESR 60 output here? And does that change with the patch for bug 1270217 landed in Firefox (that is with Firefox 62)? What does the firefox binary for ESR 68 say in that regard?

Here is the LC_VERSION_MIN_MACOSX info for various builds:

Buildmin OS versionSDK version
Firefox 60.8.0 ESR10.710.11
Firefox 68.0.1 ESR10.910.11
Firefox 62.010.910.11
Tor Browser 9.0a410.710.6
2019-07-28 Patched Tor Browser build10.910.6

Meanwhile I've started a test build with the patch for bug 1270217 applied, in case that buys us something. I'll post a link to a bundle later when the build has finished.

See the last row in the table above. The patch affects the minimum supported OS version but not the SDK version.

Kathy and I looked at the cctools code, and maybe the problem is that the SDK version is not defaulting to the correct value. There is code to pick it up from the SDK path, but our SDK path is just SDK/. Search for if -sdk_version not on command line, infer from -syslibroot within https://github.com/tpoechtrager/cctools-port/blob/8e9c3f2506b51cf56725eaa60b6e90e240e249ca/cctools/ld64/src/ld/Options.cpp to see the relevant code.

One solution is to leave our SDK directory name as MacOSX10.11.sdk. An alternative is to add -sdk_version 10.11 to the ld command.

By the way, we could not find an open source tool that dumps mach-o header fields like the macOS otool and objdump commands can.

comment:17 in reply to:  16 ; Changed 3 weeks ago by gk

Replying to mcs:

[snip]

Kathy and I looked at the cctools code, and maybe the problem is that the SDK version is not defaulting to the correct value. There is code to pick it up from the SDK path, but our SDK path is just SDK/. Search for if -sdk_version not on command line, infer from -syslibroot within https://github.com/tpoechtrager/cctools-port/blob/8e9c3f2506b51cf56725eaa60b6e90e240e249ca/cctools/ld64/src/ld/Options.cpp to see the relevant code.

One solution is to leave our SDK directory name as MacOSX10.11.sdk. An alternative is to add -sdk_version 10.11 to the ld command.

Thanks for the investigation! I think I have a fix for that which follows Mozilla leaving the SDK directory name as MacOSX10.11.sdk:

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_2-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_2-osx64_en-US.dmg.asc

Is Apple happier with that one? (Note: that's without the patch for bug 1270217 which we might need as well) If we are good I'll open a child bug just for the SDK issue and get that one fixed there.

By the way, we could not find an open source tool that dumps mach-o header fields like the macOS otool and objdump commands can.

That would be unfortunate, so I looked a bit around. It turns out that you are already building such a tool while building the macOS bundles :) : x86_64-apple-darwin11-otool (and a bunch of similar tools) gets built when assembling the macosx-toolchain and works for me for the purposes at hand (you need to put the path to clang/lib into LD_LIBRARY_PATH to find libc++abi.so.1).

comment:18 in reply to:  17 ; Changed 3 weeks ago by cypherpunks

Replying to gk:

Is Apple happier with that one?

Apple is happier with https://bugzilla.mozilla.org/show_bug.cgi?id=1471004#c41

(Note: that's without the patch for bug 1270217 which we might need as well)

It's preferred, but only SDK is required (tested by other apps).

x86_64-apple-darwin11-otool (and a bunch of similar tools) gets built when assembling the macosx-toolchain and works for me for the purposes at hand

So, what sdk did it show? :)

comment:19 in reply to:  18 Changed 3 weeks ago by gk

Replying to cypherpunks:

Replying to gk:

Is Apple happier with that one?

Apple is happier with https://bugzilla.mozilla.org/show_bug.cgi?id=1471004#c41

(Note: that's without the patch for bug 1270217 which we might need as well)

It's preferred, but only SDK is required (tested by other apps).

Maybe preferred, but I think we should stick to what Mozilla is doing with esr60 and so far it does not look like they are backporting the fix for bug 1270217 to it.

x86_64-apple-darwin11-otool (and a bunch of similar tools) gets built when assembling the macosx-toolchain and works for me for the purposes at hand

So, what sdk did it show? :)

10.11.

comment:20 in reply to:  17 ; Changed 3 weeks ago by mcs

Replying to gk:

Thanks for the investigation! I think I have a fix for that which follows Mozilla leaving the SDK directory name as MacOSX10.11.sdk:

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_2-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_2-osx64_en-US.dmg.asc

Is Apple happier with that one? (Note: that's without the patch for bug 1270217 which we might need as well) If we are good I'll open a child bug just for the SDK issue and get that one fixed there.

It is almost perfect. Apple complains about the following three files which have sdk 10.7 in the mach-o header:

Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client-torbrowser
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/obfs4proxy

Is the build process different for those binaries?

After finding those anomolies, Kathy and I did some more checking and found that all of our other binaries have (min) version 10.7 and sdk 10.11 (as expected) with the exception of two files. The following have sdk 10.11 (good) but for some reason have (min) version 10.11 (possibly bad):

Tor Browser.app/Contents/MacOS/Tor/libevent-2.1.6.dylib
Tor Browser.app/Contents/MacOS/Tor/tor.real

That won't break notarization, but I wonder if it will cause problems when trying to run on older macOS systems.

In any case, after Kathy and I removed meek-client, meek-client-torbrowser, and obfs4proxy we followed the steps from comment:11 again and notarization (finally) succeeded. There is one more required step to avoid macOS having to contact Apple to check notarization status every time the app is opened: stapling. This also requires Internet access but at least it did its job quickly:

xcrun stapler staple Tor\ Browser.app

The above command adds one new file to the app bundle (Tor Browser.app/Contents/CodeResources) and makes no other changes. Near the end of the following article there is some info about network access requirements:

https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution/customizing_the_notarization_workflow

Now we can check the app status and note that the source is a "Notarized Developer ID":

spctl -av Tor\ Browser.app
Tor Browser.app: accepted
source=Notarized Developer ID

Finally, the prompt displayed when a user tries to open a downloaded app ("Tor Browser.app is an app downloaded from the Internet. Are you sure you want to open it?") now includes the "Apple checked it for malicious software and none was detected" text as expected for a notarized app.

By the way, we could not find an open source tool that dumps mach-o header fields like the macOS otool and objdump commands can.

That would be unfortunate, so I looked a bit around. It turns out that you are already building such a tool while building the macOS bundles :) : x86_64-apple-darwin11-otool (and a bunch of similar tools) gets built when assembling the macosx-toolchain and works for me for the purposes at hand (you need to put the path to clang/lib into LD_LIBRARY_PATH to find libc++abi.so.1).

Nice! (and good to know).

comment:21 in reply to:  20 ; Changed 3 weeks ago by gk

Replying to mcs:

[snip]

It is almost perfect. Apple complains about the following three files which have sdk 10.7 in the mach-o header:

Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client-torbrowser
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/obfs4proxy

Is the build process different for those binaries?

Those are pure go builds. Thus, I suspect we need to find some magic way to set the required flags when CGO is not involved. Hrm.

After finding those anomolies, Kathy and I did some more checking and found that all of our other binaries have (min) version 10.7 and sdk 10.11 (as expected) with the exception of two files. The following have sdk 10.11 (good) but for some reason have (min) version 10.11 (possibly bad):

Tor Browser.app/Contents/MacOS/Tor/libevent-2.1.6.dylib
Tor Browser.app/Contents/MacOS/Tor/tor.real

That won't break notarization, but I wonder if it will cause problems when trying to run on older macOS systems.

Nice catch! That can get solved by setting the proper MACOSX_DEPLOYMENT_TARGET version. To answer my IRC question: we were not affected by that previously as we did not get the proper SDK version but fixing that and not setting MACOSX_DEPLOYMENT_TARGET just takes the SDK version as min OS version it seems.

(Oh and I finally realized that my concerns about snowflake not running on 10.9 were a non-issue as we set the corresponding -mmacosx-version-min=10.7 flag.)

comment:22 in reply to:  21 Changed 3 weeks ago by gk

Replying to gk:

Replying to mcs:

[snip]

It is almost perfect. Apple complains about the following three files which have sdk 10.7 in the mach-o header:

Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client-torbrowser
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/obfs4proxy

Is the build process different for those binaries?

Those are pure go builds. Thus, I suspect we need to find some magic way to set the required flags when CGO is not involved. Hrm.

I suspect that's because we use internal linking mode here which hardcodes 10.7: https://go-review.googlesource.com/c/go/+/18588/

comment:24 in reply to:  23 Changed 3 weeks ago by gk

Replying to cypherpunks:

https://github.com/golang/go/issues/30488

Yeah, I guess we go here with bumping the go version instead of trying to get external linker mode linking working for obfs4 and meek.

comment:25 Changed 3 weeks ago by cypherpunks

The security updates for Go 1.12 turn out to be as important as for Firefox. Go > 1.12.5 is required to fix the issue and 1.12.7 is available.

comment:26 Changed 3 weeks ago by gk

Alright: mcs/brade: could you give

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_3-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_3-osx64_en-US.dmg.asc

a round of testing? I think I've fixed all the blockers for notarization in our build system now. making sure the whole bundle is still running (even on a system < 10.11) would be neat as well. :)

comment:27 in reply to:  26 ; Changed 3 weeks ago by mcs

Replying to gk:

Alright: mcs/brade: could you give

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_3-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_3-osx64_en-US.dmg.asc

a round of testing? I think I've fixed all the blockers for notarization in our build system now. making sure the whole bundle is still running (even on a system < 10.11) would be neat as well. :)

The good news is that the notarization process works with that build.

I am not sure it matters, but the following three files have different min OS version and SDK values in their mach-o headers:

Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client-torbrowser
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/obfs4proxy

While all other binaries have min OS version 10.7 and SDK 10.11, the above three files have 10.9 and 10.9.

We did find one blocker though: when we tested on a macOS 10.9.x system we experienced #26876 again. Since Tor Browser 9.0a4 does not have this problem, there must besome difference in how tor.real is built for your test builds.

comment:28 in reply to:  27 ; Changed 3 weeks ago by gk

Replying to mcs:

Replying to gk:

Alright: mcs/brade: could you give

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_3-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_3-osx64_en-US.dmg.asc

a round of testing? I think I've fixed all the blockers for notarization in our build system now. making sure the whole bundle is still running (even on a system < 10.11) would be neat as well. :)

The good news is that the notarization process works with that build.

\o/

I am not sure it matters, but the following three files have different min OS version and SDK values in their mach-o headers:

Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/meek-client-torbrowser
Tor Browser.app/Contents/MacOS/Tor/PluggableTransports/obfs4proxy

While all other binaries have min OS version 10.7 and SDK 10.11, the above three files have 10.9 and 10.9.

That's expected with the switch to Go >= 1.12.6 and is okay I think

We did find one blocker though: when we tested on a macOS 10.9.x system we experienced #26876 again. Since Tor Browser 9.0a4 does not have this problem, there must besome difference in how tor.real is built for your test builds.

If you look at bug_30126_v2 in my tor-browser-build repo you see the changes I am doing. For tor.real in particular I set export MACOSX_DEPLOYMENT_TARGET=10.7. _That_ alone should not cause this issue. A couple of possible options come to mind here: 1) We might need to set it earlier, that is before the configure step. 2) It's somehow caused by us setting the SDK option to 10.11 now.

I suspect you get the same problem with the build 2 in comment:17?

Last edited 3 weeks ago by gk (previous) (diff)

comment:29 in reply to:  28 Changed 3 weeks ago by teor

Replying to gk:

Replying to mcs:

We did find one blocker though: when we tested on a macOS 10.9.x system we experienced #26876 again. Since Tor Browser 9.0a4 does not have this problem, there must besome difference in how tor.real is built for your test builds.

If you look at bug_30126_v2 in my tor-browser-build repo you see the changes I am doing. For tor.real in particular I set export MACOSX_DEPLOYMENT_TARGET=10.7. _That_ alone should not cause this issue. A couple of possible options come to mind here: 1) We might need to set it earlier, that is before the configure step. 2) It's somehow caused by us setting the SDK option to 10.11 now.

I suspect you get the same problem with the build 2 in comment:17?

Tor's workaround for Apple's header declaration bug happens in the configure step, so you'll need to export MACOSX_DEPLOYMENT_TARGET=10.7 before tor's configure runs,

comment:30 in reply to:  28 Changed 3 weeks ago by mcs

Replying to gk:

I suspect you get the same problem with the build 2 in comment:17?

Yes.

comment:32 in reply to:  31 ; Changed 2 weeks ago by mcs

Replying to gk:

Alright: mcs/brade how does:

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_4-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_4-osx64_en-US.dmg.asc

That build fixes the problem with tor.real. With tor working, we could test further on macOS 10.9. Unfortunately, meek is not working. Trying to exec meek-client-torbrowser shows:

 dyld: Symbol not found: _unlinkat
  Referenced from: .../Tor Browser.app/Contents/MacOS/./Tor/PluggableTransports/meek-client-torbrowser
  Expected in: flat namespace

According to the unlink/unlinkat man page on macOS 10.14, "The unlinkat() system call appeared in OS X 10.10."

comment:33 in reply to:  32 ; Changed 2 weeks ago by gk

Cc: dcf added
Keywords: TorBrowserTeam201908 added; TorBrowserTeam201907 removed

Replying to mcs:

Replying to gk:

Alright: mcs/brade how does:

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_4-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_4-osx64_en-US.dmg.asc

That build fixes the problem with tor.real. With tor working, we could test further on macOS 10.9. Unfortunately, meek is not working. Trying to exec meek-client-torbrowser shows:

 dyld: Symbol not found: _unlinkat
  Referenced from: .../Tor Browser.app/Contents/MacOS/./Tor/PluggableTransports/meek-client-torbrowser
  Expected in: flat namespace

According to the unlink/unlinkat man page on macOS 10.14, "The unlinkat() system call appeared in OS X 10.10."

So, we have min OS version 10.9 and SDK version 10.9 but a syscall got used that appeared in 10.10? Weird.

comment:34 Changed 2 weeks ago by gk

mcs/brade: could you assemble a list of minimal requirements we have to have to get the notarization going, with the focus on what we'd need for our signing machine (plus a script or something you used).

Another thought I had: can we buy us some time if we pretend we have signed our stuff _before_ June 2019? IIRC the notarization requirement is only a requirement for binaries signed _after_ that threshold.

comment:35 in reply to:  33 Changed 2 weeks ago by teor

Replying to gk:

Replying to mcs:

That build fixes the problem with tor.real. With tor working, we could test further on macOS 10.9. Unfortunately, meek is not working. Trying to exec meek-client-torbrowser shows:

 dyld: Symbol not found: _unlinkat
  Referenced from: .../Tor Browser.app/Contents/MacOS/./Tor/PluggableTransports/meek-client-torbrowser
  Expected in: flat namespace

According to the unlink/unlinkat man page on macOS 10.14, "The unlinkat() system call appeared in OS X 10.10."

So, we have min OS version 10.9 and SDK version 10.9 but a syscall got used that appeared in 10.10? Weird.

In 10.10.0, Apple correctly declared unlinkat() as only available in 10.10 and later:

int	unlinkat(int, const char *, int) __OSX_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0);

https://opensource.apple.com/source/xnu/xnu-2782.1.97/bsd/sys/unistd.h.auto.html

In 10.9.5 and earlier, unlinkat() does not exist, and is not declared by Apple.

Maybe Go's bindings are wrong?

Last edited 2 weeks ago by teor (previous) (diff)

comment:36 in reply to:  32 ; Changed 6 days ago by gk

Replying to mcs:

Replying to gk:

Alright: mcs/brade how does:

https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_4-osx64_en-US.dmg
https://people.torproject.org/~gk/testbuilds/TorBrowser-tbb-nightly-30126_4-osx64_en-US.dmg.asc

That build fixes the problem with tor.real. With tor working, we could test further on macOS 10.9. Unfortunately, meek is not working. Trying to exec meek-client-torbrowser shows:

 dyld: Symbol not found: _unlinkat
  Referenced from: .../Tor Browser.app/Contents/MacOS/./Tor/PluggableTransports/meek-client-torbrowser
  Expected in: flat namespace

According to the unlink/unlinkat man page on macOS 10.14, "The unlinkat() system call appeared in OS X 10.10."

Okay, I am not convinced yet this is caused by the potential changes making notarization possible. mcs/brade: If I am reading your comments right, you did _not_ test whether meek as we ship it in 9.0a4 works on your 10.9 system? If so, could you do that just to be sure whether the above problem is actually a new bug caused by my patch?

I suspect the underlying problem is a change in Go 1.12 where Go is starting to use libSystem.dylib etc. for syscalls: https://github.com/golang/go/issues/17490. But in that case we should hit the issue in our already existing bundles. (The mac-ports folks had to work around that e.g. in https://trac.macports.org/ticket/58138).

comment:37 in reply to:  36 ; Changed 3 days ago by mcs

Replying to gk:

Okay, I am not convinced yet this is caused by the potential changes making notarization possible. mcs/brade: If I am reading your comments right, you did _not_ test whether meek as we ship it in 9.0a4 works on your 10.9 system? If so, could you do that just to be sure whether the above problem is actually a new bug caused by my patch?

I suspect the underlying problem is a change in Go 1.12 where Go is starting to use libSystem.dylib etc. for syscalls: https://github.com/golang/go/issues/17490. But in that case we should hit the issue in our already existing bundles. (The mac-ports folks had to work around that e.g. in https://trac.macports.org/ticket/58138).

I don't think we tested 9.0a4 + meek on macOS 10.9.x but we can do so.

comment:38 in reply to:  37 ; Changed 3 days ago by mcs

Replying to mcs:

I don't think we tested 9.0a4 + meek on macOS 10.9.x but we can do so.

As you predicted, TB 9.0a4 has the same issue: meek-client-torbrowser expects to find_unlinkat, but that entry point does not exist on macOS 10.9.x.

comment:39 in reply to:  38 Changed 3 days ago by gk

Replying to mcs:

Replying to mcs:

I don't think we tested 9.0a4 + meek on macOS 10.9.x but we can do so.

As you predicted, TB 9.0a4 has the same issue: meek-client-torbrowser expects to find_unlinkat, but that entry point does not exist on macOS 10.9.x.

That's #31464 now.

comment:40 Changed 3 hours ago by ha

Are the entitlement files Tor plans to use available online somewhere to look at?

If you're using the Firefox production entitlements as a starting point, you might be able to change some rules to be more restrictive.

Assuming Tor only loads shared libraries signed by Tor or Apple, you should be able to set the disable library validation entitlement[1] to false. Firefox needs to load libraries signed by Adobe and Google for Flash and Widevine video decoding respectively.

com.apple.security.cs.disable-library-validation=false

In Firefox, we had to recently set this[2] to true because some WebExtensions using the native message API relied on helper applications that use Apple Events. I suspect Tor wouldn't need this and could set the entitlement to false.

com.apple.security.automation.apple-events=false

  1. https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_disable-library-validation
  2. https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_automation_apple-events
Last edited 3 hours ago by ha (previous) (diff)
Note: See TracTickets for help on using tickets.