Opened 3 years ago

Last modified 13 months ago

#20254 new enhancement

Update marsigning-check.sh to cope with signed OS X MAR files

Reported by: gk Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Quality Assurance and Testing Version:
Severity: Normal Keywords: tbb-rbm, GeorgKoppen201806, TorBrowserTeam201806
Cc: boklm, brade, mcs Actual Points:
Parent ID: #18925 Points:
Reviewer: Sponsor:

Description (last modified by gk)

Now that the fix for #19410 landed our marsigning-check.sh script can't check the correctness of our MAR file signatures easily anymore as we use the SHA256 sums of the unsigned MAR files currently. We should adapt the script.

Child Tickets

Change History (33)

comment:1 Changed 3 years ago by gk

Description: modified (diff)

comment:2 Changed 2 years ago by gk

Keywords: TorBrowserTeam201701 added

comment:3 Changed 2 years ago by gk

Cc: boklm brade mcs added

comment:4 Changed 2 years ago by gk

Here comes an update on where I am right now. Input is highly appreciated:

To strip the code signature off a macOS binary the following things must be done:

1) Adjust the number of commands in the header (- LC_CODE_SIGNATURE)
2) Adjust the size of the commands in the header (- 16 as this is the size of LC_CODE_SIGNATURE)
3) Remove the LC_CODE_SIGNATURE load command
4) Adapt __LINKEDIT segment and respective load command

Doing 1)-3) is not overly complicated but adapting the __LINKEDIT segment properly turns out to be tricky. During code signing the signature is appended to the __LINKEDIT segment. When stripping the signature some approaches just overwrite the signature with 0-bytes. That's not working for us as we want to restore the original binary to be able to compare the SHA256 sums. But, unfortunately, removing the signature from the __LINKEDIT segment is not enough to achieve this. That's because the segemt is padded to be aligned with 0x10 if needed before the signature gets added. E.g. here is the relevant part of a library's __LINKEDIT segment without the code signature:

000034a0: 352f2f3a 35343136 00003234 00000000  ://5614542......
000034b0: 00000000                             ....

While it looks like this after code signing:

000034a0: 352f2f3a 35343136 00003234 00000000  ://5614542......
000034b0: 00000000 00000000 00000000 00000000  ................

So, the question is: How do we find out how many padding bytes got added during the code signing (and need now get removed)? A naive approach looking at the above hexdump output would be: "Leave 16 0-bytes and remove the remaining ones as padding bytes". But that does not work as there are binaries where the __LINKEDIT segment ends with less than 16 0-Bytes.

Three ways forward come to mind:

a) Align the files to be code-signed to 0x10 (+ adapt the size of the symbol table accordingly which is usually the last section in the __LINKEDIT segment) before starting the signing process.
We are doing something similar with our .exe installers (see: #15539) already which is working pretty well. Additionally, as we need to code-sign the binaries anyway one could argue it's perfectly fine to do the padding during the build.

b) Find out if there is the amount of 0-bytes at the end of the __LINKEDIT segment follows a pattern we could use to reliably strip the padding after removing the signature.

c) Record the size (or laste X bytes or) of all OS X binaries we ship during the build and make that available. The script removing the signatures could then consult that information when stripping the signature. This might not work for incremental MAR files as expected. I have not checked that yet.

Thoughts? Better alternatives? I find a) scary, have no much hope for b) and dislike c) so far.

General material about code signing on macOS:

https://developer.apple.com/library/content/technotes/tn2206/_index.html

(This document has the following fun Q&A :) ):

I wrote some data to the Mach-O file before signing. Is that allowed?

No. Do not tamper with Mach-O files, outside of using macOS build tools and Xcode workflows.

Material about the Mach-O file format:

https://lowlevelbits.org/parsing-mach-o-files/
https://github.com/aidansteele/osx-abi-macho-file-format-reference

comment:5 Changed 2 years ago by mcs

Kathy and I cannot think of a better approach than a) or c). I like the elegance of a) but I wonder why Apple warns developers not to "tamper" with Mach-O files. So maybe c) is the best solution, even if having a file that records the "before signing" length of each Mach-O file is ugly.

comment:6 in reply to:  5 Changed 2 years ago by gk

Replying to mcs:

Kathy and I cannot think of a better approach than a) or c). I like the elegance of a) but I wonder why Apple warns developers not to "tamper" with Mach-O files.

I think that's the usual "There be dragons!1!!" scaremongering. :) So, it might be a) then? Exciting!

comment:7 Changed 2 years ago by gk

Keywords: TorBrowserTeam201702 added; TorBrowserTeam201701 removed

Moving our tickets to Feb 2017.

comment:8 Changed 2 years ago by gk

Keywords: TorBrowserTeam201707 GeorgKoppen201707 added; TorBrowserTeam201702 removed

comment:9 Changed 2 years ago by gk

Parent ID: #18925

comment:10 Changed 23 months ago by gk

Keywords: TorBrowserTeam201708 added; TorBrowserTeam201707 removed

Moving our Tickets to August.

comment:11 Changed 23 months ago by gk

Keywords: GeorgKoppen201708 added; GeorgKoppen201707 removed

Moving my tickets to August.

comment:12 Changed 22 months ago by gk

Keywords: GeorgKoppen201709 added; GeorgKoppen201708 removed

Moving my tickets to the new month.

comment:13 Changed 22 months ago by gk

Keywords: TorBrowserTeam201709 added; TorBrowserTeam201708 removed

Items for September 2017.

comment:14 Changed 22 months ago by gk

Keywords: tbb-rbm added; tbb-gitian removed

Moving over to rbm

comment:15 Changed 21 months ago by gk

Keywords: TorBrowserTeam201710 added; TorBrowserTeam201709 removed

Items for October 2017

comment:16 Changed 21 months ago by gk

Keywords: GeorgKoppen201710 added; GeorgKoppen201709 removed

comment:17 Changed 20 months ago by gk

Keywords: GeorgKoppen201711 added; GeorgKoppen201710 removed

Moving my tickets to November.

comment:18 Changed 20 months ago by gk

Keywords: TorBrowserTeam201711 added; TorBrowserTeam201710 removed

Moving tickets over to November.

comment:19 Changed 19 months ago by gk

Moving tickets to December 2017

comment:20 Changed 19 months ago by gk

Keywords: TorBrowserTeam201712 added; TorBrowserTeam201711 removed

Moving tickets to December 2017, for realz.

comment:21 Changed 19 months ago by gk

Keywords: GeorgKoppen201712 added; GeorgKoppen201711 removed

Moving my tickets to December.

comment:22 Changed 18 months ago by gk

Keywords: GeorgKoppen201801 added; GeorgKoppen201712 removed

Moving my tickets to 2018

comment:23 Changed 18 months ago by gk

Keywords: TorBrowserTeam201801 added; TorBrowserTeam201712 removed

Moving tickets to 2018.

comment:24 Changed 17 months ago by gk

Keywords: GeorgKoppen201802 added; GeorgKoppen201801 removed

Moving my tickets to Feb.

comment:25 Changed 17 months ago by gk

Keywords: TorBrowserTeam201802 added; TorBrowserTeam201801 removed

Moving tickets to Feb

comment:26 Changed 16 months ago by gk

Keywords: GeorgKoppen201803 added; GeorgKoppen201802 removed

Moving my tickets to March.

comment:27 Changed 16 months ago by gk

Keywords: TorBrowserTeam201803 added; TorBrowserTeam201802 removed

Adding to our March plate.

comment:28 Changed 15 months ago by gk

Keywords: GeorgKoppen201804 added; GeorgKoppen201803 removed

Moving my tickets to April 2018

comment:29 Changed 15 months ago by gk

Keywords: TorBrowserTeam201804 added; TorBrowserTeam201803 removed

Moving our tickets to April.

comment:30 Changed 14 months ago by gk

Keywords: TorBrowserTeam201805 added; TorBrowserTeam201804 removed

Moving remaining tickets to May.

comment:31 Changed 14 months ago by gk

Keywords: GeorgKoppen201805 added; GeorgKoppen201804 removed

Moving my tickets.

comment:32 Changed 13 months ago by gk

Keywords: GeorgKoppen201806 added; GeorgKoppen201805 removed

Moving my tickets to June 2018.

comment:33 Changed 13 months ago by gk

Keywords: TorBrowserTeam201806 added; TorBrowserTeam201805 removed

Moving our tickets to June 2018

Note: See TracTickets for help on using tickets.