BusyBox-style bundling of Go programs can save space
Mike suggested that compiling all the Go programs into one executable à la BusyBox could save space because there would be only one static copy of the Go runtime.
Activity
Here's a script that generates a multi-executable from Go programs. I tested it on obfs4proxy, meek-client, meek-client-torbrowser, and the new tor-fw-helper (#13338 (moved)). With these four programs, bundling saves this much in the linux build:
- uncompressed: saves 10 MB (16 MB goes to 5.8 MB)
- compressed: saves 0.7 MB (2.0 MB goes to 1.3 MB)
mkmulti is a shell script. Run it like this to create the source for a multi-executable:
export GOPATH=~/go PROGS="git.torproject.org/pluggable-transports/obfs4.git/obfs4proxy git.torproject.org/pluggable-transports/meek.git/meek-client git.torproject.org/pluggable-transports/meek.git/meek-client-torbrowser github.com/yawning/tor-fw-helper" go get -d $PROGS ./mkmulti multi $PROGSThe script writes a source file to multi/multi.go. Build it using the same flags we use in tor-browser-bundle.
cd multi go build -ldflags '-s'This is the multi-executable:
5992352 (5.8M) multiAnd these are the constituent programs compiled separately:
4531488 (4.4M) meek-client 2090368 (2.0M) meek-client-torbrowser 5074976 (4.9M) obfs4proxy 4603264 (4.5M) tor-fw-helper --------------------- 16300096 (16M) totalHere are the compressed sizes:
1263988 (1.3M) multi.tar.xz 2013728 (2.0M) separate.tar.xzHey, that's really neat.
Replying to dcf:
On linux and mac, we can use symbolic links to the executable. I don't know what is done on Windows. Maybe batch files, or stub programs that change argv[0] and then exec?
NTFS kind of sort of has symlinks but assuming that the target filesystem is NTFS instead of FAT32/exFAT will end in tears.
How about adding code to the begining of
main()that:- Looks at
os.Args[1]for--progname=<name>(Do this manually, and not with the flag package.)- If present (and
nameinprogName), do something like: {{{ // Rewrite argv[0], and strip out progname from the arguments, so that using the // 'flag' package doesn't break. newArgs := make([]string, 0, 1 + len(os.Args) - 2) newArgs = append(newArgs, progName) if len(os.Args) > 2 { newArgs = append(newArgs, os.Args[2:]...) } os.Args = newArgs }}}
- If present (and
Edit the Windows torrc as appropriate. It's a bit kludgey, but it probably beats batchfiles and stub executables, and it's relatively robust, and having that there is relatively harmless across all platforms (assuming that missing argument is treated as a non-fatal error).
- Looks at
Replying to yawning:
Hey, that's really neat.
Replying to dcf:
On linux and mac, we can use symbolic links to the executable. I don't know what is done on Windows. Maybe batch files, or stub programs that change argv[0] and then exec?
NTFS kind of sort of has symlinks but assuming that the target filesystem is NTFS instead of FAT32/exFAT will end in tears.
How about adding code to the begining of
main()that:- Looks at
os.Args[1]for--progname=<name>(Do this manually, and not with the flag package.)
Or just always switch on argv[1], and ignore argv[0]. For example,
ClientTransportPlugin obfs4 exec TorBrowser\Tor\PluggableTransports\multi obfs4proxyand do the same thing on all platforms:
ClientTransportPlugin obfs4 exec ./TorBrowser/Tor/PluggableTransports/multi obfs4proxy- Looks at
mentioned in issue #26043 (moved)
mentioned in issue #28205 (moved)
mentioned in issue #33047 (moved)
