Opened 5 years ago

Last modified 16 months ago

#13461 new enhancement

Point to Tor.framework in contrib, for iOS and macOS

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: easy, doc, osx, build, tor-client portability
Cc: nickm, rl1987@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by teor)

I've created an OS X Xcode project in Xcode 6.1, and I'm happy to share it with the community.

I'll post the branch as soon as I've added the changes file.

We should point to Tor.framework instead:
https://trac.torproject.org/projects/tor/ticket/13461#comment:30

Child Tickets

Change History (42)

comment:1 Changed 5 years ago by teor

Status: newneeds_review

Branch: osxbuild
Repository: ​​https://github.com/teor2345/tor.git

commit 2392be530a015642c648b0e28265f7ae1effe512
Author: teor <teor2345@…>
Date: Sat Oct 18 12:26:16 2014 +1100

Add an OS X Xcode build project in contrib/osxbuild.


Project requires ./configure to be run beforehand.
Builds x86-64 by default. YMMV.


Resolves issue 13461.

comment:2 Changed 5 years ago by Sebastian

that link seems to be dead?

comment:3 Changed 5 years ago by teor

From GitHub:

"One of our mostly harmless robots seems to think you are not a human.

Because of that, it's hidden your profile from the public. If you really are human, please contact support to have your profile reinstated.

We promise we won't require DNA proof of your humanity."

Such is the cost of anonymity. Hopefully it will all be sorted soon.

comment:4 Changed 5 years ago by teor

All sorted now - I have passed the turing test :-)

comment:5 Changed 5 years ago by Sebastian

How hard is it to make that project? What are the steps involved? And what does it help with?

comment:6 Changed 5 years ago by teor

It was surprisingly difficult to make the project, far more than I expected:

I had to run autoconf and configure, add the tor source files, change the build settings, then add the relevant libraries (it's configured for libraries in /opt/local, used by MacPorts). All but the first step had to be repeated for every executable (6 times).

I then created a few different "schemes" - runtime settings for tor that allow it to be launched using different settings (for example, torrc locations).

The Xcode project allows OS X users to use OS X-specific development tools, including graphical editing, build, static analysis, debugging, profiling, and git tools. In particular, I've found it far more difficult to use the debugger and clang static analysis without it.

comment:7 Changed 5 years ago by teor

I've edited the branch to remove two schemes that contained local paths. They won't be much use to anyone else.

comment:8 Changed 5 years ago by teor

It's also possible to add i386 and x64 build variants, which I haven't bothered to set up yet, as it would require multiple orconfig.h files (or maybe a bit of tricky undefine/redefine in the OS X project).

comment:9 Changed 5 years ago by nickm

I need to look at the patch, but I'm fine with this in principle... especially if you'd mark this as unsupported, and volunteer to maintain it going forward. :)

comment:10 Changed 5 years ago by teor

Everyone needs a pet lorax, Nick...

comment:11 Changed 5 years ago by nickm

Milestone: Tor: 0.2.6.x-final

comment:12 Changed 5 years ago by nickm

Sounds fine to me. Could you throw in the a README documenting:

  • What the xcode project is for
  • How to use it
  • How you made it
  • What to do to maintain it going forward
  • How to ask for help with it and how to submit fixes for it

I'd be happy to merge it then.

comment:13 Changed 5 years ago by rl1987

This does not seem to work on my system.

  • First, it fails to find libevent, specifically event2/util.h since I used Homebrew to install libevent. Fixed it manually by adding /usr/local/Cellar/libevent/2.0.21 (recursive) and /usr/local/Cellar/libevent/2.0.21/include) (recursive) to Library Search Paths and Header Search Paths respectively. Obviously this is not a solution for general case.
  • src/or/or_sha.i is generated by compilation process so you have to make the codebase for this file to appear.
  • ld: library not found for -lminiupnpc Seems to be related to miniupnp [1] library.

I think it would be worthwhile to research ways to generate Xcode project automatically, e.g. by doing make xcodeproj. That way, we could account for machine-specific things, such as library paths.

[1] http://miniupnp.free.fr/

comment:14 Changed 5 years ago by rl1987

Cc: rl1987@… added

comment:15 Changed 5 years ago by teor

Status: needs_reviewneeds_revision

Yes, you're right - I should at the very least source the paths and libraries generated by configure.

I thought I had included a "make" step for some of the generated *.i files, but it's obviously not always working for all targets.

comment:16 Changed 5 years ago by nickm

It would be nice if this could be made to work on homebrew too, and if it has a clear upgrade path for when orconfig.h needs to change. (Also I want a pony. Neither of these is a requirement.)

comment:17 Changed 5 years ago by teor

Keywords: osx build want-pony added

You'd like orconfig.h to be rebuilt by the Xcode project when the configure script's source files change? (I think the user will have to do this manually, unless the configure arguments are cached somewhere.)

Or you'd like the Xcode project to be updated when orconfig.h changes?

I'm just trying to work out the size and colour of the pony before I spend time breeding one.

comment:18 Changed 5 years ago by nickm

I'd like to have an easy way to make sure that the xcode project won't lag behind changes in the configure script and the makefiles. That could be as simple as a script to regenerate whatever we need, or a script that detects that it's out of sync, or something like that. It would be even better if it were automated, but being automated is not as important as being and easy to diagnose and fast to fix. Otherwise I foresee many wasted developer hours spent investigating "why did Tor stop working" and coming up with the answer "the Xcode files are out of date again".

comment:19 Changed 5 years ago by teor

I'm starting to feel like I'm actually breeding a lorax :-)

I see several complementary solutions:

  1. Dynamically source as many variables in the project from the configure/makefiles
  2. Create Xcode target(s) that just run the makefile, allowing a quick comparison ("tor stopped working in Xcode? Try running the makefile target.)
  3. Automatically rebuild as much as possible
  4. Detect when anything else is out of date

cmake generates generic Xcode files already, so it can't be that hard to do project-specific ones.

comment:20 Changed 5 years ago by nickm

Wow; that does actually sound like a lot of work. Maybe it should be a separate ticket from this one?

comment:21 Changed 5 years ago by teor

It doesn't seem that difficult now I'm most of the way through...

The current status is:

  1. Almost done, need to work out how to get header & library paths from configure/Makefile
  1. Done, Xcode has an "External Build System" target for Makefiles, I have created two targets, one which uses the existing configure invocation, and one which runs configure inside Xcode.
  1. Almost done, can build all generated .i files and architecture-specific orconfig.h (x86_64 & i386) as required. Dependencies still need to be sorted out, but that's nice-to-have for build speed. (Who wants to run configure every time they build?)
  1. The most common case here is a new file, which can be detected automatically using find and grep, then manually added to the tor, test, bench, test-bt-cl and/or test-ntor-cl targets as appropriate. We can even print out the relevant Makefile variables to assist the dev. Instructions for this aren't hard.

I'm just working through testing the new project file - I'll push it to GitHub once it's done.

comment:22 Changed 5 years ago by teor

Owner: set to teor
Status: needs_revisionassigned

Status Update:

  1. TODO - move configure arguments out of the project (into an editable script?) and source header & library paths from configure/Makefile.
  2. Done
  3. Done, including architecture-specific includes for curve25519-donna[-64].c, and dependencies between the various components. The project uses configure to generate orconfig.h for x86_64 & i386, make to build micro-revision.i, common_sha1.i, and or_sha1.i, and Xcode to build universal binaries for tor, bench, and test*
  4. TODO - check for new files / print out Makefile variables
  5. Sort out the paths so that test/make can find test-bt-cl, test-ntor-cl, and test-child (these tests currently fail)
  6. Write README & changes file

It's not going to work with HomeBrew until step 1 is completed, but I've pushed it to github.

comment:23 Changed 5 years ago by teor

Oh, the i386 build will crash in donna unless one or more of the following compiler arguments are removed: -fsanitize=undefined-trap -fsanitize-undefined-trap-on-error -ftrapv

I'm working on a patch for curve25519-donna.c similar to the ed25519 SHL8/32/64 safe left shift changes.

comment:24 Changed 5 years ago by teor

Affected by #13540, as we replace the orconfig.h file with a short file that conditionally includes either the x86_64 or i386 config.

Resolution: concatenate the files with the appropriate #ifs in between, rather than including them.

comment:25 Changed 5 years ago by nickm

Milestone: Tor: 0.2.6.x-finalTor: 0.2.7.x-final

comment:26 Changed 5 years ago by nickm

Keywords: 027-triaged-1-out added

Marking triaged-out items from first round of 0.2.7 triage.

comment:27 Changed 5 years ago by nickm

Milestone: Tor: 0.2.7.x-finalTor: 0.2.???

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

comment:28 Changed 3 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:29 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:30 Changed 3 years ago by teor

Keywords: doc added
Severity: Normal
Version: Tor: unspecified

Instead of adding a macOS project ourselves, we should reference Tor.framework in our documentation:
https://github.com/icepa/Tor.framework

It sets up macOS and iOS builds, using git submodules for tor, libevent, and openssl.
It uses shell scripts to build the submodules, and then packages them into a framework.

It's already used by Onion Browser and iCepa (iOS Tor as a system-wide VPN).

comment:31 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:32 Changed 3 years ago by nickm

Keywords: 027-triaged-in added

comment:33 Changed 3 years ago by nickm

Keywords: 027-triaged-in removed

comment:34 Changed 3 years ago by nickm

Keywords: 027-triaged-1-out removed

comment:35 Changed 3 years ago by nickm

Keywords: tor-client needs-osx-knowldge portability added; want-pony removed
Status: assignednew

comment:36 Changed 2 years ago by teor

Owner: teor deleted
Status: newassigned

Disowning tickets I don't intend to work on in the next 6 months.

comment:37 Changed 2 years ago by teor

Status: assignednew

Mark all tickets that are assigned to nobody as "new".

comment:38 Changed 17 months ago by teor

Description: modified (diff)
Keywords: easy added; needs-osx-knowldge removed
Summary: Add OS X Xcode Project to contribPoint to Tor.framework in contrib, for iOS and macOS

comment:39 Changed 16 months ago by rl1987

Not sure a mention of Tor.framework belongs in contrib or doc/contrib... I would rather close it as wontfix, as nothing really came off the efforts here.

Alternatively, we can look into: https://github.com/yonaskolb/XcodeGen .

comment:40 in reply to:  39 Changed 16 months ago by teor

Replying to rl1987:

Not sure a mention of Tor.framework belongs in contrib or doc/contrib... I would rather close it as wontfix, as nothing really came off the efforts here.

I'm not sure I understand what you mean.
Tor.framework is actively used in OnionBrowser and various other projects.

Alternatively, we can look into: https://github.com/yonaskolb/XcodeGen .

Good idea! If I get some time I'll have a look at it.

comment:41 Changed 16 months ago by rl1987

I mean, we generally don't link to community projects in any documentation that we have inside core tor repo. We probably should be linking it from somewhere in our wiki.

comment:42 in reply to:  41 Changed 16 months ago by teor

Replying to rl1987:

I mean, we generally don't link to community projects in any documentation that we have inside core tor repo. We probably should be linking it from somewhere in our wiki.

We bundle a win32 build project in:
https://github.com/teor2345/tor/tree/master/contrib/win32build

My suggestion is that instead of bundling a project file for macOS Xcode, we could tell people to use Tor.framework.

Note: See TracTickets for help on using tickets.