Opened 3 months ago

Closed 6 weeks ago

Last modified 6 weeks ago

#33522 closed enhancement (wontfix)

Add iOS support in your CI

Reported by: tla Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords:
Cc: ahf, gaba Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

AFAIK, Tor.framework is the most advanced library to package Tor for iOS.

You can find it here:
https://github.com/iCepa/Tor.framework

It has a Travis-CI configuration, which I just updated to work on the latest macOS/Xcode image:

https://github.com/iCepa/Tor.framework/blob/master/.travis.yml

Currently, we have issues in getting past Tor 0.4.0.6 on iOS. When I try to use a newer core, I get this error message:

Unknown type name 'dispatch_queue_t'

in CFStream of Apple's CoreFoundation framework.

But "dispatch_queue_t" is actually a valid symbol from Apple's Foundation libraries.

So somehow, it gets cancelled out through something which changed in Tor recently.

We're really stuck here, so it would be a huge help, if you could add Tor.framework to your Continuous Integration so you'd be able to support us here and detect problems earlier in the future.

For your purposes, there's no need to have Macs and Xcode.
The most important files to control the build are these shell scripts:

https://github.com/iCepa/Tor.framework/blob/master/Tor/tor.sh
https://github.com/iCepa/Tor.framework/blob/master/Tor/libevent.sh
https://github.com/iCepa/Tor.framework/blob/master/Tor/openssl.sh
https://github.com/iCepa/Tor.framework/blob/master/Tor/xz.sh

Tor and the other libraries come in via Git submodules:
https://github.com/iCepa/Tor.framework/blob/master/.gitmodules

So the dev process on your side would be:

Update submodules to your latest state, let it build in Travis. If broken, fix above build scripts. Re-run through Travis.

Could that work for you?

Child Tickets

Change History (8)

comment:1 Changed 3 months ago by nickm

Keywords: 042-backport 043-should added
Milestone: Tor: 0.4.3.x-final

comment:2 in reply to:  description Changed 7 weeks ago by teor

Hi, I'm one of the maintainers of Tor's CI.

There are two very different issues in this ticket, so I'll try to address them separately:

Replying to tla:

It has a Travis-CI configuration, which I just updated to work on the latest macOS/Xcode image:

https://github.com/iCepa/Tor.framework/blob/master/.travis.yml

Currently, we have issues in getting past Tor 0.4.0.6 on iOS. When I try to use a newer core, I get this error message:

Unknown type name 'dispatch_queue_t'

in CFStream of Apple's CoreFoundation framework.

But "dispatch_queue_t" is actually a valid symbol from Apple's Foundation libraries.

So somehow, it gets cancelled out through something which changed in Tor recently.

This looks like a bug in tor's code, or perhaps in the Tor.framework embedding scripts.

We'd be happy to help you diagnose this issue.

Can you tell us the first release that has this issue? Is it 0.4.1, 0.4.2, or 0.4.3 ?
Have you done a git bisect, to track down the commit that introduced the issue?

Let's open a separate ticket, so we can fix this bug in tor's code.
Or help you find a workaround when you embed tor.

We're really stuck here, so it would be a huge help, if you could add Tor.framework to your Continuous Integration so you'd be able to support us here and detect problems earlier in the future.

For your purposes, there's no need to have Macs and Xcode.
The most important files to control the build are these shell scripts:

https://github.com/iCepa/Tor.framework/blob/master/Tor/tor.sh
https://github.com/iCepa/Tor.framework/blob/master/Tor/libevent.sh
https://github.com/iCepa/Tor.framework/blob/master/Tor/openssl.sh
https://github.com/iCepa/Tor.framework/blob/master/Tor/xz.sh

Tor and the other libraries come in via Git submodules:
https://github.com/iCepa/Tor.framework/blob/master/.gitmodules

So the dev process on your side would be:

Update submodules to your latest state, let it build in Travis. If broken, fix above build scripts. Re-run through Travis.

Could that work for you?

This is a CI configuration change request for Tor.

We already struggle with the speed of Tor's CI.
We build about 10 jobs per branch, and we support 4-6 branches.

We've recently added stem and chutney tests to our CI, and it's taken a lot of time to diagnose issues.
When stem or chutney have failures, Tor developers have to check every CI failures.
And volunteers get very confused by the failures.

So I don't think we can maintain Tor.framework in Tor's CI.

I have an alternative proposal, based on the way that chutney and sbws do CI:

Tor.framework adds a CI job for each supported Tor maint-* branch.
You don't have to build every release, just the ones you want to support.

Here's a list of the releases we support:

Maybe you could skip our long-term support release (0.3.5), and start with:

  • maint-0.4.1
  • maint-0.4.2
  • maint-0.4.3
  • master

Then, when one of our releases fails to build, you can log a bug in our bug tracker, and we'll fix it.

You can add a cron job for your Travis CI jobs, so they run every day (or week), even if Tor.framework hasn't had any code changes.

Here's how that looks for chutney and sbws:

They use tor's pre-compiled nightly binaries. But you'd need to use git branches instead.

What do you think?

comment:3 Changed 7 weeks ago by teor

Keywords: 042-backport 043-should removed
Milestone: Tor: 0.4.3.x-finalTor: unspecified

I opened #33837 for the compilation error.

Let's focus on CI in this ticket.

comment:4 Changed 6 weeks ago by tla

Yesterday, nickm was so friendly to look at our code and fix the issue.

I could finally release a new version containing the latest stable Tor.

We don't support multiple Tor releases, just the latest one and we only build a create a new release on demand. So your suggestion doesn't make a lot of sense for us in the current situation.

However, nickm also advised other improvements for our build scripts, so we can hopefully reach better stability in the build process in the future.

Thanks for all the support!

You can close this ticket now, if you want!

comment:5 in reply to:  4 Changed 6 weeks ago by arma

Replying to tla:

However, nickm also advised other improvements for our build scripts, so we can hopefully reach better stability in the build process in the future.

Thanks for all the support!

You can close this ticket now, if you want!

Hi tla,

I've just made you a more special person on trac, so you should be able to close and open and modify tickets now. Hopefully that will make your trac interactions more fun. :) Thanks!

comment:6 Changed 6 weeks ago by tla

Resolution: wontfix
Status: newclosed

Oh, wow. Thanks a lot!

comment:7 in reply to:  4 Changed 6 weeks ago by teor

Replying to tla:

We don't support multiple Tor releases, just the latest one and we only build a create a new release on demand. So your suggestion doesn't make a lot of sense for us in the current situation.

That's understandable.

If you would like to find issues in tor before we do a stable release, please run a weekly CI cron job with our latest alpha or master branches.

If you can't do that, that's ok. But if you discover a major issue that needs a big fix, we might not be able to backport the change. So you might have to skip one or two tor releases.

comment:8 Changed 6 weeks ago by tla

So you might have to skip one or two tor releases.

Oh, we're not worried about that. We're not releasing every Tor release anyway.

Note: See TracTickets for help on using tickets.