Add GitLab CI configs

changed milestone to %Tor: 0.3.2.x-final

added best-practice ci component::core tor/tor continuous-integration milestone::Tor: 0.3.2.x-final new-developers owner::hiro priority::high resolution::implemented review-group-22 review-group-23 reviewer::ahf reviewer::dgoulet severity::normal status::closed testing type::defect unit-testing labels

We'll need some VMs to setup runners on, and the GitLab official runner implementation is written in Go (so it might be a good idea to have an ansible script for deploying the VMs, since Go is a huge pain to set up on Debian).

Trac:
Cc: N/A to isis

Hi,

I can have VM deployed with docker as runner. I have a docker server VM that can do this for you guys. I can start to do some tests and send a patch with a .gitlab-ci that you can evaluate if you want.

Hi,

I am running a test setup for gitlab-ci w/ the tor repository. You can have a look at it here: https://oniongit.eu/hiro/tor/pipelines

Runs two task every time something is pushed to master or a branch. The first task test and build the branch. The second task merge updates from torgit.

Here is the gitlab-ci.yml that I am running: https://oniongit.eu/hiro/tor/blob/master/.gitlab-ci.yml

It's just a test, so I thought it was better to show it to you instead of sending over a merge request. The reason behind this is that you might want to customise your tasks differently and I only wanted to show what it can be done.

To sync the oniongit repo with the tor git repo you need to setup a key in oniongit w write access to the oniongit repo (so that it can push changes).

I can setup a number of shared or dedicated runners that can be used to build and test the repositories and branches that you need.

Let me know what you think.

Trac:
Status: new to needs_review

Trac:
Keywords: ci continuous-integration testing best-practice unit-testing new-developers deleted, testing, review-group-22, continuous-integration, ci, best-practice, new-developers, unit-testing added

Hm. I don't know all the docker+ssh stuff, so I'll assume it's right. The rest seems plausible.

What does this imply for people's branches when uploaded to gitlab? Everything builds inside a container?

Yes, the idea is that you can run a set of task inside a container. This tasks include tests and build and also syncing from torgit master.

I think ahf and dgoulet might be able to evaluate if this is what they wanted to begin with.

Trac:
Cc: isis to isis, dgoulet, ahf

okay, cool! I'm happy to merge this as soon as they say "ok". Setting owner and reviewer.

Trac:
Owner: N/A to hiro
Status: needs_review to assigned
Reviewer: N/A to ahf, dgoulet

Trac:
Status: assigned to needs_review

Should we merge upstream this file with git clone git@oniongit.eu:hiro/tor.git in it ? What are we trying to update here from git.tpo/tor.git ?

Trac:
Status: needs_review to needs_information

Hi dgoulet,

We shouldn't merge this gitlab-ci.yml. This is an example of what can be done regarding updating the master branch and running build and tests.

We should create a gitlab-ci.yml for the tor repository if we want to merge something that work. But before doing anything like that I should now what we want:

Do we want something that just sync master?

Do we want something that also builds and runs tests?

When we fetch the repository to sync master what would be the best strategy to avoid loosing data? In the example I am running a pull -Xtheirs but maybe we do not want this.

I think it's OK to just mirror the repository (git push --mirror etc.). (I think gitlab.com does this for its repository syncing.) I think nobody should be pushing changes to oniongit directly, so forced updates should be fine. I don't know whether it would delete the special gitlab refs for merge requests though.

Ok I can try this and let you guys know.

Here is the update: !https://oniongit.eu/hiro/tor/blob/gitlab-ci/.gitlab-ci.yml

Please let me know if you have any feedback. There is another bit of work in terms of keys that need to have write access to the oniongit tor repo.

Also, in case you want to run this in your own branches and repos, we will need to setup runners for it.

Trac:
Status: needs_information to needs_review

Put 0.3.2 needs_review and merge_ready tickets into review-group-23.

Trac:
Keywords: N/A deleted, review-group-23 added

Replying to hiro:

Here is the update: !https://oniongit.eu/hiro/tor/blob/gitlab-ci/.gitlab-ci.yml

That link doesn't exists... ?

Trac:
Status: needs_review to needs_information

Sorry: https://oniongit.eu/hiro/tor/blob/gitlab-ci/.gitlab-ci.yml

The branch got deleted when I merged into master.

Trac:
Status: needs_information to needs_review

Looks reasonable to me!

I'm not very familiar with what this implies: ssh-add <(echo "$DEPLOY_KEY") that is I doubt anyone can just control that env variable? Giving commit access to network/tor.git to anyone would be bad :S...

Trac:
Username: krichter
Cc: isis, dgoulet, ahf to isis, dgoulet, ahf, krichter722@aol.de

The DEPLOY_KEY variable is just a name of a ssh key that is authorised to push to oniongit master. That key has to be stored in oniongit and can be changed by a master or owner of the repository. You can check: https://docs.gitlab.com/ce/ssh/README.html#deploy-keys for more info.

Trac:
Status: needs_review to needs_revision