Opened 7 months ago

Closed 5 months ago

#33593 closed task (fixed)

Create versions and changelogs for Snowflake pieces

Reported by: cohosh Owned by:
Priority: Medium Milestone:
Component: Circumvention/Snowflake Version:
Severity: Normal Keywords:
Cc: cohosh, phw, arlolra, dcf Actual Points:
Parent ID: #19409 Points:
Reviewer: Sponsor:

Description

This is a prerequisite for packaging Snowflake for Debian (#19409).

We already have versions for the snowflake browser proxy. It could make sense to version different pieces of snowflake (client, browser proxy, proxy-go) separately since these pieces are largely distinct. That would be more work though. I'm ok with having one version/changelog for all the pieces and then just bumping the version number whenever it's convenient.

Child Tickets

Change History (26)

comment:1 Changed 7 months ago by cohosh

We discussed in the anti-censorship meeting today how to handle this. My proposal for this would be to:
1) split up the Snowflake mono repo into two repositories: one for the browser-based Snowflake proxy code (written in JavaScript), and one for the main Go Snowflake components (client, server, proxy-go, and broker code)

2) Each of these repos have a repo-wide version. So we'll keep doing webextension versioning as before (though maybe remove the webext prefix since it applies to the badge as well), and then have a version for all Go Snowflake components (this ties in with the recent move to Go modules in #33330 as well).

Full meeting logs are available here: http://meetbot.debian.net/tor-meeting/2020/tor-meeting.2020-03-12-17.59.html

comment:2 Changed 7 months ago by cohosh

Status: newneeds_review

comment:3 Changed 7 months ago by arlolra

My proposal for this would be to

Seems ok to me if others are on board. As mentioned in the logs, it's important to me to maintain the git history.

comment:4 in reply to:  3 Changed 7 months ago by phw

Replying to arlolra:

My proposal for this would be to

Seems ok to me if others are on board. As mentioned in the logs, it's important to me to maintain the git history.


Agreed on the proposal and on the importance of preserving our git history!

comment:5 Changed 7 months ago by cohosh

Okay here's a test-run of how this could look. I ran the following commands from the snowflake repository root:

$ git subtree split -P proxy proxy-only
$ mkdir ../snowflake-webext
$ cd ../snowflake-webext
$ git init
$ git pull ../snowflake proxy-only

This split out the proxy subdirectory into it's own repo, which you can see here: https://github.com/cohosh/snowflake-webext

I had to do a couple fixes to get submodules working: https://github.com/cohosh/snowflake-webext/commit/70f32b18afb48e97d7a3794f952fb6dbcc7ce690

And then transferring the tags was tricky because this command changes the commit hashes. I did this manually and then pushed all the tags to the remote using git push <remote> --tags.

The original snowflake repository keeps all the commit history and all the old tags. I cleaned it up a bit in this branch: https://github.com/cohosh/snowflake/tree/ticket/33593
The changes I made were to:

  • Remove the translation submodule
  • Remove the proxy/ subdirectory
  • Remove web proxy instructions from README.md
  • tagged snowflake as v1.0.0

If everyone is okay with this plan we can create a new snowflake-webext repository on gitweb.tp.o and then once that's up and running, merge this branch to master on the main snowflake repository.

comment:6 Changed 7 months ago by arlolra

And then transferring the tags was tricky because this command changes the commit hashes.

Since the current tags are more relevant to the webext repo, it might be more worth trying to preserve them there.

An alternative approach might just be to fork the repo and make commits removing the code that's no longer relevant as you did for the main repo.

Remove the proxy/ subdirectory

Probably also want to rename proxy-go to proxy

comment:7 in reply to:  6 Changed 7 months ago by cohosh

Replying to arlolra:

And then transferring the tags was tricky because this command changes the commit hashes.

Since the current tags are more relevant to the webext repo, it might be more worth trying to preserve them there.

An alternative approach might just be to fork the repo and make commits removing the code that's no longer relevant as you did for the main repo.

By "preserve", do you mean keep the same mapping with git hashes, or have the tags refer to the same commits? Because this setup currently does the latter.

comment:8 Changed 7 months ago by arlolra

By "preserve", do you mean keep the same mapping with git hashes, or have the tags refer to the same commits? Because this setup currently does the latter.

The former

comment:9 in reply to:  6 Changed 7 months ago by dcf

Replying to arlolra:

An alternative approach might just be to fork the repo and make commits removing the code that's no longer relevant as you did for the main repo.

I too would slightly prefer a fork-and-delete that preserves commit hashes over a git subtree split or git filter-branch --subdirectory-filter that rewrites commit hashes.

comment:10 Changed 7 months ago by cohosh

Okay, let's do a full fork, that's pretty easy to accomplish. I'll make the appropriate ticket with the sysadmin team.

comment:11 Changed 7 months ago by cohosh

Made #33740

comment:12 Changed 6 months ago by cohosh

Thanks to irl, we now have a new repository just for the web extension: https://gitweb.torproject.org/snowflake-webext.git/

Here are some pull requests to tidy up each of the respositories:

comment:13 Changed 6 months ago by irl

(I didn't make any GitHub mirrors, if those would be useful please do make some tickets)

comment:14 in reply to:  12 ; Changed 6 months ago by arlolra

Replying to cohosh:

Thanks to irl, we now have a new repository just for the web extension: https://gitweb.torproject.org/snowflake-webext.git/

I assumed this was going to nest under pluggable-transports/ also but I guess it's no big deal.

Here are some pull requests to tidy up each of the respositories:

The pulls seem fine. I left comments about some trivial things that were missed.

comment:15 in reply to:  14 ; Changed 6 months ago by irl

Replying to arlolra:

Replying to cohosh:
I assumed this was going to nest under pluggable-transports/ also but I guess it's no big deal.

I could move it quickly now before anyone bookmarks links to it, let me know.

comment:16 in reply to:  15 Changed 6 months ago by cohosh

Replying to irl:

Replying to arlolra:

Replying to cohosh:
I assumed this was going to nest under pluggable-transports/ also but I guess it's no big deal.

I could move it quickly now before anyone bookmarks links to it, let me know.

Ah. I think since this is a stand-alone thing and doesn't have any PT code (there's no client or server interactions with the PT spec) that top-level is the best place.

I... don't have a strong opinion. Our meeting is in ~90 minutes, how about we discuss it there.

Sorry irl for the confusion.

comment:17 Changed 6 months ago by irl

No problem, just highlight me in IRC if you want it changed. I'll do it after dinner.

comment:18 in reply to:  17 Changed 6 months ago by cohosh

Replying to irl:

No problem, just highlight me in IRC if you want it changed. I'll do it after dinner.

Okay we've decided to move it to pluggable-transports/. Thanks again for handling this.

comment:19 Changed 6 months ago by irl

Done. (The old repo will disappear from gitweb shortly).

comment:20 in reply to:  14 ; Changed 6 months ago by cohosh

Replying to arlolra:

The pulls seem fine. I left comments about some trivial things that were missed.

Thanks! Made these changes. I'll just wait for an ok from someone before merging. And then we can move forward with the two separate repos.

I also noticed our documentation is outdated now: https://trac.torproject.org/projects/tor/wiki/doc/Snowflake#HowtorunaSnowflakeproxy

We can use this ticket to track those changes after the merge.

comment:21 Changed 6 months ago by arlolra

Status: needs_reviewmerge_ready

I'll just wait for an ok from someone before merging.

lgtm

comment:22 Changed 6 months ago by cohosh

Merged! We now have two snowflake repos :D

comment:23 in reply to:  20 Changed 6 months ago by cohosh

Replying to cohosh:

I also noticed our documentation is outdated now: https://trac.torproject.org/projects/tor/wiki/doc/Snowflake#HowtorunaSnowflakeproxy

We can use this ticket to track those changes after the merge.

Okay there weren't many edits necessary:
https://trac.torproject.org/projects/tor/wiki/doc/Snowflake?action=diff&version=56

comment:24 Changed 5 months ago by dcf

Status: merge_readyneeds_information

Is the only remaining task under this ticket to write changelogs for the two repos? As I understand it, that's a requirement of Debian packaging?

comment:25 Changed 5 months ago by cohosh

Thanks for bringing this back up.

IMO, we are nowhere near being about to move forward on Debian packaging. We have too many dependencies that are changing very rapidly. I think we should wait and let snowflake development settle down a bit before trying.

So we can either start using changelogs or decide not to and close this ticket. I don't have strong opinions either way. I like the idea of having snowflake versions to track major changes but it's not high priority for me.

comment:26 Changed 5 months ago by dcf

Resolution: fixed
Status: needs_informationclosed

Okay, I'll just close the ticket.

Note: See TracTickets for help on using tickets.