Opened 2 weeks ago

Last modified 17 hours ago

#32627 needs_review enhancement

deploy torspec as HTML to GitLab Pages

Reported by: eighthave Owned by:
Priority: Medium Milestone: Tor: 0.4.3.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: torspec, doc, extra-review
Cc: nickm Actual Points:
Parent ID: Points:
Reviewer: ahf Sponsor:

Description will deploy torspec in HTML to any GitLab fork setup with CI and Pages (default on it only adds one file: .gitlab-ci.yml

Once merged, the site will show up automatically on, and it'll sync every commit from the canonical repo and automatically rebuild the HTML.

The sed regexps in .gitlab-ci.yml could be used as the beginnings of a conversion to Markdown format, as needed.

Child Tickets

Change History (11)

comment:1 Changed 2 weeks ago by eighthave

Status: newneeds_review

comment:2 Changed 2 weeks ago by eighthave

Summary: deploy torspec as HTML todeploy torspec as HTML to GitLab Pages

comment:3 Changed 2 weeks ago by eighthave

Here's a Python version of the same script, for anyone who wants it.

#!/usr/bin/env python3

import glob
import os
import re

if not os.path.exists('public'):

titles = dict()
for f in glob.glob('*.txt'):
    with open(f) as fp:
        contents =
    contents = re.sub(r'^\n +', r'# ', contents)
    contents = re.sub(r'\n {1,3}([^ ])', r'\n\1', contents)
    contents = re.sub(r'\n([0-9]+\. (?!http))', r'\n## \1', contents)
    contents = re.sub(r'\n([0-9]+\.[0-9]+\. )', r'\n### \1', contents)
    contents = re.sub(r'\n([0-9]+\.[0-9]+\.[0-9]+\. )', r'\n#### \1', contents)
    contents = re.sub(r'\n([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\. )', r'\n##### \1', contents)
    with open(f, 'w') as fp:
    m = re.match(r'\s*# (.*)\n', contents)
    titles[re.sub(r'\.txt$', r'', f)] =

with open('public/index.html', 'w') as fp:
    fp.write('<!DOCTYPE html>\n\n<html><body><h1>%s</h1><ul>' % os.getenv('CI_PROJECT_PATH'))
    for k, v in sorted(titles.items()):
        fp.write('<li><a href="%s.html"><tt>%s</tt>: %s</a></li>' % (k, k, v))

comment:4 Changed 2 weeks ago by teor

Component: Core TorCore Tor/Tor
Keywords: doc added
Milestone: Tor: 0.4.3.x-final
Type: defectenhancement

Thanks for this code!

I think we'd prefer python to sed, but I'll leave that to the reviewer to decide.

comment:5 Changed 2 weeks ago by eighthave

Having it in sed makes the whole GitLab CI process a single file, that's working. The Python script is there for anyone who wants to improve the automated conversion, or use it for doing a final conversion to a standard format.

comment:6 Changed 9 days ago by asn

Reviewer: ahf

comment:7 Changed 7 days ago by ahf

Status: needs_reviewmerge_ready

I think we should merge this for the following reasons:

  1. The output is a bit rough in the edges, but we can fix the formatting over time.
  2. We can extend it to include proposals over time and also fix their formatting over time.
  3. It will integrate well if/when we move to Gitlab ourselves.
  4. It's only a single additional file in the repository.

comment:8 Changed 7 days ago by teor

Cc: nickm added
Keywords: extra-review added

Since this is a new feature, I just want to check with nickm before merging it?

comment:9 Changed 28 hours ago by eighthave

I updated the merge request to fix all blockquote issues I could find, only by adding blank newlines. And a second commit that makes the generated HTML complete with a header and footer.

comment:10 Changed 28 hours ago by eighthave

The current output is visible here:

comment:11 Changed 17 hours ago by teor

Status: merge_readyneeds_review

Putting back in needs_review, because the PR changed.

Note: See TracTickets for help on using tickets.