Sponsor 31

Project Title - Modularization work Project Period: December 1, 2018 - November 30, 2019 Teams Involved: Network

Project Goals/Activities

The key deliverables of the modularization project are:

  • Refactored architecture for how Tor modules should communicate, designed to reduce module complexity and maintenance burden.
  • Architectural documentation for how Tor modules work with one another, covering both the actuality and the refactored architecture.
  • New architecture for several key Tor modules.
  • Improved testing for those modules.
  • Improved documentation for those modules.


  1. Determine how to best refactor modules to make them more resilient,segregated, and automatically testable. (288 hours) -- March 2019

1.1. Identify pieces that we have to refactor. (96 hours)

1.2. Draft initial design for each piece. (192 hours)

  1. Identify which parts of the codebase are most important to refactor first. (256 hours) -- April and August 2019

2.1. Discuss which parts to refactor first. (16 hours)

2.2. Research and discuss automated tooling. (48 hours)

2.3. Design tooling as needed. (48 hours)

2.4. Repeat the same tasks 2.1, 2.2 and 2.3 after milestone 4 is completed. (144 hours)

  1. Revise the system of interactions between modules to improve usability and maintainability. (88 hours) -- May 2019

3.1. Review publisher/subscriber design that Tor uses. (16 hours)

3.2. Implement refactoring. (72 hours)

  1. Create and deploy plan to incorporate refactored modules. (184 hours) -- August 2019

4.1. For each module and each architectural change, form a plan to apply that change to that module. This plan should include increased testing, if the module's test coverage is not already high. (160 hours)

4.2. For each plan formed in 4.1, deploy that plan. (24 hours)

  1. Create documentation for how Tor modules work with one another with both old and new architecture. (432 hours) -- December 2018

5.1. Update module documentation and incorporate into Tor. (96 hours)

5.2. Document existing refactoring. (144 hours)

5.3. Document new elements. (144 hours)

5.4. For modules we are not refactoring, document important interactions. (48 hours)

Project Tracking

Refactoring cross-module communications

#28226 Backend for an async publish-subcribe messaging system for cross-module communications (Tor 0.4.1)

Coding best practices

#29221 Tooling to track code-style/best-practices violations #29219 Write guidelines for Tor coding best practices #29220 Update review guidelines for practices #29223 List canonical abbreviations to use in Tor functions and identifiers

Refactoring modules

#29201 Tor bootstrap hangs when offline - Split up control.c #29209 Reduce visibility of more data type internals #29218 Revise "handles" to be opaque, safe. Document standard usage #29224 Abstractions and best practices for disabled modules #29217 Script to add new tor files and modules in rust and/or C #29227 Audit Tor roles (relay, bridge, etc) and remove needless dependencies where possible


#29216 Document how to make new files/modules in Tor


Sponsor: Sponsor31-can (77 matches)

Ticket Summary Status Owner Keywords Priority
#25950 Run "accounting_run_housekeeping" with a periodic event. assigned 034-roadmap-subtask, 034-triage-20180328, 034-included-20180328 Medium
#28170 Test stem pull requests against all supported tor versions, and tor nightly builds new ci, travis Medium
#28208 Run bridges+hs-v23 for make test-network in 0.4.2 and later assigned test, fast-fix, teor-backlog-ci, network-team-roadmap-november Medium
#28377 Add libnss to the travis apt and macOS packages assigned fast-fix, tor-ci, travis, postfreeze-ok, teor-backlog-ci Medium
#28378 Add libsystemd-devel to the travis config, if available assigned fast-fix, tor-systemd, 029-backport, 035-backport, postfreeze-ok, tor-ci, teor-backlog-ci, 033-unreached-backport Medium
#29209 Reduce visibility of more data type internals new technical-debt refactoring Medium
#29210 Distribute control.c functionality across various modules new catalyst network-team-roadmap-september Medium
#29211 Distribute config.c functionality across more modules assigned nickm network-team-roadmap-august Medium
#29212 Create more modularity inside src/core new Medium
#29213 Minimize the boundary between src/core and src/{feature,app} new Medium
#29214 Update 'tor-guts' archictecture documentation to describe current (actual, as-built) architecture. new network-team-roadmap-november Medium
#29215 Document target, modular tor architecture new network-team-roadmap-september Medium
#29216 Document how to make new files/modules in Tor new Medium
#29217 Script to add new tor files and modules in rust and/or C new network-team-roadmap-september Medium
#29218 Revise "handles" to be opaque, safe. Document standard usage assigned nickm Medium
#29219 Write (more) guidelines for Tor coding best practices new doc, ex-catalyst-20190619 Medium
#29220 Update review guidelines to list best practices assigned doc network-team-roadmap-2019-Q1Q2 Medium
#29222 Fix biggest violations of coding best-practices new Medium
#29224 Abstractions and best practices for disabled modules new Medium
#29225 Improve efficiency and flexibility of threadpool/workqueue API new Medium
#29226 Automate application of C formatting to code new network-team-roadmap-october Medium
#29227 Audit Tor roles (relay, bridge, etc) and remove needless dependencies where possible new Medium
#29228 Unify some or all of Tor's various message/event/work queues. new Medium
#29437 test-stem times out intermittently assigned tor-ci-fail-sometimes Medium
#29642 Free Windows library handles when we free_all in a module new tor-windows, memory-management, technical-debt Low
#29656 describe global initialization in our tinytest-based unit tests new developer-doc network-team-roadmap-october Medium
#29689 Make Tor's consistent with chutney's assigned tor-chutney, tor-test, tor-ci, fast-fix, technical-debt, ex-sponsor-19, ex-sponsor19, teor-backlog-ci Medium
#29746 Improve Tor best practices tracker assigned nickm practracker, tech-debt, refactoring, easy, 041-deferred-20190530, network-team-roadmap-july, dgoulet-merge Medium
#29793 show-distdir-core and friends should fail more quietly new tor-ci, travis Medium
#29821 Make circuit/channel padding correctly respect dormant mode new wtf-pad padding dormant dormant-mode Low
#29881 Create a GitHub bot that makes sure there is a pull request on master for every branch new tor-ci, 041-deferred-20190530, sponsor31-maybe Medium
#29883 Enforce lower practracker limits for every new release new 042-proposed Medium
#29895 avoid storing ints in void* in mainloop event data new portability, technical-debt Medium
#30225 Run clang's scan-build in Tor's CI new tor-ci, correctness, coverity, clang, scan-build Medium
#30235 Stem hangs waiting for control port data needs_information tor-ci-fail-sometimes Medium
#30243 Stem's tests use fixed ports for tor, which may already be in use new atagar tor-ci-single-failure Medium
#30257 Propagate USR1 and ABRT signals from stem tests through to tor new atagar Medium
#30292 Refactor periodic event system so each subsystem "owns" its own periodic events assigned nickm Medium
#30349 Document member-hiding conventions for structs new technical-debt, refactoring, network-team-roadmap-october Medium
#30527 Fix coverity warnings in test_parsecommon.c assigned nickm coverity technical-debt Medium
#30651 Make stem's Travis jobs log controller messages from stem and tor assigned ci, travis, teor-backlog-ci Medium
#30695 Consider running tor's stem CI job in --target ONLINE mode, maybe with chutney new Medium
#30720 Use for the examples in Chutney's README assigned doc, easy, teor-backlog-ci, network-team-roadmap-november High
#30745 Document disabled CI assigned teor 041-can process Medium
#30816 Remove ping ::1 from tor's test-network-all and simplify the logic new refactor, simplify, network-team-roadmap-2019-Q1Q2, tor-ci Medium
#30827 Stem CI: remove 0.3.4 builds; add 0.4.1 needs_review teor Medium
#30831 Separate front-end and back-end of handle implementation. assigned nickm Medium
#30839 Update with our latest end of life process needs_revision teor 041-should network-team-roadmap-september Medium
#30858 Load geoip and geoip6 files during the unit tests assigned technical-debt, geoip, fast-fix?, easy?, teor-backlog-ci Low
#30860 Add a chutney job that runs on macOS, so that IPv6 chutney tests work assigned CI, PTs, sponsor-28-maybe, 029-backport, 035-backport, 040-backport, network-team-roadmap-2019-Q1Q2, tor-ci, teor-backlog-ci Medium
#30865 Move option-listing, setting, validation code out of confparse.c assigned nickm Medium
#30866 Teach config.c to work with options configured in other modules assigned nickm Medium
#30890 Make a useful .gitlab-ci.yml mirroring job new tor-ci, gitlab Medium
#30894 Memory leak on invalid CSV_INTERVAL value merge_ready nickm consider-backport-after-0415, 035-backport 040-backport 041-backport Medium
#30901 Add control port trace logging to tor needs_revision tor-ci-fail-sometimes network-team-roadmap-august Medium
#30914 Move struct manipulation code out of confparse.c needs_review nickm Medium
#30935 Move variable definition code out of confparse.c, and refactor needs_review nickm Medium
#30967 Explicitly select the top-level directories that we want to shellcheck assigned fast-fix, teor-backlog-ci Medium
#30984 Make a key-value line abstraction to output control replies accepted catalyst Medium
#31074 Use tor_queue.h macros in config_line_t new refactor technical-debt network-team-roadmap-september Low
#31078 improve docs for config var abstraction new Medium
#31081 GETCONF allows zero arguments, contrary to spec needs_revision tor-control, tor-spec Medium
#31121 Use publish-subscribe system in more places new network-team-roadmap-october Medium
#31122 Use subsystems architecture in more places new network-team-roadmap-november Medium
#31175 Teach practracker to apply separate rules for C headers merge_ready nickm network-team-roadmap-august Medium
#31176 Teach practracker about .may_include files needs_review nickm practracker, tech-debt, refactoring, easy, 041-deferred-20190530, network-team-roadmap-august Medium
#31177 Start conversation about auto-formatting our C code new network-team-roadmap-september Medium
#31179 Eliminate non tor_queue.h linked lists. new network-team-roadmap-october Medium
#31180 Remove deprecated options in 0.4.3. assigned network-team-roadmap-november Medium
#31181 Deprecate more options in 0.4.3 assigned network-team-roadmap-november Medium
#31182 Add more typesafe containers to tor new network-team-roadmap-november Medium
#31238 Write a nontrivial "preferred" rust module new Medium
#31240 Make confparse able to handle multiple config_format_t objects at once needs_review nickm network-team-roadmap-august Medium
#31241 Refactor config validation assigned nickm network-team-roadmap-august Medium
#31335 make check-includes does not check out of tree builds correctly new 043-should, network-team-roadmap-september Medium
#31336 Fix usage for new 043-should, network-team-roadmap-september Medium
#31338 Practracker --list-overbroad produces confusing output when there is an exception assigned nickm 042-should, practracker, easy, network-team-roadmap-july Medium

Sponsor: Sponsor31-must (4 matches)

Ticket Summary Status Owner Keywords Priority
#29223 List canonical abbreviations to use in Tor functions and identifiers needs_revision nickm doc Medium
#29879 Make push branches in a specific order needs_review teor tor-ci, git-scripts, fast-fix, 041-deferred-20190530 Medium
#31178 Scripts to help with backport branches assigned network-team-roadmap-october Medium
#31314 Modify so it can create test merge branches needs_review teor git-scripts, network-team-roadmap-october Medium

Last modified 7 weeks ago Last modified on Jun 28, 2019, 6:33:58 PM