Opened 7 years ago

Closed 6 years ago

#6118 closed enhancement (fixed)

Implement a more scalable method for storing / querying rulesets

Reported by: pde Owned by: pde
Priority: Medium Milestone:
Component: HTTPS Everywhere/EFF-HTTPS Everywhere Version:
Severity: Keywords:
Cc: joris.goovaerts@… Actual Points:
Parent ID: #4804 Points:
Reviewer: Sponsor:

Description

The only thing that we definitely need in RAM all the time a mapping from target hosts to rulesets (currently this.targets). The ruleset XML objects themselves could be lazily constructed at use time.

What to do about rulesetsByName and rulesetsByID also requires some thought.

Child Tickets

Change History (7)

comment:1 Changed 7 years ago by pde

Component: - Select a componentEFF-HTTPS Everywhere
Owner: set to pde
Type: defectenhancement

comment:2 Changed 7 years ago by JGO

Cc: joris.goovaerts@… added

comment:3 Changed 7 years ago by pde

Hi Joris, are you by any chance the person who was asking in IRC about working on this bug?

In any case, the "perfect" solution to this ticket might be some compressed binary indexing approach. But there may also be some useful strategies available which are less ambitious than that.

comment:4 Changed 7 years ago by JGO

Yes I was, thanks for making the connection.

I was looking at a sqlite implementation to implement this:

Meaning we could outsource the whole lookup, index and storage stuff and occasionally I was looking at startup-times for the addon. I noticed that the calls to HTTPSRules.init() take about 2.35 seconds (I added some timing information around calls to this) and perhaps the way we handle this could improve that timing as well.

But I must say this (sqlite) is perhaps not what you had in mind, and 'compressed binary indexing' is not what I usually do :P. But I will have a little research into that.

comment:5 Changed 7 years ago by Drzraf

having the list in prefs.js is obviously suboptimal: from a maintenance POV it should clearly move in its own file.
But is it possible to switch to a simple text-based file (like an ini-file or so) so that it's easily understandable, editable, versionable, ...
Maybe a sqlite db (if really needed from a performance POV) could be used as an optional cache mechanism, rebuilt as needed at browser startup ?

comment:6 Changed 7 years ago by Drzraf

(I should have read #6496 before)

comment:7 Changed 6 years ago by pde

Resolution: fixed
Status: newclosed

Implemented by jsha in git master (see this thread)

Note: See TracTickets for help on using tickets.