Opened 14 months ago

#27181 new enhancement

Avoid unnecessary disk writes

Reported by: irl Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Onionoo Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:


Right now if you run the updater, every status document will be touched by Onionoo regardless of whether or not the content actually changed. Additionally every file in the out directory will be touched, although I'm less concerned about this as I've seen that the out directory is reproducibly built from the status directory where no changes have occurred. My primary motivation for this ticket is enabling consistency in backups and the creation of test instances where we'd like to copy out the status directory between updater runs, where there is a limited time window.

I propose that we add a field to the Document class that marks a document as having changed since it was loaded from disk. Initially this flag would be clear but if any field in the document is changed then this would trigger a write. There may be conditions where a value is set to the value it already has and we should detect these cases and not set the flag to perform a write.

Alternatively, we can store the serialization of the document along with the parsed fields. When serializing the document we can compare the document with its original serialization to determine if it needs to be written to disk.

We will need to have good test coverage for this as if we miss setting the flag to write data we may end up with gaps in data on the Onionoo hosts.

Child Tickets

Change History (0)

Note: See TracTickets for help on using tickets.