Opened 4 years ago

Last modified 3 years ago

#20236 assigned enhancement

Make changes to bridgedescs module for bulk-processing tarballs

Reported by: karsten Owned by: metrics-team
Priority: Low Milestone:
Component: Metrics/CollecTor Version:
Severity: Normal Keywords: metrics-2018
Cc: iwakeh Actual Points:
Parent ID: Points:
Reviewer: Sponsor:


I recently finished re-processing the entire bridge descriptor archive for #19317. However, I had to make some changes to avoid running out of memory or wasting time on unnecessary operations. I now went through the changes and cleaned them up a bit, because I'd like to merge some/most/all (?) of them for the next time we need to bulk-process the bridge descriptor archive. I'll post a branch once I have a ticket number.

We should discuss which of these commits should go in by default (maybe ed48f03, ae5c53c, and e514d30?), which should only be enabled in a special bulk-processing mode (maybe df96751, 27cbfc8, and 68b29c2?), which should have their own config option (ugh!), or which we drop because we don't need as badly for processing descriptors in bulk.

Clearly, these commits need work, but I figured it's better to clean them up a bit now than attempt to do that in four or eight weeks. Branch follows in a minute.

Child Tickets

Change History (5)

comment:2 Changed 3 years ago by karsten

This issue is closely related to #19778, though I couldn't decide which one to close as duplicate.

comment:3 Changed 3 years ago by iwakeh

Make sure to address all topics raised in the closed #19778.

comment:4 Changed 3 years ago by karsten

Keywords: metrics-2018 added

comment:5 Changed 3 years ago by karsten

Owner: set to metrics-team
Status: newassigned
Note: See TracTickets for help on using tickets.