Opened 3 years ago

Closed 3 years ago

#22190 closed defect (fixed)

DescriptorIndexCollector does not delete extraneous local files if remote paths with leading /

Reported by: karsten Owned by: iwakeh
Priority: Medium Milestone: metrics-lib 1.7.0
Component: Metrics/Library Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Today I realized that metrics-web does not delete any extraneous local files since we made DescriptorIndexCollector the default in metrics-lib version 1.6.0.

Turns out that the following metrics-web patch works around the issue:

diff --git a/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java b/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
index 499dff9..39b3b69 100644
--- a/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
+++ b/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
@@ -17,14 +17,14 @@ public class Main {
         DescriptorSourceFactory.createDescriptorCollector();
     collector.collectDescriptors(
         "https://collector.torproject.org", new String[] {
-            "/recent/bridge-descriptors/extra-infos/",
-            "/recent/bridge-descriptors/server-descriptors/",
-            "/recent/bridge-descriptors/statuses/",
-            "/recent/exit-lists/",
-            "/recent/relay-descriptors/consensuses/",
-            "/recent/relay-descriptors/extra-infos/",
-            "/recent/relay-descriptors/server-descriptors/",
-            "/recent/torperf/"
+            "recent/bridge-descriptors/extra-infos/",
+            "recent/bridge-descriptors/server-descriptors/",
+            "recent/bridge-descriptors/statuses/",
+            "recent/exit-lists/",
+            "recent/relay-descriptors/consensuses/",
+            "recent/relay-descriptors/extra-infos/",
+            "recent/relay-descriptors/server-descriptors/",
+            "recent/torperf/"
         }, 0L, new File("../../shared/in"), true);
   }
 }

However, I think we need to fix this in metrics-lib by accepting remote paths with leading / for fetching remote files and deleting extraneous local files.

iwakeh, want to look into this?

Child Tickets

Change History (4)

comment:1 Changed 3 years ago by iwakeh

Owner: changed from metrics-team to iwakeh
Status: newassigned

comment:2 Changed 3 years ago by iwakeh

Status: assignedaccepted

comment:3 Changed 3 years ago by iwakeh

Status: acceptedneeds_review

Please review my task-22190 branch (three commits).

comment:4 Changed 3 years ago by karsten

Resolution: fixed
Status: needs_reviewclosed

Looks good, merged! (In the future, please also provide a change log entry. I might feel the urge to rephrase that to use the same writing style for all change log entries, but rewriting is often easier than writing.) Thanks! Closing.

Note: See TracTickets for help on using tickets.