DescriptorIndexCollector does not delete extraneous local files if remote paths with leading /
Today I realized that metrics-web does not delete any extraneous local files since we made DescriptorIndexCollector
the default in metrics-lib version 1.6.0.
Turns out that the following metrics-web patch works around the issue:
diff --git a/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java b/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
index 499dff9..39b3b69 100644
--- a/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
+++ b/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
@@ -17,14 +17,14 @@ public class Main {
DescriptorSourceFactory.createDescriptorCollector();
collector.collectDescriptors(
"https://collector.torproject.org", new String[] {
- "/recent/bridge-descriptors/extra-infos/",
- "/recent/bridge-descriptors/server-descriptors/",
- "/recent/bridge-descriptors/statuses/",
- "/recent/exit-lists/",
- "/recent/relay-descriptors/consensuses/",
- "/recent/relay-descriptors/extra-infos/",
- "/recent/relay-descriptors/server-descriptors/",
- "/recent/torperf/"
+ "recent/bridge-descriptors/extra-infos/",
+ "recent/bridge-descriptors/server-descriptors/",
+ "recent/bridge-descriptors/statuses/",
+ "recent/exit-lists/",
+ "recent/relay-descriptors/consensuses/",
+ "recent/relay-descriptors/extra-infos/",
+ "recent/relay-descriptors/server-descriptors/",
+ "recent/torperf/"
}, 0L, new File("../../shared/in"), true);
}
}
However, I think we need to fix this in metrics-lib by accepting remote paths with leading /
for fetching remote files and deleting extraneous local files.
iwakeh, want to look into this?