NetworkStatusEntryImpl#parseSLine (and probably other methods) is not thread-safe
Turns out that creating multiple DescriptorReader
instances in separate threads can cause thread-safety issues in NetworkStatusEntryImpl#parseSLine
:
18:02:24.650 [Thread-1] INFO org.torproject.descriptor.DescriptorSourceFactory - Serving implementation org.torproject.descriptor.impl.DescriptorReaderImpl for descriptor.reader.
18:02:24.650 [Thread-5] INFO org.torproject.descriptor.DescriptorSourceFactory - Serving implementation org.torproject.descriptor.impl.DescriptorReaderImpl for descriptor.reader.
18:02:24.650 [Thread-3] INFO org.torproject.descriptor.DescriptorSourceFactory - Serving implementation org.torproject.descriptor.impl.DescriptorReaderImpl for descriptor.reader.
18:02:24.947 [Thread-10] ERROR org.torproject.descriptor.impl.DescriptorReaderImpl - Bug: uncaught exception or error while reading descriptors.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.torproject.descriptor.impl.DescriptorParserImpl.parseOneDescriptor(DescriptorParserImpl.java:236)
at org.torproject.descriptor.impl.DescriptorParserImpl.parseOneOrMoreDescriptors(DescriptorParserImpl.java:213)
at org.torproject.descriptor.impl.DescriptorParserImpl.detectTypeAndParseDescriptors(DescriptorParserImpl.java:61)
at org.torproject.descriptor.impl.DescriptorParserImpl.parseDescriptors(DescriptorParserImpl.java:33)
at org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.readTarball(DescriptorReaderImpl.java:320)
at org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.readTarballs(DescriptorReaderImpl.java:270)
at org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.run(DescriptorReaderImpl.java:161)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.torproject.descriptor.impl.DescriptorParserImpl.parseOneDescriptor(DescriptorParserImpl.java:229)
... 7 common frames omitted
Caused by: java.lang.NullPointerException: null
at org.torproject.descriptor.impl.NetworkStatusEntryImpl.parseSLine(NetworkStatusEntryImpl.java:177)
at org.torproject.descriptor.impl.NetworkStatusEntryImpl.parseStatusEntryBytes(NetworkStatusEntryImpl.java:98)
at org.torproject.descriptor.impl.NetworkStatusEntryImpl.<init>(NetworkStatusEntryImpl.java:58)
at org.torproject.descriptor.impl.RelayNetworkStatusConsensusImpl.parseStatusEntry(RelayNetworkStatusConsensusImpl.java:121)
at org.torproject.descriptor.impl.NetworkStatusImpl.parseStatusEntries(NetworkStatusImpl.java:85)
at org.torproject.descriptor.impl.NetworkStatusImpl.splitAndParseParts(NetworkStatusImpl.java:59)
at org.torproject.descriptor.impl.NetworkStatusImpl.<init>(NetworkStatusImpl.java:27)
at org.torproject.descriptor.impl.RelayNetworkStatusConsensusImpl.<init>(RelayNetworkStatusConsensusImpl.java:28)
... 12 common frames omitted
18:02:24.948 [Thread-1] INFO org.torproject.descriptor.DescriptorSourceFactory - Serving implementation org.torproject.descriptor.impl.DescriptorReaderImpl for descriptor.reader.
Code to reproduce this in 1 out of 10 cases:
import org.torproject.descriptor.Descriptor;
import org.torproject.descriptor.DescriptorSourceFactory;
import org.torproject.descriptor.RelayNetworkStatusConsensus;
import java.io.File;
import java.util.SortedSet;
import java.util.TreeSet;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class Main {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(15,
(Runnable r) -> {
Thread t = new Thread(r);
//t.setDaemon(true);
return t;
});
SortedSet<File> descriptorFiles = new TreeSet<>();
for (File descriptorFile : new File("in").listFiles()) {
descriptorFiles.add(descriptorFile);
}
for (int i = 0; i < 5; i++) {
for (File descriptorFile : descriptorFiles) {
Thread readerThread = new Thread() {
@Override
public void run() {
parseDescriptorFile(descriptorFile);
}
};
executor.submit(readerThread);
}
}
}
private static void parseDescriptorFile(File descriptorFile) {
for (Descriptor descriptor
: DescriptorSourceFactory.createDescriptorReader()
.readDescriptors(descriptorFile)) {
// ignore
if (descriptor instanceof RelayNetworkStatusConsensus) {
continue;
}
}
}
}
There are probably similar issues with other methods.