Update README to get a development environment for metrics-web going

Attempting to set up a development environment for metrics-web is not easy.

There is a reference to Debian packages that don't appear to exist, for example I think r-cran-java should be r-cran-rjava maybe?

It would be good to have a knowledgable person go over the whole thing and double check it.

added component::metrics/website owner::metrics-team priority::medium resolution::implemented severity::normal status::closed type::enhancement labels

Corrected title.

Trac:
Summary: Update README to get a development environment for metrics-lib going to Update README to get a development environment for metrics-web going
Type: task to enhancement

The packages might refer to older debian names or contain typos and need an update, that's true.

Please post more of your findings or questions.

Thanks!

Alright, I made a start in updating the README in my task-23830 branch. However, I can't determine whether we ever needed that r-cran-[r]java package, and I ran into issues with JSTL packages being missing in Stretch that were still present in Jessie.

iwakeh, did you succeed in setting up a metrics-web instance? If so, can you resolve that JSTL mystery? If not, can you try setting up an instance and, erm, resolve that JSTL mystery? ;) If not, I'll give it another try another time.

From talking to iwakeh: we might want to look into switching ExoneraTor and then metrics-web from Tomcat to Jetty, and maybe that will magically resolve the JSTL mystery.

I can also look into this metrics-web issue here first to make instances 'reproducible'.

Whatever is easier. That's why I wrote ExoneraTor first, because it is so much smaller. And metrics-web is not on fire at the moment. Whatever is easier.

There is a patch branch of metrics-web available now on ticket #24175 (moved). It is not completely finished yet, but usable for web testing and looking at how to integrate a new module, see #24175#comment:2 for more details especially //How to test web related things and new modules//.

The list of packages won't be part of the future 'readme', b/c releases contain all necessary libraries and enable a better way of reproducing the productive environment as well as a faster start for a development environment. The debian packages also contain way more libraries than necessary.

All referenced jar (except metrics-lib) are available on debian stretch and their packages can be identified with the following query: https://packages.debian.org/search?suite=stretch&searchon=contents&keywords=xyz.jar

Here the list of debian packages for jetty based metrics-web:

libcommons-codec-java
libcommons-lang3-java
libcommons-lang-java
libcommons-compress-java
liblogback-java
libslf4j-java
libgoogle-gson-java
r-cran-rserve or r-cran-rjava both supply REngine.jar and Rserve.jar
libservlet3.1-java
libxz-java
libjetty9-java
libjetty9-extra-java
libasm-java
libtaglibs-standard-spec-java
libtomcat8-embed-java
libecj-java

libpostgresql-jdbc-java is missing from this list.

Only r-cran-rjava doesn't contains Rserve.jar, but r-cran-rserve has both.

I found that I also required tomcat8-annotations-api-8.5.24.jar but this may be due to me using different versions on Debian testing. Next year I'll get a Debian stable virtual machine set up for doing development/testing on.

I also had to disable duplicate="fail" for the war target in the ant script. This is probably suboptimal although I don't really understand what that does. Maybe it's only working for me by chance.

Other than that, using iwakeh's task-24175-stepbystep branch, I was able to set up a development instance! (:

Replying to irl:

I found that I also required tomcat8-annotations-api-8.5.24.jar but this may be due to me using different versions on Debian testing. Next year I'll get a Debian stable virtual machine set up for doing development/testing on.

Thanks for running it! Hopefully, we get out a release before that. Our releases always include a full set of dependencies. The end of jar hunting.

I also had to disable duplicate="fail" for the war target in the ant script. This is probably suboptimal although I don't really understand what that does. Maybe it's only working for me by chance.

That might be due to the additional dependency; another dependency gets tweaked in order to only include one file of each. This (i.e. disabling the duplication check) will also not be necessary after a release anymore.

Other than that, using iwakeh's task-24175-stepbystep branch, I was able to set up a development instance! (:

Great!

Replying to irl:

libpostgresql-jdbc-java is missing from this list.

Only r-cran-rjava doesn't contains Rserve.jar, but r-cran-rserve has both.

Good to know. Thanks for investigating!

Note that I had to install libtaglibs-standard-impl-java as well.

Replying to karsten:

Note that I had to install libtaglibs-standard-impl-java as well.

(Looks like it's a dependency of libjetty9-extra-java which I didn't install explicitly. Might be okay to leave it out, though it wouldn't hurt to list it as well.)

The README should contain a note that a missing taglibs-standard-impl-1.2.5.jar will not be noticed during build time but cause the following error log entry: ... org.apache.jasper.JasperException: The absolute uri: http://java.sun.com/jsp/jstl/core cannot be resolved in either web.xml or the jar files deployed with this application ...

NB: Now the embedded jetty changes are merged to the main git repository.

A first release with all dependencies is available here; documentation needs still updating.

I think this is obsolete now as Relay Search is integrated into metrics.torproject.org and can be closed now.

Please reopen, if there are any unresolved issues left.

Trac:
Status: new to closed
Resolution: N/A to implemented

closed

mentioned in issue #24175 (moved)