Opened 12 months ago

Closed 8 months ago

#23830 closed enhancement (implemented)

Update README to get a development environment for metrics-web going

Reported by: irl Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Website Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Attempting to set up a development environment for metrics-web is not easy.

There is a reference to Debian packages that don't appear to exist, for example I think r-cran-java should be r-cran-rjava maybe?

It would be good to have a knowledgable person go over the whole thing and double check it.

Child Tickets

Change History (18)

comment:1 Changed 12 months ago by iwakeh

Summary: Update README to get a development environment for metrics-lib goingUpdate README to get a development environment for metrics-web going
Type: taskenhancement

Corrected title.

comment:2 Changed 12 months ago by iwakeh

The packages might refer to older debian names or contain typos and need an update, that's true.

Please post more of your findings or questions.

Thanks!

comment:3 Changed 12 months ago by karsten

Alright, I made a start in updating the README in my task-23830 branch. However, I can't determine whether we ever needed that r-cran-[r]java package, and I ran into issues with JSTL packages being missing in Stretch that were still present in Jessie.

iwakeh, did you succeed in setting up a metrics-web instance? If so, can you resolve that JSTL mystery? If not, can you try setting up an instance and, erm, resolve that JSTL mystery? ;) If not, I'll give it another try another time.

comment:4 Changed 12 months ago by karsten

From talking to iwakeh: we might want to look into switching ExoneraTor and then metrics-web from Tomcat to Jetty, and maybe that will magically resolve the JSTL mystery.

comment:5 Changed 12 months ago by iwakeh

I can also look into this metrics-web issue here first to make instances 'reproducible'.

comment:6 Changed 12 months ago by karsten

Whatever is easier. That's why I wrote ExoneraTor first, because it is so much smaller. And metrics-web is not on fire at the moment. Whatever is easier.

comment:7 Changed 11 months ago by iwakeh

There is a patch branch of metrics-web available now on ticket #24175. It is not completely finished yet, but usable for web testing and looking at how to integrate a new module, see #24175#comment:2 for more details especially How to test web related things and new modules.

comment:8 Changed 10 months ago by iwakeh

The list of packages won't be part of the future 'readme', b/c releases contain all necessary libraries and enable a better way of reproducing the productive environment as well as a faster start for a development environment. The debian packages also contain way more libraries than necessary.

All referenced jar (except metrics-lib) are available on debian stretch and their packages can be identified with the following query:
https://packages.debian.org/search?suite=stretch&searchon=contents&keywords=xyz.jar

Here the list of debian packages for jetty based metrics-web:

libcommons-codec-java
libcommons-lang3-java
libcommons-lang-java
libcommons-compress-java
liblogback-java
libslf4j-java
libgoogle-gson-java
r-cran-rserve or r-cran-rjava both supply REngine.jar and Rserve.jar
libservlet3.1-java
libxz-java
libjetty9-java
libjetty9-extra-java
libasm-java
libtaglibs-standard-spec-java
libtomcat8-embed-java
libecj-java

comment:9 Changed 10 months ago by irl

libpostgresql-jdbc-java is missing from this list.

Only r-cran-rjava doesn't contains Rserve.jar, but r-cran-rserve has both.

comment:10 Changed 10 months ago by irl

I found that I also required tomcat8-annotations-api-8.5.24.jar but this may be due to me using different versions on Debian testing. Next year I'll get a Debian stable virtual machine set up for doing development/testing on.

I also had to disable duplicate="fail" for the war target in the ant script. This is probably suboptimal although I don't really understand what that does. Maybe it's only working for me by chance.

Other than that, using iwakeh's task-24175-stepbystep branch, I was able to set up a development instance! (:

comment:11 in reply to:  10 Changed 10 months ago by iwakeh

Replying to irl:

I found that I also required tomcat8-annotations-api-8.5.24.jar but this may be due to me using different versions on Debian testing. Next year I'll get a Debian stable virtual machine set up for doing development/testing on.

Thanks for running it!
Hopefully, we get out a release before that. Our releases always include a full set of dependencies. The end of jar hunting.

I also had to disable duplicate="fail" for the war target in the ant script. This is probably suboptimal although I don't really understand what that does. Maybe it's only working for me by chance.

That might be due to the additional dependency; another dependency gets tweaked in order to only include one file of each. This (i.e. disabling the duplication check) will also not be necessary after a release anymore.

Other than that, using iwakeh's task-24175-stepbystep branch, I was able to set up a development instance! (:

Great!

Last edited 10 months ago by iwakeh (previous) (diff)

comment:12 in reply to:  9 Changed 10 months ago by iwakeh

Replying to irl:

libpostgresql-jdbc-java is missing from this list.

Only r-cran-rjava doesn't contains Rserve.jar, but r-cran-rserve has both.

Good to know. Thanks for investigating!

comment:13 Changed 10 months ago by karsten

Note that I had to install libtaglibs-standard-impl-java as well.

comment:14 in reply to:  13 Changed 10 months ago by karsten

Replying to karsten:

Note that I had to install libtaglibs-standard-impl-java as well.

(Looks like it's a dependency of libjetty9-extra-java which I didn't install explicitly. Might be okay to leave it out, though it wouldn't hurt to list it as well.)

comment:15 Changed 10 months ago by iwakeh

The README should contain a note that a missing taglibs-standard-impl-1.2.5.jar will not be noticed during build time but cause the following error log entry:
... org.apache.jasper.JasperException: The absolute uri: http://java.sun.com/jsp/jstl/core cannot be resolved in either web.xml or the jar files deployed with this application ...

comment:16 Changed 10 months ago by iwakeh

NB: Now the embedded jetty changes are merged to the main git repository.

comment:17 Changed 10 months ago by iwakeh

A first release with all dependencies is available here; documentation needs still updating.

comment:18 Changed 8 months ago by iwakeh

Resolution: implemented
Status: newclosed

I think this is obsolete now as Relay Search is integrated into metrics.torproject.org and can be closed now.

Please reopen, if there are any unresolved issues left.

Note: See TracTickets for help on using tickets.