Upgrade metrics-web to Debian buster libraries
We'll need to upgrade metrics-web to Debian buster libraries, though we should first finish the metrics-base review in #31194 (moved).
Activity
Trac:
Parent Ticket: #31193 (moved)
I'm hitting a brick wall here.
Upgrading embedded Jetty and its internal dependencies on Tomcat9 turned out to be as complicated as expected. Here's what I have right now, and it's not working:
diff --git a/build.xml b/build.xml index 0c9fa0e4..0d4979cb 100644 --- a/build.xml +++ b/build.xml @@ -9,9 +9,9 @@ <property name="javadoc-title" value="MetricsWeb API Documentation"/> <property name="implementation-title" value="metrics-web" /> <property name="release.version" value="1.2.0-dev" /> - <property name="metricslibversion" value="2.4.0" /> + <property name="metricslibversion" value="2.6.2" /> <property name="exoneratorversion" value="4.1.0" /> - <property name="jetty.version" value="-9.2.21.v20170120" /> + <property name="jetty.version" value="-9.4.15.v20190215" /> <property name="warfile" value="metrics-web-${release.version}.war"/> @@ -38,27 +38,27 @@ </target> <patternset id="common" > - <include name="commons-codec-1.10.jar"/> - <include name="commons-lang3-3.5.jar"/> + <include name="commons-codec-1.11.jar"/> + <include name="commons-lang3-3.8.jar"/> <include name="commons-math3-3.6.1.jar"/> - <include name="jackson-annotations-2.8.6.jar"/> - <include name="jackson-core-2.8.6.jar"/> - <include name="jackson-databind-2.8.6.jar"/> - <include name="logback-core-1.1.9.jar" /> - <include name="logback-classic-1.1.9.jar" /> + <include name="jackson-annotations-2.9.8.jar"/> + <include name="jackson-core-2.9.8.jar"/> + <include name="jackson-databind-2.9.8.jar"/> + <include name="logback-core-1.2.3.jar" /> + <include name="logback-classic-1.2.3.jar" /> <include name="REngine.jar"/> <include name="Rserve.jar"/> - <include name="slf4j-api-1.7.22.jar"/> + <include name="slf4j-api-1.7.25.jar"/> </patternset> <patternset id="library" > <patternset refid="common" /> - <include name="metrics-lib-${metricslibversion}.jar"/> - <include name="commons-compress-1.13.jar"/> + <include name="metrics-lib-${metricslibversion}-thin.jar"/> + <include name="commons-compress-1.18.jar"/> <include name="commons-math3-3.6.1.jar"/> - <include name="postgresql-9.4.1212.jar"/> + <include name="postgresql-42.2.5.jar"/> <include name="servlet-api-3.1.jar"/> - <include name="xz-1.6.jar"/> + <include name="xz-1.8.jar"/> </patternset> <patternset id="web" > @@ -76,18 +76,18 @@ @@ -76,18 +76,18 @@ </patternset> <patternset id="webruntime" > - <include name="asm-5.2.jar" /> - <include name="asm-commons-5.2.jar" /> - <include name="commons-compress-1.13.jar"/> + <include name="asm-all-7.0.jar" /> + <include name="commons-compress-1.18.jar"/> <include name="jetty9-plus${jetty.version}.jar"/> <include name="jetty9-jndi${jetty.version}.jar"/> <include name="jetty9-apache-jsp${jetty.version}-tweaked.jar" /> <include name="metrics-lib-${metricslibversion}.jar"/> <include name="taglibs-standard-spec-1.2.5.jar"/> - <include name="tomcat8-embed-jasper-8.5.14.jar" /> - <include name="tomcat8-embed-el-8.5.14.jar" /> - <include name="tomcat8-embed-core-8.5.14.jar" /> - <include name="eclipse-ecj-3.11.1.jar" /> + <include name="tomcat9-annotations-api-9.0.16.jar" /> + <include name="tomcat9-embed-jasper-9.0.16-tweaked.jar" /> + <include name="tomcat9-embed-el-9.0.16.jar" /> + <include name="tomcat9-embed-core-9.0.16.jar" /> + <include name="eclipse-ecj-3.16.0.jar" /> </patternset> <patternset id="runtime" > @@ -139,16 +139,22 @@ <target name="war" depends="submoduleupdate,jar"> <echo message="Removing problematic service definitions from"/> - <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and using "/> - <echo message="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar below." /> + <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and "/> + <echo message="${libs}/tomcat9-embed-jasper-9.0.16.jar." /> <delete file="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar" quiet="true" /> - + <delete file="${libs}/tomcat9-embed-jasper-9.0.16-tweaked.jar" + quiet="true" /> <jar destfile="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar"> <zipfileset src="${libs}/jetty9-apache-jsp${jetty.version}.jar" > <exclude name="**/javax.servlet.ServletContainerInitializer"/> </zipfileset> </jar> + <jar destfile="${libs}/tomcat9-embed-jasper-9.0.16-tweaked.jar"> + <zipfileset src="${libs}/tomcat9-embed-jasper-9.0.16.jar" > + <exclude name="**/javax.servlet.ServletContainerInitializer"/> + </zipfileset> + </jar> <antcall target="generate-ml-javadoc" />The error messages aren't very helpful, but I think it's related to some internal changes in Jetty or embedded Tomcat using a different library for JSP compilation.
I feel like we're doing it wrong.
The original idea of using libraries shipped with Debian stable was a good one, because it would allow anyone on a Debian machine to build our software with minimal effort.
But this doesn't come for free. We're basically doing manual dependency management not only for our dependencies but also for their dependencies.
Another major drawback, in addition to having to fix the issue above, is that it's almost prohibitively expensive to add new dependencies. For example, I'd very much want to add something like FindBugs. But I'm very much afraid of adding all its dependencies, which is why I'm not doing it. This hurts us, because we cannot improve our code quality.
Suggestion: We look into tools for managing dependencies. This could include Ant Ivy or Maven or others, we discuss what we learned, and then we switch. The goal would still be to run our .jar and .war files on a Debian stable machine only with standard packages. But building would require more than just Debian packages.
Let's discuss this at today's meeting. This is urgent, because we need to resolve this before merging other patches. Ugh.
Trac:
Priority: Medium to Highmentioned in issue #31193 (moved)