Opened 3 months ago

Closed 2 months ago

#31197 closed enhancement (wontfix)

Upgrade metrics-web to Debian buster libraries

Reported by: karsten Owned by: metrics-team
Priority: High Milestone:
Component: Metrics/Website Version:
Severity: Normal Keywords:
Cc: metrics-team Actual Points:
Parent ID: #31193 Points:
Reviewer: Sponsor:


We'll need to upgrade metrics-web to Debian buster libraries, though we should first finish the metrics-base review in #31194.

Child Tickets

Change History (2)

comment:1 Changed 3 months ago by karsten

Priority: MediumHigh

I'm hitting a brick wall here.

Upgrading embedded Jetty and its internal dependencies on Tomcat9 turned out to be as complicated as expected. Here's what I have right now, and it's not working:

diff --git a/build.xml b/build.xml
index 0c9fa0e4..0d4979cb 100644
--- a/build.xml
+++ b/build.xml
@@ -9,9 +9,9 @@
   <property name="javadoc-title" value="MetricsWeb API Documentation"/>
   <property name="implementation-title" value="metrics-web" />
   <property name="release.version" value="1.2.0-dev" />
-  <property name="metricslibversion" value="2.4.0" />
+  <property name="metricslibversion" value="2.6.2" />
   <property name="exoneratorversion" value="4.1.0" />
-  <property name="jetty.version" value="-9.2.21.v20170120" />
+  <property name="jetty.version" value="-9.4.15.v20190215" />
   <property name="warfile"
@@ -38,27 +38,27 @@
   <patternset id="common" >
-    <include name="commons-codec-1.10.jar"/>
-    <include name="commons-lang3-3.5.jar"/>
+    <include name="commons-codec-1.11.jar"/>
+    <include name="commons-lang3-3.8.jar"/>
     <include name="commons-math3-3.6.1.jar"/>
-    <include name="jackson-annotations-2.8.6.jar"/>
-    <include name="jackson-core-2.8.6.jar"/>
-    <include name="jackson-databind-2.8.6.jar"/>
-    <include name="logback-core-1.1.9.jar" />
-    <include name="logback-classic-1.1.9.jar" />
+    <include name="jackson-annotations-2.9.8.jar"/>
+    <include name="jackson-core-2.9.8.jar"/>
+    <include name="jackson-databind-2.9.8.jar"/>
+    <include name="logback-core-1.2.3.jar" />
+    <include name="logback-classic-1.2.3.jar" />
     <include name="REngine.jar"/>
     <include name="Rserve.jar"/>
-    <include name="slf4j-api-1.7.22.jar"/>
+    <include name="slf4j-api-1.7.25.jar"/>
   <patternset id="library" >
     <patternset refid="common" />
-    <include name="metrics-lib-${metricslibversion}.jar"/>
-    <include name="commons-compress-1.13.jar"/>
+    <include name="metrics-lib-${metricslibversion}-thin.jar"/>
+    <include name="commons-compress-1.18.jar"/>
     <include name="commons-math3-3.6.1.jar"/>
-    <include name="postgresql-9.4.1212.jar"/>
+    <include name="postgresql-42.2.5.jar"/>
     <include name="servlet-api-3.1.jar"/>
-    <include name="xz-1.6.jar"/>
+    <include name="xz-1.8.jar"/>
   <patternset id="web" >
@@ -76,18 +76,18 @@
@@ -76,18 +76,18 @@
   <patternset id="webruntime" >
-    <include name="asm-5.2.jar" />
-    <include name="asm-commons-5.2.jar" />
-    <include name="commons-compress-1.13.jar"/>
+    <include name="asm-all-7.0.jar" />
+    <include name="commons-compress-1.18.jar"/>
     <include name="jetty9-plus${jetty.version}.jar"/>
     <include name="jetty9-jndi${jetty.version}.jar"/>
     <include name="jetty9-apache-jsp${jetty.version}-tweaked.jar" />
     <include name="metrics-lib-${metricslibversion}.jar"/>
     <include name="taglibs-standard-spec-1.2.5.jar"/>
-    <include name="tomcat8-embed-jasper-8.5.14.jar" />
-    <include name="tomcat8-embed-el-8.5.14.jar" />
-    <include name="tomcat8-embed-core-8.5.14.jar" />
-    <include name="eclipse-ecj-3.11.1.jar" />
+    <include name="tomcat9-annotations-api-9.0.16.jar" />
+    <include name="tomcat9-embed-jasper-9.0.16-tweaked.jar" />
+    <include name="tomcat9-embed-el-9.0.16.jar" />
+    <include name="tomcat9-embed-core-9.0.16.jar" />
+    <include name="eclipse-ecj-3.16.0.jar" />
   <patternset id="runtime" >
@@ -139,16 +139,22 @@
   <target name="war"
     <echo message="Removing problematic service definitions from"/>
-    <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and using "/>
-    <echo message="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar below." />
+    <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and "/>
+    <echo message="${libs}/tomcat9-embed-jasper-9.0.16.jar." />
     <delete file="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar"
             quiet="true" />
+    <delete file="${libs}/tomcat9-embed-jasper-9.0.16-tweaked.jar"
+            quiet="true" />
     <jar destfile="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar">
       <zipfileset src="${libs}/jetty9-apache-jsp${jetty.version}.jar" >
         <exclude name="**/javax.servlet.ServletContainerInitializer"/>
+    <jar destfile="${libs}/tomcat9-embed-jasper-9.0.16-tweaked.jar">
+      <zipfileset src="${libs}/tomcat9-embed-jasper-9.0.16.jar" >
+        <exclude name="**/javax.servlet.ServletContainerInitializer"/>
+      </zipfileset>
+    </jar>
     <antcall target="generate-ml-javadoc" />

The error messages aren't very helpful, but I think it's related to some internal changes in Jetty or embedded Tomcat using a different library for JSP compilation.

I feel like we're doing it wrong.

The original idea of using libraries shipped with Debian stable was a good one, because it would allow anyone on a Debian machine to build our software with minimal effort.

But this doesn't come for free. We're basically doing manual dependency management not only for our dependencies but also for their dependencies.

Another major drawback, in addition to having to fix the issue above, is that it's almost prohibitively expensive to add new dependencies. For example, I'd very much want to add something like FindBugs. But I'm very much afraid of adding all its dependencies, which is why I'm not doing it. This hurts us, because we cannot improve our code quality.

Suggestion: We look into tools for managing dependencies. This could include Ant Ivy or Maven or others, we discuss what we learned, and then we switch. The goal would still be to run our .jar and .war files on a Debian stable machine only with standard packages. But building would require more than just Debian packages.

Let's discuss this at today's meeting. This is urgent, because we need to resolve this before merging other patches. Ugh.

comment:2 Changed 2 months ago by karsten

Resolution: wontfix
Status: newclosed

Closing all tickets related to Debian buster libraries, now that we're using Apache Ivy for external dependency management.

Note: See TracTickets for help on using tickets.