Opened 5 weeks ago

Closed 5 weeks ago

#31979 closed defect (fixed)

TOPL: Sort dependency list

Reported by: sisbell Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-rbm, ff68-esr, tbb-9.0-must-alpha, TorBrowserTeam201910
Cc: boklm, sisbell, gk Actual Points:
Parent ID: #30324 Points:
Reviewer: Sponsor:

Description

Sort dependency list and follow latest how to update gradle dependencies instructions.

Child Tickets

Change History (11)

comment:2 Changed 5 weeks ago by sisbell

Status: newneeds_review

I regenerated the list of dependencies using the exact instructions in the updated projects/common/how-to-create-gradle-dependencies-list.txt file. I also verified that firefox and tor-android-service projects already have the dependencies file generated from these instructions.

https://github.com/sisbell/tor-browser-build/commit/e5d93174b0f7e40511fd74dc72fe167e47147d20

comment:3 in reply to:  2 Changed 5 weeks ago by gk

Status: needs_reviewneeds_information

Replying to sisbell:

I regenerated the list of dependencies using the exact instructions in the updated projects/common/how-to-create-gradle-dependencies-list.txt file. I also verified that firefox and tor-android-service projects already have the dependencies file generated from these instructions.

https://github.com/sisbell/tor-browser-build/commit/e5d93174b0f7e40511fd74dc72fe167e47147d20

Do we really need to bump gradle_dependencies_version? After all it's just a sorting of dependencies which should not affect the gradle dependencies themselves.

comment:4 Changed 5 weeks ago by gk

Status: needs_informationneeds_revision

Alright. Looking at gradle-dependencies-list.txt as we have it on master results in 277 entries we need to download. Looking at the same file in your branch after sorting results in 313 entries to download. So, there is something going wrong here. If we only need 277 entries to build (TOPL) then we should take those (maybe we need even less). Either way, this needs revision.

Last edited 5 weeks ago by gk (previous) (diff)

comment:5 in reply to:  4 Changed 5 weeks ago by sisbell

Replying to gk:

Alright. Looking at gradle-dependencies-list.txt as we have it on master results in 277 entries we need to download. Looking at the same file in your branch after sorting results in 313 entries to download. So, there is something going wrong here. If we only need 277 entries to build (TOPL) then we should take those (maybe we need even less). Either way, this needs revision.

The resolver can download more than it actually needs at compile time. So its possible that it needs less. I can look at what's going on with the extra entries but the method used to generate the lists from the logs is different between these two lists.

comment:6 Changed 5 weeks ago by sisbell

comment:7 in reply to:  6 Changed 5 weeks ago by sisbell

I checked the method I was using in August (from the link below) and it generates 493 URLs from the logs. The program I use for downloading and creating the gradle-dependency list downloads 313. So the most recent method and the method is August match. So 313 is the correct count. 277 must have come from some earlier version that I was working in (possibly different gradle plugin/version) entirely.

Replying to sisbell:

I believe this contains the command that generated 277

https://github.com/sisbell/tor-browser-build/commit/221348850ecacda7c6eac7681603b720adb14f5e

comment:8 Changed 5 weeks ago by sisbell

Its interesting that the additional dependencies are all from the android tools. Things like ddms are obviously not used.

Checking the dependencies of the gradle plugin 3.1, we can see these additional ones are being pulled in by the android gradle plugin.

https://android.googlesource.com/platform/prebuilts/gradle-plugin/+/6ef3f69959e89df946045097d84a3502c64a18c1

So the 313 count looks correct.

 https://repo.spring.io/plugins-release/com/android/tools/analytics-library/protos/26.1.0/protos-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/analytics-library/protos/26.1.0/protos-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/analytics-library/shared/26.1.0/shared-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/analytics-library/shared/26.1.0/shared-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/analytics-library/tracker/26.1.0/tracker-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/analytics-library/tracker/26.1.0/tracker-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/annotations/26.1.0/annotations-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/annotations/26.1.0/annotations-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/build/apksig/3.1.0/apksig-3.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/build/apksig/3.1.0/apksig-3.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/build/builder-model/3.1.0/builder-model-3.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/build/builder-model/3.1.0/builder-model-3.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/build/builder-test-api/3.1.0/builder-test-api-3.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/build/builder-test-api/3.1.0/builder-test-api-3.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/build/builder/3.1.0/builder-3.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/build/builder/3.1.0/builder-3.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/build/gradle-api/3.1.0/gradle-api-3.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/build/gradle-api/3.1.0/gradle-api-3.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/build/manifest-merger/26.1.0/manifest-merger-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/build/manifest-merger/26.1.0/manifest-merger-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/common/26.1.0/common-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/common/26.1.0/common-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/ddms/ddmlib/26.1.0/ddmlib-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/ddms/ddmlib/26.1.0/ddmlib-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/dvlib/26.1.0/dvlib-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/dvlib/26.1.0/dvlib-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/layoutlib/layoutlib-api/26.1.0/layoutlib-api-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/layoutlib/layoutlib-api/26.1.0/layoutlib-api-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/lint/lint-gradle-api/26.1.0/lint-gradle-api-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/lint/lint-gradle-api/26.1.0/lint-gradle-api-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/repository/26.1.0/repository-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/repository/26.1.0/repository-26.1.0.pom
 https://repo.spring.io/plugins-release/com/android/tools/sdk-common/26.1.0/sdk-common-26.1.0.jar
 https://repo.spring.io/plugins-release/com/android/tools/sdk-common/26.1.0/sdk-common-26.1.0.pom

comment:9 Changed 5 weeks ago by sisbell

So the logs to tor-onion-proxy was unique in the sense it does download the same artifact twice from two different locations. These downloads occur on different threads so I think one resolver is unaware of what the other resolver is doing.

To be able to remove these duplicates, we do need to rev the characters in each line that then match multiple fields (since artifacts with different groupIds have the same artifact name and version). So I added that to the documentation and tested it out. We are back to the 277 artifacts after this.

https://github.com/sisbell/tor-browser-build/commit/b68e087cc157920c3af14cd82b6047f7d5c6e13b

comment:10 Changed 5 weeks ago by sisbell

Status: needs_revisionneeds_review

comment:11 Changed 5 weeks ago by gk

Resolution: fixed
Status: needs_reviewclosed

The sorted result looks good to me. Cherry-picked to master (commit 8b1f9f1e655ed7cad832d8bfe2d1ee617231688e), thanks.

Note: See TracTickets for help on using tickets.