Opened 7 years ago

Last modified 7 months ago

#6311 new enhancement

Migrate TOR_SEARCH_LIBRARY to use pkg-config

Reported by: nickm Owned by:
Priority: High Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-relay, build autotools pkgconfig autoconf
Cc: catalyst, ahf Actual Points:
Parent ID: Points: 5
Reviewer: Sponsor:

Description

TOR_SEARCH_LIBRARY is a hard-to-maintain fragile garden of venomous snakes, all waiting to bite you at the same time.

I'm working on a patch to migrate to pkg-config. pkg-config is everywhere these days, and where it isn't, we can fall back to the old thing for a while.

Child Tickets

TicketTypeStatusOwnerSummary
#1354defectassignednickmConfiguring Tor with --with*dir gives wrong directories
#4692defectneeds_revisionIf only a working static OpenSSL is available, ./configure fails
#10304defectclosedlibevent2 detection fails on FreeBSD 9.2
#13817defectnewUntange kludgey library detection, particularly for SSL forks

Change History (24)

comment:1 Changed 7 years ago by nickm

Status: newneeds_review

See branch "pkgconfig" in my public repo. It needs more work; the commit message explains what it needs. (Specifically, static var support.) It also needs way more testing.

comment:2 Changed 7 years ago by nickm

Summary: Migrate TOR_SEARCH_LIBRAR Y to use pkg-configMigrate TOR_SEARCH_LIBRARY to use pkg-config

comment:3 Changed 6 years ago by nickm

Owner: set to nickm
Priority: normalmajor
Status: needs_reviewassigned

comment:4 Changed 6 years ago by nickm

Keywords: tor-relay added

comment:5 Changed 6 years ago by nickm

Component: Tor RelayTor

comment:6 Changed 6 years ago by nickm

Keywords: build added
Milestone: Tor: 0.2.4.x-finalTor: 0.2.5.x-final

Ugh. I REALLY want this one, but I don't think I can get it right by the small-feature deadline

comment:7 Changed 6 years ago by nickm

Returning to look at this again, I think I actually need to do some design here. So let's see. The current behavior is, more or less:

  • Check the command line for a --with-libraryname-dir=xxx option. If so, set trylibrarydir to xxx.
  • Save the initial values of LDFLAGS, LIBS, and CPPFLAGS.
  • Iterating over $trylibrarydir, "", /usr/local, /usr/pkg, and other directories optionally specified in $8:
    • Try to extend the saved LDFLAGS and CPPFLAGS with appropriate -L and -I options for that directory.
    • Extend LIBS with the library options specified in $3
    • See if we can build and link a program that includes no headers for the library, and tries to require one of its identifiers
    • If so, see if we can compile a program that does include headers for the library
    • If so, this is the one we want: remember that directory, and break out of the loop.
  • Then, extend LDFLAGS and CPPFLAGS with the options given by the chosen directory, and extend LIBS with the chosen libs.
  • Additionally, set TOR_LDFLAGS_libraryname and TOR_CPPFLAGS_libraryname and TOR_LIBDIR_libraryname to appropriate values, and AC_SUBST the CPPDLAGS and LDFLAGS variables.
  • Then, see whether we need any rpath-style options to link the program by iterating over various possibilities and seeing if they make the test program runnable.

So right now our outputs are a series of changes to LDFLAGS and CPPFLAGS and LIBS that carry across all the things we build (although some callers immediately roll them back), as well as the TOR_CPPFLAGS_* / TOR_LDFLAGS_* / TOR_LIBDIR_* values.

We need to make our pkg-config alternative produce each of those, or remove the need for those.

comment:8 in reply to:  7 Changed 6 years ago by nickm

Replying to nickm:

Returning to look at this again, I think I actually need to do some design here. So let's see. The current behavior is, more or less:

  • Check the command line for a --with-libraryname-dir=xxx option. If so, set trylibrarydir to xxx.
  • Save the initial values of LDFLAGS, LIBS, and CPPFLAGS.
  • Iterating over $trylibrarydir, "", /usr/local, /usr/pkg, and other directories optionally specified in $8:
    • Try to extend the saved LDFLAGS and CPPFLAGS with appropriate -L and -I options for that directory.
    • Extend LIBS with the library options specified in $3
    • See if we can build and link a program that includes no headers for the library, and tries to require one of its identifiers
    • If so, see if we can compile a program that does include headers for the library
    • If so, this is the one we want: remember that directory, and break out of the loop.
  • Then, extend LDFLAGS and CPPFLAGS with the options given by the chosen directory, and extend LIBS with the chosen libs.
  • Additionally, set TOR_LDFLAGS_libraryname and TOR_CPPFLAGS_libraryname and TOR_LIBDIR_libraryname to appropriate values, and AC_SUBST the CPPDLAGS and LDFLAGS variables.
  • Then, see whether we need any rpath-style options to link the program by iterating over various possibilities and seeing if they make the test program runnable.
  • Then, roll back all of CPPFLAGS, LDFLAGS, and LIBS to their original values.


So right now our outputs are a series of changes to LDFLAGS and CPPFLAGS and LIBS that carry across all the things we build (although some callers immediately roll them back), as well as the TOR_CPPFLAGS_* / TOR_LDFLAGS_* / TOR_LIBDIR_* values.

Actually, LDFLAGS and CPPFLAGS and LIBS are rolled back in all cases. The TOR_{CPPFLAGS,LDFLAGS,LIBDIR}_* values are the only outputs.

comment:9 Changed 5 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.???

comment:10 Changed 4 years ago by nickm_mobile

As a side note, since people are getting interested in this ticket again...

The hard part isn't supporting pkg-config. The hard part is doing so while still working with systems that don't have pkg-config, and while keeping our existing search semantics working. If we are willing to take the risk of making people update their build scripts and /or install pkgconfig, and if we're willing to rip out our library search code, this gets much easier.

comment:11 Changed 4 years ago by nickm

Milestone: Tor: 0.2.???Tor: 0.2.7.x-final

Tentatively move some tickets to 0.2.7

comment:12 Changed 4 years ago by nickm

Keywords: 027-triaged-1-out added

Marking triaged-out items from first round of 0.2.7 triage.

comment:13 Changed 4 years ago by nickm

Milestone: Tor: 0.2.7.x-finalTor: 0.2.???

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

comment:14 Changed 3 years ago by Sebastian

Severity: Normal

Hrm, breaking backwards compat for 0.3.0 is looking more and more appealing to me.

comment:15 Changed 2 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:16 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:17 Changed 21 months ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:18 Changed 21 months ago by nickm

Keywords: 027-triaged-in added

comment:19 Changed 21 months ago by nickm

Keywords: 027-triaged-in removed

comment:20 Changed 21 months ago by nickm

Keywords: 027-triaged-1-out removed

comment:21 Changed 21 months ago by nickm

Owner: nickm deleted
Points: 5

comment:22 Changed 21 months ago by nickm

Status: assignednew

comment:23 Changed 21 months ago by nickm

Cc: catalyst ahf added
Keywords: autotools pkgconfig autoconf added

comment:24 Changed 7 months ago by Hello71

So it's been many years since pkg-config was the de facto standard, but nobody has been willing to say "let's just break backwards compat". Therefore, I propose this plan: switch to pkg-config everywhere, with a fallback to the old logic with a warning. Two releases later, require an explicit flag to switch to the old logic. Two releases after that, delete the old logic. It looks like "just break it" more or less means "just wait another two years" anyways; this plan takes about as long but results in eventual task completion.

I think it would help to write a short document on how to migrate from the old logic to pkg-config, including what to do if you refuse to install pkg-config (i.e. export everything_CFLAGS/LIBS).

Note: See TracTickets for help on using tickets.