Opened 4 months ago

Last modified 3 months ago

#31390 needs_information defect

--enable-rust with pre-downloaded Rust dependencies fails: no .cargo-checksum.json files

Reported by: yurivict271 Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version: Tor: 0.4.0.5
Severity: Normal Keywords: rust, 041-backport, 040-backport?, 035-backport?, 042-deferred-20190918
Cc: chelseakomlo Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

In the FreeBSD port I am downloading the dependencies and supplying
digest-0.7.2 libc-0.2.39 directories as they appear on GitHub pointed by the TOR_RUST_DEPENDENCIES environment variable.

The build fails:

error: failed to load source for a dependency on `digest`

Caused by:
  Unable to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to update replaced source registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to load checksum `.cargo-checksum.json` of libc v0.2.39

Caused by:
  failed to read `/usr/ports/security/tor/work/tor-0.4.0.5/rs/libc-0.2.39/.cargo-checksum.json`

Is some command supposed to be run that would build .cargo-checksum.json ?

Child Tickets

Attachments (1)

tor.patch (2.7 KB) - added by yurivict271 4 months ago.
FreeBSD port patch

Download all attachments as: .zip

Change History (13)

comment:1 Changed 4 months ago by teor

Component: - Select a componentCore Tor/Tor
Keywords: 041-should added
Milestone: Tor: 0.4.2.x-final
Status: newneeds_information
Version: Tor: 0.4.0.5

What configure and make command lines are you using?
Please copy and paste them.

comment:2 Changed 4 months ago by yurivict271

configure:

===>  Configuring for asciidoc-8.6.10_1
(cd /usr/ports/textproc/asciidoc/work/asciidoc-8.6.10 &&  _LATE_CONFIGURE_ARGS="" ;  if [ -z "" ] &&  ./configure --help 2>&1 | /usr/bin/grep -- --localstatedir > /dev/null; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --localstatedir=/var" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--mandir'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --mandir=/usr/local/man" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--disable-silent-rules'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --disable-silent-rules" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--enable-jobserver\[.*#\]'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --enable-jobserver=8" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--infodir'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --infodir=/usr/local/share/info/" ;  fi ;  if [ -z "`./configure --version 2>&1 | /usr/bin/egrep -i '(autoconf.*2\.13|Unrecognized option)'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --build=amd64-portbld-freebsd12.0" ;  else  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} amd64-portbld-freebsd12.0" ;  fi ;  if ! /usr/bin/env CC="cc" CPP="cpp" CXX="c++"  CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -fno-strict-aliasing " CPPFLAGS="-fno-omit-frame-pointer" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer  "  LDFLAGS=" -fstack-protector-strong " LIBS=""  INSTALL="/usr/bin/install -c"  INSTALL_DATA="install  -m 0644"  INSTALL_LIB="install  -s -m 0644"  INSTALL_PROGRAM="install  -s -m 555"  INSTALL_SCRIPT="install  -m 555"  MAKE=gmake PYTHON="/usr/local/bin/python2.7" XDG_DATA_HOME=/usr/ports/textproc/asciidoc/work  XDG_CONFIG_HOME=/usr/ports/textproc/asciidoc/work  HOME=/usr/ports/textproc/asciidoc/work PATH=/usr/ports/textproc/asciidoc/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin SHELL=/bin/sh CONFIG_SHELL=/bin/sh LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 CONFIG_SITE=/usr/ports/Templates/config.site lt_cv_sys_max_cmd_len=262144 ./configure --prefix=/usr/local ${_LATE_CONFIGURE_ARGS}; then  echo "===>  Script \"configure\" failed unexpectedly.";  (echo "Please report the problem to sunpoet@FreeBSD.org [maintainer] and attach the \"/usr/ports/textproc/asciidoc/work/asciidoc-8.6.10/config.log\" including the output of the failure of your make command. Also, it might be a good idea to provide an overview of all packages installed on your system (e.g. a /usr/local/sbin/pkg-static info -g -Ea).") | /usr/bin/fmt 75 79 ;  false;  fi)

build:

===>  Building for tor-0.4.0.5_3
(cd /usr/ports/security/tor/work/tor-0.4.0.5; if ! /usr/bin/env TOR_RUST_DEPENDENCIES=/usr/ports/security/tor/work/tor-0.4.0.5/rs OPENSSLBASE=/usr OPENSSLDIR=/etc/ssl OPENSSLINC=/usr/include OPENSSLLIB=/usr/lib XDG_DATA_HOME=/usr/ports/security/tor/work  XDG_CONFIG_HOME=/usr/ports/security/tor/work  HOME=/usr/ports/security/tor/work PATH=/usr/ports/security/tor/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -fno-strict-aliasing -std=gnu99 "  CPP="cpp" CPPFLAGS="-fno-omit-frame-pointer"  LDFLAGS="  -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer  "  MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" gmake -f Makefile -j8  all; then  if [ -n "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer." ] ; then  echo "===> Compilation failed unexpectedly.";  (echo "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer.") | /usr/bin/fmt 75 79 ;  fi;  false;  fi)

comment:3 in reply to:  2 Changed 4 months ago by teor

Replying to yurivict271:

configure:

===>  Configuring for asciidoc-8.6.10_1
(cd /usr/ports/textproc/asciidoc/work/asciidoc-8.6.10 &&  _LATE_CONFIGURE_ARGS="" ;  if [ -z "" ] &&  ./configure --help 2>&1 | /usr/bin/grep -- --localstatedir > /dev/null; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --localstatedir=/var" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--mandir'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --mandir=/usr/local/man" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--disable-silent-rules'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --disable-silent-rules" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--enable-jobserver\[.*#\]'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --enable-jobserver=8" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--infodir'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --infodir=/usr/local/share/info/" ;  fi ;  if [ -z "`./configure --version 2>&1 | /usr/bin/egrep -i '(autoconf.*2\.13|Unrecognized option)'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --build=amd64-portbld-freebsd12.0" ;  else  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} amd64-portbld-freebsd12.0" ;  fi ;  if ! /usr/bin/env CC="cc" CPP="cpp" CXX="c++"  CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -fno-strict-aliasing " CPPFLAGS="-fno-omit-frame-pointer" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer  "  LDFLAGS=" -fstack-protector-strong " LIBS=""  INSTALL="/usr/bin/install -c"  INSTALL_DATA="install  -m 0644"  INSTALL_LIB="install  -s -m 0644"  INSTALL_PROGRAM="install  -s -m 555"  INSTALL_SCRIPT="install  -m 555"  MAKE=gmake PYTHON="/usr/local/bin/python2.7" XDG_DATA_HOME=/usr/ports/textproc/asciidoc/work  XDG_CONFIG_HOME=/usr/ports/textproc/asciidoc/work  HOME=/usr/ports/textproc/asciidoc/work PATH=/usr/ports/textproc/asciidoc/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin SHELL=/bin/sh CONFIG_SHELL=/bin/sh LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 CONFIG_SITE=/usr/ports/Templates/config.site lt_cv_sys_max_cmd_len=262144 ./configure --prefix=/usr/local ${_LATE_CONFIGURE_ARGS}; then  echo "===>  Script \"configure\" failed unexpectedly.";  (echo "Please report the problem to sunpoet@FreeBSD.org [maintainer] and attach the \"/usr/ports/textproc/asciidoc/work/asciidoc-8.6.10/config.log\" including the output of the failure of your make command. Also, it might be a good idea to provide an overview of all packages installed on your system (e.g. a /usr/local/sbin/pkg-static info -g -Ea).") | /usr/bin/fmt 75 79 ;  false;  fi)

This is the asciidoc configure, but we need to see the tor configure.

build:

===>  Building for tor-0.4.0.5_3
(cd /usr/ports/security/tor/work/tor-0.4.0.5; if ! /usr/bin/env TOR_RUST_DEPENDENCIES=/usr/ports/security/tor/work/tor-0.4.0.5/rs OPENSSLBASE=/usr OPENSSLDIR=/etc/ssl OPENSSLINC=/usr/include OPENSSLLIB=/usr/lib XDG_DATA_HOME=/usr/ports/security/tor/work  XDG_CONFIG_HOME=/usr/ports/security/tor/work  HOME=/usr/ports/security/tor/work PATH=/usr/ports/security/tor/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -fno-strict-aliasing -std=gnu99 "  CPP="cpp" CPPFLAGS="-fno-omit-frame-pointer"  LDFLAGS="  -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer  "  MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" gmake -f Makefile -j8  all; then  if [ -n "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer." ] ; then  echo "===> Compilation failed unexpectedly.";  (echo "Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer.") | /usr/bin/fmt 75 79 ;  fi;  false;  fi)

"Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer."

Did you try to set MAKE_JOBS_UNSAFE=yes?

We have had some trouble with parallel builds in the past, but we haven't been able to reproduce the errors, so they are hard to fix.

comment:4 Changed 4 months ago by yurivict271

I tried MAKE_JOBS_UNSAFE=yes but the sequential build breaks with the same message.

I am trying to add the RUST option to build the Rust code that can be optionally enabled in the project.

comment:5 Changed 4 months ago by teor

Thanks.

I also need to see the tor configure to reproduce this issue. You pasted the asciidoc configure.

And it would help to know your rust version.

comment:6 Changed 4 months ago by yurivict271

rust-nightly-1.38.0.20190811_1

Sorry for the confusion, here is tor's configure:

===>  Configuring for tor-0.4.0.5_3
(cd /usr/ports/security/tor/work/tor-0.4.0.5 &&  _LATE_CONFIGURE_ARGS="" ;  if [ -z "" ] &&  ./configure --help 2>&1 | /usr/bin/grep -- --localstatedir > /dev/null; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --localstatedir=/var" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--mandir'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --mandir=/usr/local/man" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--disable-silent-rules'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --disable-silent-rules" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--enable-jobserver\[.*#\]'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --enable-jobserver=1" ;  fi ;  if [ ! -z "`./configure --help 2>&1 | /usr/bin/grep -- '--infodir'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --infodir=/usr/local/share/info/" ;  fi ;  if [ -z "`./configure --version 2>&1 | /usr/bin/egrep -i '(autoconf.*2\.13|Unrecognized option)'`" ]; then  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} --build=amd64-portbld-freebsd12.0" ;  else  _LATE_CONFIGURE_ARGS="${_LATE_CONFIGURE_ARGS} amd64-portbld-freebsd12.0" ;  fi ;  if ! /usr/bin/env CC="cc" CPP="cpp" CXX="c++"  CFLAGS="-O2 -pipe -fno-omit-frame-pointer  -fstack-protector-strong -fno-strict-aliasing -std=gnu99 " CPPFLAGS="-fno-omit-frame-pointer" CXXFLAGS="-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing -fno-omit-frame-pointer  "  LDFLAGS="  -fstack-protector-strong " LIBS=""  INSTALL="/usr/bin/install -c"  INSTALL_DATA="install  -m 0644"  INSTALL_LIB="install  -s -m 0644"  INSTALL_PROGRAM="install  -s -m 555"  INSTALL_SCRIPT="install  -m 555"  TOR_CPPFLAGS_libevent="-I/usr/local/include"  TOR_LDFLAGS_libevent="-L/usr/local/lib/"  TOR_LIBEVENT_LIBS="-levent" TOR_RUST_DEPENDENCIES=/usr/ports/security/tor/work/tor-0.4.0.5/rs TOR_ZSTD_LIBS="-lzstd" MAKE=gmake PKG_CONFIG=pkgconf XDG_DATA_HOME=/usr/ports/security/tor/work  XDG_CONFIG_HOME=/usr/ports/security/tor/work  HOME=/usr/ports/security/tor/work PATH=/usr/ports/security/tor/work/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin SHELL=/bin/sh CONFIG_SHELL=/bin/sh CONFIG_SITE=/usr/ports/Templates/config.site lt_cv_sys_max_cmd_len=262144 ./configure --with-openssl-dir="/usr" --enable-lzma --enable-rust --enable-zstd --enable-gcc-hardening --enable-linker-hardening --with-malloc=tcmalloc --prefix=/usr/local ${_LATE_CONFIGURE_ARGS}; then  echo "===>  Script \"configure\" failed unexpectedly.";  (echo "Please report the problem to yuri@FreeBSD.org [maintainer] and attach the \"/usr/ports/security/tor/work/tor-0.4.0.5/config.log\" including the output of the failure of your make command. Also, it might be a good idea to provide an overview of all packages installed on your system (e.g. a /usr/local/sbin/pkg-static info -g -Ea).") | /usr/bin/fmt 75 79 ;  false;  fi)

comment:7 Changed 4 months ago by teor

Based on https://github.com/torproject/tor/blob/master/doc/HACKING/GettingStartedRust.md , it looks like you might be missing a "crates" directory in TOR_RUST_DEPENDENCIES:

TOR_RUST_DEPENDENCIES=pwd/src/ext/rust/crates ./configure --enable-rust

If that doesn't work, please check that online mode works:

./configure --enable-rust --enable-cargo-online-mode

comment:8 Changed 4 months ago by teor

I also need to see a copy of the command you use to download the TOR_RUST_DEPENDENCIES directory.

comment:9 Changed 4 months ago by yurivict271

The online mode works, I checked.

The directory contains crate sources as they are downloaded from GitHub in tarballs with the same versions.

This isn't done through scripts, rather through port tags.

I can share the port patch.

Changed 4 months ago by yurivict271

Attachment: tor.patch added

FreeBSD port patch

comment:10 in reply to:  7 Changed 4 months ago by teor

Cc: chelseakomlo added

Replying to teor:

Based on https://github.com/torproject/tor/blob/master/doc/HACKING/GettingStartedRust.md , it looks like you might be missing a "crates" directory in TOR_RUST_DEPENDENCIES:

TOR_RUST_DEPENDENCIES=pwd/src/ext/rust/crates ./configure --enable-rust

If that doesn't work, please check that online mode works:

./configure --enable-rust --enable-cargo-online-mode

I'm still struggling to work out what's going on here.

It could be an issue with your environment.

Or maybe cargo has a new requirement for .cargo-checksum.json and we need to update the tor rust dependencies repository with the latest version of cargo?

I've cc'd komlo for advice.

comment:11 Changed 4 months ago by nickm

Keywords: 041-backport 040-backport? 035-backport? added; 041-should removed

comment:12 Changed 3 months ago by nickm

Keywords: 042-deferred-20190918 added
Milestone: Tor: 0.4.2.x-finalTor: unspecified

Deferring various tickets from 0.4.2 to Unspecified.

Note: See TracTickets for help on using tickets.