How to apply a patch to a spack package?

112 Views Asked by At

spack install gzip fails.

> spack install gzip
==> Installing gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep [1/1]
==> No binary for gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/5b/5b4fb14d38314e09f2fc8a1c510e7cd540a3ea0e3eb9b0420046b82c3bf41085.tar.gz
==> No patches needed for gzip
==> gzip: Executing phase: 'autoreconf'
==> gzip: Executing phase: 'configure'
==> gzip: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j16' 'V=1'

13 errors found in build log:
     1634    /umbc/rs/iharp/common/chhaya/spack/lib/spack/env/gcc/gcc -DHAVE_CONFIG_H -I. -I/tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/
             lib      -Wno-cast-qual -Wno-conversion -Wno-float-equal -Wno-sign-compare -Wno-undef -Wno-unused-function -Wno-unused-parameter -Wno-float-conversion -Wimplicit-fallthro
             ugh -Wno-pedantic -Wno-sign-conversion -Wno-type-limits -Wno-unsuffixed-float-constants -g -O2 -MT libgzip_a-fflush.o -MD -MP -MF .deps/libgzip_a-fflush.Tpo -c -o libgzip
             _a-fflush.o `test -f 'fflush.c' || echo '/tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/'`fflush.c
     1635    In file included from /usr/include/string.h:633,
     1636                     from ./string.h:41,
     1637                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/filename.h:24,
     1638                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/dirname.h:24,
     1639                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/stripslash.c:21:
  >> 1640    ./string.h:1091:1: error: expected identifier or '(' before '__extension__'
     1641     1091 | _GL_FUNCDECL_SYS (strndup, char *,
     1642          | ^~~~~~~~~~~~~~~~
  >> 1643    make[3]: *** [libgzip_a-stripslash.o] Error 1
     1644    make[3]: *** Waiting for unfinished jobs....
     1645    In file included from /usr/include/string.h:633,
     1646                     from ./string.h:41,
     1647                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/filename.h:24,
     1648                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/dirname.h:24,
     1649                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/dirname-lgpl.c:21:
  >> 1650    ./string.h:1091:1: error: expected identifier or '(' before '__extension__'
     1651     1091 | _GL_FUNCDECL_SYS (strndup, char *,
     1652          | ^~~~~~~~~~~~~~~~
     1653    mv -f .deps/libgzip_a-open-safer.Tpo .deps/libgzip_a-open-safer.Po
     1654    In file included from /usr/include/string.h:633,
     1655                     from ./string.h:41,
     1656                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/basename-lgpl.c:25:
  >> 1657    ./string.h:1091:1: error: expected identifier or '(' before '__extension__'
     1658     1091 | _GL_FUNCDECL_SYS (strndup, char *,
     1659          | ^~~~~~~~~~~~~~~~
  >> 1660    make[3]: *** [libgzip_a-dirname-lgpl.o] Error 1
     1661    mv -f .deps/libgzip_a-exitfail.Tpo .deps/libgzip_a-exitfail.Po
  >> 1662    make[3]: *** [libgzip_a-basename-lgpl.o] Error 1
     1663    mv -f .deps/libgzip_a-fd-hook.Tpo .deps/libgzip_a-fd-hook.Po
     1664    mv -f .deps/libgzip_a-creat-safer.Tpo .deps/libgzip_a-creat-safer.Po
     1665    mv -f .deps/libgzip_a-dup-safer-flag.Tpo .deps/libgzip_a-dup-safer-flag.Po
     1666    In file included from /usr/include/string.h:633,
     1667                     from ./string.h:41,
     1668                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/chdir-long.c:27:
  >> 1669    ./string.h:1091:1: error: expected identifier or '(' before '__extension__'
     1670     1091 | _GL_FUNCDECL_SYS (strndup, char *,
     1671          | ^~~~~~~~~~~~~~~~
     1672    mv -f .deps/libgzip_a-fclose.Tpo .deps/libgzip_a-fclose.Po
     1673    mv -f .deps/libgzip_a-cloexec.Tpo .deps/libgzip_a-cloexec.Po
     1674    In file included from /usr/include/string.h:633,
     1675                     from ./string.h:41,
     1676                     from /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/lib/openat-proc.c:30:
  >> 1677    ./string.h:1091:1: error: expected identifier or '(' before '__extension__'
     1678     1091 | _GL_FUNCDECL_SYS (strndup, char *,
     1679          | ^~~~~~~~~~~~~~~~
     1680    mv -f .deps/libgzip_a-fcntl.Tpo .deps/libgzip_a-fcntl.Po
     1681    mv -f .deps/libgzip_a-fd-safer-flag.Tpo .deps/libgzip_a-fd-safer-flag.Po
     1682    mv -f .deps/libgzip_a-opendir-safer.Tpo .deps/libgzip_a-opendir-safer.Po
  >> 1683    make[3]: *** [libgzip_a-openat-proc.o] Error 1
  >> 1684    make[3]: *** [libgzip_a-chdir-long.o] Error 1
     1685    mv -f .deps/libgzip_a-fflush.Tpo .deps/libgzip_a-fflush.Po
     1686    make[3]: Leaving directory `/tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/spack-build/lib'
  >> 1687    make[2]: *** [all] Error 2
     1688    make[2]: Leaving directory `/tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/spack-build/lib'
  >> 1689    make[1]: *** [all-recursive] Error 1
     1690    make[1]: Leaving directory `/tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/spack-build'
  >> 1691    make: *** [all] Error 2

See build log for details:
  /tmp/ckulkar1/spack-stage/spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-build-out.txt

I've located the offending file: .../spack-stage-gzip-1.12-hjaryhgrkevrdht64fqblm3pnyyd26ep/spack-src/spack-build/lib/string.h

I can comment out the offending codeblock:

# if 0  // __GNUC__ >= 11
/* For -Wmismatched-dealloc: Associate strndup with free or rpl_free.  */
_GL_FUNCDECL_SYS (strndup, char *,
                  (char const *__s, size_t __n)
                  _GL_ARG_NONNULL ((1))
                  _GL_ATTRIBUTE_MALLOC _GL_ATTRIBUTE_DEALLOC_FREE);
# endif

I can diff -u string.h string.h.modified > gzip.patch

Then I can fiddle the first 2 lines of the patch to get:

> cat patch
--- string.h    2023-10-31 21:07:57.752141778 -0400
+++ string.h    2023-10-31 21:13:39.283715230 -0400
@@ -1086,7 +1086,7 @@
 # endif
 _GL_CXXALIASWARN (strndup);
 #else
-# if __GNUC__ >= 11
+# if 0  // __GNUC__ >= 11
 /* For -Wmismatched-dealloc: Associate strndup with free or rpl_free.  */
 _GL_FUNCDECL_SYS (strndup, char *,
                   (char const *__s, size_t __n)

Now I can fiddle /home/ckulkar1/iharp_common/chhaya/spack/var/spack/repos/builtin/packages/gzip/package.py

class Gzip(AutotoolsPackage):
    """GNU Gzip is a popular data compression program originally written by
    Jean-loup Gailly for the GNU project."""

    homepage = "https://www.gnu.org/software/gzip/"
    url = "https://ftp.gnu.org/gnu/gzip/gzip-1.10.tar.gz"

    version("1.12", sha256="5b4fb14d38314e09f2fc8a1c510e7cd540a3ea0e3eb9b0420046b82c3bf41085")
    version("1.11", sha256="3e8a0e0c45bad3009341dce17d71536c4c655d9313039021ce7554a26cd50ed9")
    version("1.10", sha256="c91f74430bf7bc20402e1f657d0b252cb80aa66ba333a25704512af346633c68")

    # Gzip makes a recursive symlink if built in-source
    build_directory = "spack-build"

    patch('gzip.patch')

... and copy the gzip.patch into the same folder.

But when I run spack install gzip I get:

> spack install gzip
==> Installing gzip-1.12-zalk3vqblkxwfrgkyol4msfelfme64qw [1/1]
==> No binary for gzip-1.12-zalk3vqblkxwfrgkyol4msfelfme64qw found: installing from source
==> Using cached archive: /umbc/rs/iharp/common/chhaya/spack/var/spack/cache/_source-cache/archive/5b/5b4fb14d38314e09f2fc8a1c510e7cd540a3ea0e3eb9b0420046b82c3bf41085.tar.gz
The text leading up to this was:
--------------------------
|--- lib/string.h       2023-10-31 21:07:57.752141778 -0400
|+++ lib/string.h       2023-10-31 21:13:39.283715230 -0400
--------------------------
File to patch: 

... and I just don't know what to do next.

Maybe I can only patch over the SOURCE files from the downloaded .gzip?

I've tried downloading and unpacking https://ftp.gnu.org/gnu/gzip/gzip-1.12.tar.gz and inspecting gzip-1.12/lib/string.in.h (which I suppose gets converted into the offending string.h)

So should I be trying to apply a patch to this .in.h file?

And if so what should the folder-paths on the patch look like?

Somehow I need to tweak this string.h file after the spack installer has downloaded the .gzip and compiled the .in.h files to .h files. but before it tries to build the package.

I'm really stuck here. I could file an issue on spack, but spack has over 2k open issues.


Edit:

I was able to solve my original problem of bootstrapping spack (with spack install spack) by commenting out:

    # patch('gzip.patch')

... from my /path/to/spack/var/spack/repos/builtin/packages/gzip/package.py, then building gzip with the NATIVE compiler on the cluster:

> spack clean -a
> spack install gzip %[email protected]

..., modifying ~/.spack/packages.yaml:

packages:
  all:
    compiler: [[email protected]]
  gzip:
    externals:
      - spec: [email protected]
        prefix: [FOO]
    buildable: False

(where [FOO] is the output of spack location -i [email protected]), and running:

spack install spack

Now:

> spack find --explicit
-- linux-centos7-cascadelake / [email protected] -----------------------
[email protected]  [email protected]

-- linux-centos7-haswell / [email protected] ----------------------------
[email protected]

-- linux-centos7-sandybridge / [email protected] ------------------------
[email protected]

I won't post this as an answer as it's a workaround.

1

There are 1 best solutions below

0
On

You just needed to supply the full path from spack-src:

--- a/lib/string.in.h   2022-04-06 19:15:05.000000000 -0700
+++ b/lib/string.in.h   2024-03-27 14:17:12.208087106 -0700
@@ -579,7 +579,7 @@
 # endif
 _GL_CXXALIASWARN (strndup);
 #else
-# if __GNUC__ >= 11
+# if __GNUC__ >= 11 && !defined strndup
 /* For -Wmismatched-dealloc: Associate strndup with free or rpl_free.  */
 _GL_FUNCDECL_SYS (strndup, char *,
                   (char const *__s, size_t __n)

Alternatively, you could supply a working directory argument to the patch() function.