why does spack external find fail to find perl?

Question

I am working on free and open-source software for the Earth science community, using spack to install the massive list of dependencies needed for some forecast models.

The goal is that scientists all over the world (potentially with little build experience) can correctly install all the software needed to start working with these models.

I am following the instructions here: https://spack-tutorial.readthedocs.io/en/latest/tutorial_basics.html

I am trying to use spack external find to find perl, which is already on my machine. But spack does not find it:

ed@Pooh-Bah:~/spack$ spack external find
==> The following specs have been detected on this system and added to /home/ed/spack/var/spack/environments/e2/spack.yaml
[email protected]  [email protected]  [email protected]  [email protected]
ed@Pooh-Bah:~/spack$ spack external find perl
==> No new external packages detected
ed@Pooh-Bah:~/spack$ perl --version

This is perl 5, version 34, subversion 0 (v5.34.0) built for x86_64-linux-gnu-thread-multi
(with 58 registered patches, see perl -V for more detail)

How can I get spack to find the existing installation of perl?

I understand that I can manually add perl to a spack configuration file. But perl is just an example of several packages that spack is not finding that I need.

I want spack to be able to find as much as it can from the user's system. If the user has perl, then use it. If the user has mysql, then use that too.

I expected spack external find to find all such packages. But instead if finds just a few. What would be really helpful to the scientist user out there is if it would find as many as possible. How do I make that happen?

Using the -d option, I get some more output, but I'm not sure how it helps:

        spack -d external find cmake gmake
==> [2023-09-25-05:07:39.578423] Reading config from file /home/ed/tmp12345/spack/etc/spack/defaults/config.yaml
==> [2023-09-25-05:07:39.591727] Reading config from file /home/ed/tmp12345/spack/var/spack/environments/g2-env/spack.yaml
==> [2023-09-25-05:07:39.594273] Reading config from file /home/ed/tmp12345/spack/etc/spack/defaults/config.yaml
==> [2023-09-25-05:07:39.606558] Using environment 'g2-env'
==> [2023-09-25-05:07:39.611549] Imported external from built-in commands
==> [2023-09-25-05:07:39.612392] Imported external from built-in commands
==> [2023-09-25-05:07:39.612667] Reading config from file /home/ed/tmp12345/spack/etc/spack/defaults/repos.yaml
==> [2023-09-25-05:07:39.710738] Reading config from file /home/ed/tmp12345/spack/etc/spack/defaults/packages.yaml
==> [2023-09-25-05:07:39.730591] Reading config from file /home/ed/.spack/packages.yaml
==> [2023-09-25-05:07:59.709544] '/bin/make' '--version'
==> [2023-09-25-05:07:59.709534] '/bin/cmake' '--version'
==> [2023-09-25-05:07:59.711458] no bin/ dir found in /bin. Cannot add it as a Spack package
==> [2023-09-25-05:07:59.711587] '/usr/bin/make' '--version'
==> [2023-09-25-05:07:59.715755] no bin/ dir found in /bin. Cannot add it as a Spack package
==> [2023-09-25-05:07:59.715920] '/usr/bin/cmake' '--version'
==> [2023-09-25-05:07:59.748980] No new external packages detected
ed@Pooh-Bah:~/tmp12345$ 

Answer 1

You can try to manually add the external package to your ~/.spack/packages.yml adding something like

  packages:
    perl:
      externals:
      - spec: [email protected] %[email protected]
        prefix: /usr

Source: https://spack-tutorial.readthedocs.io/en/latest/tutorial_configuration.html?highlight=perl#external-packages