Error from Pants pex install: "FAILURE: Unable to detect a suitable interpreter for compatibilities: (Conflicting targets: )"

Question

I am using pants version 0.0.32 + a few more commits from master.

I want to use a pex distribution that bundles in both Linux and MacOS support. I am building the pex from the Pants OSS repo using:

git clean -fdx
PANTS_DEV=1 ./pants binary ./src/python/pants/bin:pants

I took the freshly created file dist/pants.pex and it works fine on my mac. When I try to run it in my Linux environment, it exits with an error:

./pants test foo
Running Pants version square-20150331-02
...
11:53:22 00:08     [pytest]
11:53:22 00:08       [run]
                 WARN] /data/app/kochiku-worker/.pex/install/requests-2.5.3-py2.py3-none-any.whl.ed9d28acc3c467062b25b9dc2e2084d6efa8ee1e/requests-2.5.3-py2.py3-none-any.whl/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning


FAILURE: Unable to detect a suitable interpreter for compatibilities:  (Conflicting targets: )

[32m
           Waiting for background workers to finish.[0m[31m
           FAILURE[0m

What I think is happening here is that for some reason, pants is running under Python 2.6. I investigated the security error and it appears it can be accidentally tripped when using python version earlier than python 2.7.2. The version of python installed on the machine I'm trying to use is installed in /usr/local/bin and is version 2.7.9

Answer 1

OK - Grepping for that FAILURE message at version 0.0.32 I find this python_task.py code:

def select_interpreter_for_targets(self, targets):
  """Pick an interpreter compatible with all the specified targets."""
  allowed_interpreters = OrderedSet(self.interpreter_cache.interpreters)
  targets_with_compatibilities = []  # Used only for error messages.

  # Constrain allowed_interpreters based on each target's compatibility requirements.
  for target in targets:
    if target.is_python and hasattr(target, 'compatibility') and target.compatibility:
      targets_with_compatibilities.append(target)
      compatible_with_target = list(self.interpreter_cache.matches(target.compatibility))
      allowed_interpreters &= compatible_with_target

  if not allowed_interpreters:
    # Create a helpful error message.
    unique_compatibilities = set(tuple(t.compatibility) for t in targets_with_compatibilities)
    unique_compatibilities_strs = [','.join(x) for x in unique_compatibilities if x]
    targets_with_compatibilities_strs = [str(t) for t in targets_with_compatibilities]
    raise TaskError('Unable to detect a suitable interpreter for compatibilities: %s '
                    '(Conflicting targets: %s)' % (' && '.join(unique_compatibilities_strs),
                                                   ', '.join(targets_with_compatibilities_strs)))

The allowed_interpreters came back empty from the interpreter_cache and that code looks at configuration values whose keys have recently changed in order to bootstrap interpreters. I'm guessing you have a pants.ini config in your repo that is missing the key name changes.

There is a tool to check for stale pants.ini config keys in the pantsbuild repo. You can run it from within a clone of the pantsbuild/pants repo pointing at your own repo's pants.ini like so:

$ ./pants run src/python/pants/option/:migrate_config -- [path to your repo's pants.ini]

I did this recently when upgrading twitter/commons. The change was reviewed here and the tool output looked like so:

$ ./pants run src/python/pants/option/:migrate_config -- ~/dev-jsirois-commons/pants.ini
...
17:53:44 00:00   [run]
17:53:44 00:00     [py]
17:53:44 00:00       [chroot]
17:53:44 00:00       [run]
Checking config file at /home/jsirois/dev-jsirois-commons/pants.ini for unmigrated keys.
Found indices in section [python-repos]. Should be indexes in section [python-repos].

17:53:44 00:00     [jvm]
17:53:44 00:00     [cpp-run]
               SUCCESS

Note the Found indices in section [python-repos]. Should be indexes in section [python-repos]. You likely also have this same un-migrated inidices key tripping you up.

Answer 2

You can run with the environment variable PEX_VERBOSE=1 -- that will give you some more info about which version of Python you are running with. Your system Python might also be somehow misconfigured to use libraries from 2.6.

As a nuclear option, you could try https://github.com/wickman/python-bootstrap/blob/master/bootstrap_python.sh to see if that helps fix your python environment.