Cannot install kenlm package in anaconda environment

Question

When trying to install the python wrapper of kenlm from pip within an anaconda environment, I get the error:

(lm_1b) adamg:lm_1b adamg$ pip install https://github.com/kpu/kenlm/archive/master.zip
Collecting https://github.com/kpu/kenlm/archive/master.zip
  Cache entry deserialization failed, entry ignored
  Downloading https://github.com/kpu/kenlm/archive/master.zip
     - 4.4MB 51.1MB/s
Installing collected packages: kenlm
  Running setup.py install for kenlm ... error
    Complete output from command /Users/adamg/anaconda2/envs/lm_1b/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-aff_d2b8-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'kenlm' extension
    creating build
    creating build/temp.macosx-10.9-x86_64-3.6
    creating build/temp.macosx-10.9-x86_64-3.6/util
    creating build/temp.macosx-10.9-x86_64-3.6/lm
    creating build/temp.macosx-10.9-x86_64-3.6/util/double-conversion
    creating build/temp.macosx-10.9-x86_64-3.6/python
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/adamg/anaconda2/envs/lm_1b/include -arch x86_64 -I/Users/adamg/anaconda2/envs/lm_1b/include -arch x86_64 -I. -I/Users/adamg/anaconda2/envs/lm_1b/include/python3.6m -c util/pool.cc -o build/temp.macosx-10.9-x86_64-3.6/util/pool.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11
    In file included from util/pool.cc:1:
    In file included from ./util/pool.hh:4:
    /Users/adamg/anaconda2/envs/lm_1b/bin/../include/c++/v1/cassert:21:10: fatal error: 'assert.h' file not found
    #include <assert.h>
             ^~~~~~~~~~
    1 error generated.
    error: command 'clang' failed with exit status 1

    ----------------------------------------
Command "/Users/adamg/anaconda2/envs/lm_1b/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-aff_d2b8-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/

The pip command works outside a conda environment, but then kenlm is not active within the environment. I was also able to run this from an AWS ec2 instance running linux, so maybe it's a Mac OSX issue. Any idea how can this be solved?

Answer 1

On Mac usually there is a problem on clang with different libraries that are missing. In most of this cases install Xcode will solve the problem. Also, in case this does not, you can also run in terminal xcode-select --install
Hope this helps!