Making a python package for PIP with some data_files

Question

I'm doing a project with this layout:

project/
    bin/
        my_bin.py
    CHANGES.txt
    docs/
    LICENSE.txt
    README.txt
    MANIFEST.in
    setup.py
    project/
        __init__.py
        some_thing.py
        default_data.json
        other_datas/
            default/
                other_default_datas.json

And the problem is that when I install this using pip, it puts the "default_data.json" and the "other_datas" folder not in the same place as the rest of the app.

How am I supposed to do to make them be in the same place?

They end up on "/home/user/.virtualenvs/proj-env/project"

instead of "/home/user/.virtualenvs/proj-env/lib/python2.6/site-packages/project"

In the setup.py I'm doing it like this:

inside_dir = 'project'
data_folder= os.path.join(inside_dir,'other_datas')

data_files = [(inside_dir, [os.path.join(inside_dir,'default_data.json')])]
for dirpath, dirnames, filenames in os.walk(data_folder):
    data_files.append([dirpath, [os.path.join(dirpath, f) for f in filenames]])

Answer 1

From https://docs.python.org/3.4/distutils/setupscript.html#installing-additional-files:

If directory is a relative path, it is interpreted relative to the installation prefix (Python’s sys.prefix for pure-Python packages, sys.exec_prefix for packages that contain extension modules).

Each file name in files is interpreted relative to the setup.py script at the top of the package source distribution.

So the described behavior is simply how data_files work.

If you want to include the data files within your package you need to use package_data instead:

package_data={'project': ['default_data.json', 'other_datas/default/*.json']}

Answer 2

Change your MANIFEST.in to include those .json.

It is probably gonna work:

recursive-include project/ *.json

Answer 3

Take a look at this package https://pypi.python.org/pypi/datafolder. It makes it easy to install and use (data files: *.conf, *.ini *.db, ...) by your package and by the user.