Mapnik + TileStache + uWSGI / IOError: cannot identify image file

Question

I have a Vagrant environment with an ansible provisioning script, that brings up a fresh Ubuntu 16.04 server with Mapnik + Postgis + TileStache + uWSGI + Nginx for serving map tiles.

Half a year ago, everything worked fine. Now after bringing the Vagrant container up, the provisioning step works fine, all osm2pgsql imports worked and my TileStache bellows hello which indicates to me that nginx -> uwsgi -> tilestache is working.

I already tried to test if PIL/pillow is working. Simple scripts like the following are fully working:

from PIL import Image
import io
with open('test.png') as f:
   io = io.BytesIO(f.read())
im = Image.open(io)

My tilestache config:

{
    "cache": {
        "name": "Disk",
        "path": "./cache/",
          "umask": "0000"
    },
    "layers": {
        "osm_layer": {
            "provider": {
                "name": "proxy", 
                "url": "http://tile.openstreetmap.org/{Z}/{X}/{Y}.png"
            }
        }  
    }
}

But when I try to access a tile image like: http://localhost/osm_layer/0/0/0.png it doesn't work. Normally that should give me same tile back as http://tile.openstreetmap.org/0/0/0.png

I get following error in my logfiles:

Jun 26 09:07:16 gis uwsgi[31478]: Traceback (most recent call last):
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/__init__.py", line 379, in __call__
Jun 26 09:07:16 gis uwsgi[31478]:     status_code, headers, content = requestHandler2(self.config, path_info, query_string, script_name)
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/__init__.py", line 255, in requestHandler2
Jun 26 09:07:16 gis uwsgi[31478]:     status_code, headers, content = layer.getTileResponse(coord, extension)
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/Core.py", line 414, in getTileResponse
Jun 26 09:07:16 gis uwsgi[31478]:     tile = self.render(coord, format)
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/Core.py", line 500, in render
Jun 26 09:07:16 gis uwsgi[31478]:     tile = provider.renderTile(width, height, srs, coord)
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/Providers.py", line 250, in renderTile
Jun 26 09:07:16 gis uwsgi[31478]:     tile = Verbatim(body)
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/Providers.py", line 164, in __init__
Jun 26 09:07:16 gis uwsgi[31478]:     self.format = self.image().format
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/TileStache/Providers.py", line 170, in image
Jun 26 09:07:16 gis uwsgi[31478]:     self._image = Image.open(self.buffer)
Jun 26 09:07:16 gis uwsgi[31478]:   File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 2295, in open
Jun 26 09:07:16 gis uwsgi[31478]:     % (filename if filename else fp))
Jun 26 09:07:16 gis uwsgi[31478]: IOError: cannot identify image file <StringIO.StringIO instance at 0x7f513b7f5a28>
Jun 26 09:07:16 gis uwsgi[31478]: [pid: 31484|app: 0|req: 3/6] 192.168.20.1 () {42 vars in 681 bytes} [Wed Jun 26 09:07:16 2019] GET /osm_layer/0/0/0.png => generated 0 bytes in 138 msecs (HTTP/2.0 500) 0 headers in 0 bytes (0 switches on core 0)

It seems that there are problems with the corresponding packages, image files or with PIL/pillow packages, since for reproducing the error, it's suffficient to only proxy OSM tiles. No vector import or postgres is involved for now. These OSM tiles are proxied to http://tile.openstreetmap.org/{Z}/{X}/{Y}.png

Also I can see tilestache creating the cache directory structure but since there are IOError no actual tile/image is cached.

Answer 1

I found out, that this was no problem with Pillow. Instead, the current version of TileStache I'm using (tilestache==1.51.14) misses a user agent and OSM won't accept that. For completeness you should also pass a referrer (https://wiki.openstreetmap.org/wiki/DE:Tile_usage_policy)

See https://github.com/TileStache/TileStache/issues/360 for more information.

Workaround: Add User Agent to renderTile function in /usr/local/lib/python2.7/dist-packages/TileStache/Providers.py:

Add to line 262 of Providers.py (tilestache==1.51.14):

url_opener.addheaders = [('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3'), ('Referer', 'https://example.com/')]

I installed via pip instead of apt, so my tilestache files are at: /usr/local/lib/python2.7/dist-packages/TileStache/

From this directory you should recompile and restart tilestache afterwards:

$ sudo python -m compileall .
$ sudo service tilestache restart