Filtering openstreetmap data for postgis

Question

I am creating a postgis database and want to use filtered OpenStreetMap data.

For this i have tried the following process:

1. Downloaded the planet.osm.bz2 file from https://planet.osm.org/
2. Unpacked to *.osm using bzip2
3. Filtered the file using osmfilter through the command prompt
4. Uploaded the filtered *.osm file to my database using osm2pgsql in command prompt

For my first attempt i have filtered for land area only.

However, in step 4 using osm2pgsql, i receive the following error in the command prompt: "Osm2pgsql failed due to ERROR: XML parsing error at line 3137102, column 61: not well-formed (invalid token) "

As shown from the command prompt on my windows computer:

Z:\OpenStreetMap>osm2pgsql -U postgres -W -m -d osm -p filteredland -S "C:\Progr
am Files (x86)\HOTOSM\share\default.style" filteredland2.osm
osm2pgsql version 0.92.0 (64 bit id space)

Password:
Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: filteredland_point
Setting up table: filteredland_line
Setting up table: filteredland_polygon
Setting up table: filteredland_roads
Allocating memory for sparse node cache
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=1
Mid: Ram, scale=100

Reading in file: filteredland2.osm
Using XML parser.
Processing: Node(1230k 61.5k/s) Way(0k 0.00k/s) Relation(0 0.00/s)node cache: st
ored: 1233078(100.00%), storage efficiency: 50.00% (dense blocks: 0, sparse node
s: 1233078), hit rate: -nan(ind)%
Osm2pgsql failed due to ERROR: XML parsing error at line 3137102, column 61: not
 well-formed (invalid token)

I have also attempted two alternate routes, which also failed:

1. Downloading the planet.pbf -> Converting to .o5m using osmconvert -> Filtering using osmfilter

2. Downloading the planet.pbf -> Converting to .osm using osmconvert -> Filtering using osmfilter(Gave warnings) -> Using osm2pgsql to transfer to database

Anyone know how to avoid this error or have experience with filtering the planet.osm file and uploading to postgis?

Answer 1

I suggest using Osmium instead of osmfilter, which doesn't require to convert the planet to a different format first and natively is able to return PBF data, which can be processed directly by osm2pgsql. It's faster, too.