Error in Loading Serialized Map Data with InMemMap.deserialize

26 Views Asked by At

I am attempting to load map data from a pickle file using map_connn = InMemMap.deserialize(map_conn). After exporting the map data from memory with map_con.dump() as a dictionary, an error occurs:

RTreeError: Error in "Index_CreateWithStream": Spatial Index Error: IllegalArgumentException: SpatialIndex::DiskStorageManager: Index/Data file cannot be created." 1.

map_con.dump()
import pickle
import os
map_data_file = 'E:\jupyternotebook\共享单车路径匹配\map_con\myosm.pkl'

if os.path.exists(map_data_file):
    with open(map_data_file, 'rb') as f:
        map_conn = pickle.load(f)
map_connn = InMemMap.deserialize(map_conn)

---------------------------------------------------------------------------
RTreeError                                Traceback (most recent call last)
Cell In[5], line 1
----> 1 map_connn = InMemMap.deserialize(map_conn)

File D:\miniconda3\envs\Map_Matching\lib\site-packages\leuvenmapmatching\map\inmem.py:144, in InMemMap.deserialize(cls, data)
    141 @classmethod
    142 def deserialize(cls, data):
    143     """Create a new instance from a dictionary."""
--> 144     nmap = cls(data["name"], dir=data.get("dir", None),
    145                use_latlon=data["use_latlon"], use_rtree=data["use_rtree"],
    146                index_edges=data["index_edges"],
    147                crs_lonlat=data.get("crs_lonlat", None), crs_xy=data.get("crs_xy", None),
    148                graph=data.get("graph", None), linked_edges=data.get("linked_edges", None),
    149                deserializing=True)
    150     return nmap

File D:\miniconda3\envs\Map_Matching\lib\site-packages\leuvenmapmatching\map\inmem.py:81, in InMemMap.__init__(self, name, use_latlon, use_rtree, index_edges, crs_lonlat, crs_xy, graph, linked_edges, dir, deserializing)
     79 self.use_rtree = use_rtree
     80 if self.use_rtree:
---> 81     self.setup_index(deserializing=deserializing)
     83 self.crs_lonlat = 'EPSG:4326' if crs_lonlat is None else crs_lonlat  # GPS
     84 self.crs_xy = 'EPSG:3395' if crs_xy is None else crs_xy  # Mercator projection

File D:\miniconda3\envs\Map_Matching\lib\site-packages\leuvenmapmatching\map\inmem.py:384, in InMemMap.setup_index(self, force, deserializing)
    382 else:
    383     logger.debug(f"Creating new in-memory rtree index (args={args}) ...")
--> 384 self.rtree = rtree.index.Index(*args)
    385 t_delta = time.time() - t_start
    386 logger.debug(f"... done: rtree size = {self.rtree_size()}, time = {t_delta} sec")

File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\index.py:273, in Index.__init__(self, *args, **kwargs)
    271 if stream and self.properties.type == RT_RTree:
    272     self._exception = None
--> 273     self.handle = self._create_idx_from_stream(stream)
    274     if self._exception:
    275         raise self._exception

File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\index.py:1263, in Index._create_idx_from_stream(self, stream)
   1260     return 0
   1262 stream = core.NEXTFUNC(py_next_item)
-> 1263 return IndexStreamHandle(self.properties.handle, stream)

File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\index.py:1396, in Handle.__init__(self, *args, **kwargs)
   1395 def __init__(self, *args: Any, **kwargs: Any) -> None:
-> 1396     self._ptr = self._create(*args, **kwargs)

File D:\miniconda3\envs\Map_Matching\lib\site-packages\rtree\core.py:25, in check_void(result, func, cargs)
     23     msg = f'Error in "{func.__name__}": {s}'
     24     rt.Error_Reset()
---> 25     raise RTreeError(msg)
     26 return result

RTreeError: Error in "Index_CreateWithStream": Spatial Index Error: IllegalArgumentException: SpatialIndex::DiskStorageManager: Index/Data file cannot be created.

Whenever I create a map in-memory object, it requires a significant amount of computation, including importing points, importing edges, and performing deduplication operations. This results in a long processing time each time I generate an in-memory map object. To overcome this, I attempted to export my pre-computed map in-memory object and load it directly from a file for future use.

Initially, I tried using Python's built-in pickle module. However, when I read the exported .pkl file, I found that it couldn't be used for path matching, as the matched paths were empty. I suspected that certain attributes, such as R-tree, might have been lost during the export of the map in-memory object.

To address this, I consulted the official documentation of LeuvenMap and discovered the dump() and deserialize() methods provided by the package. I attempted to use these recommended backup and loading methods. However, during the process, I encountered the aforementioned error.

I would greatly appreciate assistance in resolving this issue.

0

There are 0 best solutions below