GraphHopper.com | Forum | GitHub | Maps | Blog

Add reading OSM version while reading OSM-file


#1

This, in fact, is not a question, but a request to add new functionality.

Can we add to GraphHopper reading the nodes and ways version from the .osm or .pbf files by analogy with how the OSM unique ID is read?

As I understand, this will basically require the improvement of the graphhopper.reader.osm.OSMReader class…


Add to the graph building only the latest OSM version of the way
#2

This should be quite straight forward to implement. Have a look at this repo, instead of storing the osm id you would store the verion.

Best,
Robin


#3

The problem here is that the ReaderElement class does not contain the OSM version, so when reading a file the version of object (node, way, relation) is not defined and is not saved (in contrast to the OSM id)


#4

What is the use case?


#5

So, the problem is following. We have a local copy OSM data for several countries. Data uploads as pbf files are taken from here:
http://gis-lab.info/projects/osm_dump/
In order to reduce network traffic, daily updates to OSM data are not downloaded entirely (~3 Gb), but as a change file (~50 Mb), and are added to the pbf file.
As a result in the course of time, older versions are accumulated in the file for all objects, and when you try to “feed” such file to GraphHopper, the resulting graph turns out to be wrong: it contains many versions of the same lines.
Perhaps there are ready programs for pre-cleaning the pbf file from older versions of objects, but I couldn’t find them.


#6

Are you sure that your updates are done correctly? For example Geofabrik uses the same approach and their pbf files build fine. Anyway it is also recommended after some time to download a fresh version to reduce potential issues being dragged along over many versions, see here for more information.


#7

No, I’m not sure of anything. Most likely in the process of updating, there are errors (for example, I also noticed that in the pbf file some lines with one Osm id and one version are repeated).
The problem is that Graph Hopper does not track all these specific situations, although it would be possible to add the stage of analyzing the OSM file and correcting errors.


#8

This sounds erroneous, please make sure to improve your osm update method or use pbfs from official sources.

Yes, but GraphHopper does not store the osm id for example. So GraphHopper does not know how often it has seen the same id now. I think it’s ok to assume that pbfs that are read with GraphHopper are valid. If we find issues in the OSM data that could create routing errors we try to fix these, for example subnetworks.