How to use map matching on Windows and how to get OSM node id?

I have a list of (lat,lng) coordinates. I want to map match them and get their corresponding OSM node ids.

I have followed the instructions from windows-set-up (and I replace the url with https://github.com/graphhopper/map-matching. But the command “./map-matching.sh action=import datasource=./some-dir/osm-file.pbf vehicle=car” from github-map-matching went wrong. It said " ‘.’ is not a command’. So now I just have a folder named map-matching and I don’t know what to do next. I intended to use java to get node ids.

Installed Git. Input “map-matching.sh action=import datasource=./some-dir/osm-file.pbf vehicle=car”. The Git window came out and said using “c:/user/java/jdk1.8”. But nothing happened. The window just closed in one sec.

And the convert issue. I looked at github-id-matching but it’s all about OSM-Way-ID<->GraphHopper Edge-Id, not node id.

I also looked at these discussions: how-to-get-the-osm-resource-reference-for-a-node-edge-in-the-match-result and how-to-retrieve-osm-node-ids-in-sequence-along-the-path-of-routing-result. But it seems there’s no solutions. I’m a little confused.

I’m new to java and all the stuff. I’ll appreciate it if you answer. Thanks a lot.

Currently you cannot return osm node IDs easily, this is a missing feature: https://github.com/graphhopper/graphhopper/issues/917

No idea regarding the windows setup. Someone else with such an OS can step in to help you :slight_smile:

Thank you. So it works more easily in Linux? Besides, do you know any other projects that can return OSM node id? Thanks a lot!

You said “cannot return osm node IDs easily”, but we can still get node IDs?

So it works more easily in Linux?

It should work on any OS as it is Java and most contributors have Linux or macOS but there are also windows users. Make sure you really have everything installed and followed the instructions closely.

but we can still get node IDs?

Yes, sure. Currently just the traversal key is returned (mainly “edge_id*2”) and you would need to change the response to e.g. return a list of edge.getBaseNode() or similar

do you mean that for each edge, there’s only one BaseNode. And the map-matching result is a list of edges, so I just need to get one BaseNode for each edge?

No, but if you have the sequence of nodes you know which edge it is. Probably safer to use GH its edge IDs until we implement the osm node ID response.

So evechainz, the system is internally creating own IDs (int) of the original OSM-Node Ids (long) while creating the Graph. . This is because of saving memory and to have a efficient data structure with own IDs counting from 0 to X not something like “234345643”
(if you work with OpenStreetMap the graph probably is created with the help of OSMReader (a DataReader) of the “graphhopper-reader-osm” packages. )

in https://github.com/karussell/graphhopper-osm-id-mapping
Karusell have showed how to make a match between the internal Edge IDs and the original Edge IDs in the proccess of reading the nodes and edges with the OSMReader class
I extended the Class “MyGraphhopper” from: https://github.com/karussell/graphhopper-osm-id-mapping
through:

Create own DataAccess-Object in the class:

private DataAccess nodeMapping;

and create the method for getting the original Node:

    public long getOSMNode(int internalNodeId) {
        long pointer = 8L * internalNodeId;
        return bitUtil.combineIntsToLong(nodeMapping.getInt(pointer), nodeMapping.getInt(pointer + 4L));
    }

and expand the DataReader within the class:

    @Override
    protected DataReader createReader(GraphHopperStorage ghStorage) {

        OSMReader reader = new OSMReader(ghStorage) {

            {
                edgeMapping.create(1000);

                // TODO this has to be added:
                nodeMapping.create(1000);

            }
..............

            @Override
            protected void storeOsmNodeID(int nodeId, long osmNodeId) {
                if (nodeId >= 0) {
                    super.storeOsmNodeID(nodeId, osmNodeId);

                    long pointer = 8L * nodeId;
                    nodeMapping.ensureCapacity(pointer + 8L);

                    nodeMapping.setInt(pointer, bitUtil.getIntLow(osmNodeId));
                    nodeMapping.setInt(pointer + 4, bitUtil.getIntHigh(osmNodeId));
               }
        }


            @Override
            protected void finishedReading() {
                super.finishedReading();

                edgeMapping.flush();

                // TODO add the flushing of the datastructure here:
                nodeMapping.flush();
            }

Now you also have to expand the OSMReader:

    int addTowerNode(long osmId, double lat, double lon, double ele) {
        if (nodeAccess.is3D())
            nodeAccess.setNode(nextTowerId, lat, lon, ele);
        else
            nodeAccess.setNode(nextTowerId, lat, lon);

        int id = -(nextTowerId + 3);
        getNodeMap().put(osmId, id);
        
        //TODO
        storeOsmNodeID(nextTowerId, osmId);

        nextTowerId++;
        

        return id;
    }

as well you need the storeOsmNodeID(…) which is overwriten/extended by the code we wrote before in “MyGraphhopper”…

    protected void storeOsmNodeID(int nodeId, long osmNodeId) {
        if(getNodeIdToOsmNodeIdMap().get(nodeId)!=0L && getNodeIdToOsmNodeIdMap().get(nodeId)!=osmNodeId)
            System.err.println(nodeId+": conflict:  old: "+getNodeIdToOsmNodeIdMap().get(nodeId)+"   new: "+osmNodeId);
        getNodeIdToOsmNodeIdMap().put(nodeId, osmNodeId);
    }

and you have to add to intruduce the data structure and it’s getter:

    protected IntLongMap nodeIdToOsmNodeIdMap;

    private IntLongMap getNodeIdToOsmNodeIdMap() {
        if (nodeIdToOsmNodeIdMap == null)
            nodeIdToOsmNodeIdMap = new GHIntLongHashMap((int) getNodeMap().getSize(), 0.5f);
        return nodeIdToOsmNodeIdMap;
    }

The reason Iam not sending a whole code or repository here, is that I have changed the core in several points. So I can not give you a working example. But this should be all snippets to make it work.
If it is interessting we can put it in the the common repository, maybe I can organize to push it.
(@karussell but you would need to help me ^^, I have no much experience so far with repos ).

So with this code you should be able to extend the project of karussell, and to use the MyGraphhopper in your project as Graphhopper-Instace

1 Like

Yes, really interesting. As the OSM node thing is something preferrable to OSM ways. (nodes are more stable and more precise, e.g. OSM ways can be very long)

Regarding github: you clone the repository which creates a ‘fork repository’ at github.com/Schmaddin/graphhopper), then you create a new branch locally e.g. ‘store_osm_nodeid’ and push this branch to your repository. Then you create a pull request. It is also described in more detail here: Creating a pull request - GitHub Docs

There are only a few requirements on the GraphHopper side: you sign the CLA (I can send this as PDF to your email if you like) and the whole thing works (and can be made optional via config) and there are a few unit tests.

I have applied this solution but it is throwing an Exception :slight_smile:

Exception in thread “main” java.lang.IllegalThreadStateException: already created
at com.graphhopper.storage.RAMDataAccess.create(RAMDataAccess.java:77)
at com.graphhopper.storage.RAMDataAccess.create(RAMDataAccess.java:33)
at com.graphhopper.reader.PillarInfo.(PillarInfo.java:41)
at com.graphhopper.reader.osm.OSMReader.(OSMReader.java:129)
at com.graphhopper.reader.osm.GraphHopperOSM.createReader(GraphHopperOSM.java:32)
at testmapmatching.MyGraphHopper.createReader(MyGraphHopper.java:105)
at com.graphhopper.GraphHopper.importData(GraphHopper.java:665)
at com.graphhopper.GraphHopper.process(GraphHopper.java:636)
at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:614)

Hi, did you finally get this work? I have similar requirement as yours. Thanks.

See my comment here and this code: Retrieve OSM node and way ID