Memory errors and requirements

Open Source Routing Engine
1

I am trying to understand the memory requirements for processing the planet and have run into two issues. I have a dedicated server with 192GB of RAM.

java -D"dw.graphhopper.datareader.file=planet-latest.osm.pbf" -jar graphhopper*.jar import config.yml

When setting my JAVA_OPTS to 150GB I am still getting a memory heap error. So I switched to using MMAP and now am getting this error:

java.lang.InternalError: a fault occurred in an unsafe memory access operation
        at com.graphhopper.reader.osm.OSMNodeData.addPillarNode(OSMNodeData.java:181)
        at com.graphhopper.reader.osm.OSMNodeData.addCoordinatesIfMapped(OSMNodeData.java:160)
        at com.graphhopper.reader.osm.WaySegmentParser$Pass2Handler.handleNode(WaySegmentParser.java:209)
        at com.graphhopper.reader.osm.WaySegmentParser$ReaderElementHandler.handleElement(WaySegmentParser.java:498)
        at com.graphhopper.reader.osm.WaySegmentParser.readOSM(WaySegmentParser.java:401)
        at com.graphhopper.reader.osm.WaySegmentParser.readOSM(WaySegmentParser.java:107)
        at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:177)
        at com.graphhopper.GraphHopper.importOSM(GraphHopper.java:922)
        at com.graphhopper.GraphHopper.process(GraphHopper.java:812)
        at com.graphhopper.GraphHopper.importAndClose(GraphHopper.java:776)
        at com.graphhopper.application.cli.ImportCommand.run(ImportCommand.java:36)
        at com.graphhopper.application.cli.ImportCommand.run(ImportCommand.java:27)
        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
        at io.dropwizard.cli.Cli.run(Cli.java:78)
        at io.dropwizard.Application.run(Application.java:94)
        at com.graphhopper.application.GraphHopperApplication.main(GraphHopperApplication.java:38)

It seems like I have plenty of memory, so I would love some guidance on if I need to change some settings?

2

Did you read the following guide? Host Your Own Worldwide Route Calculator With GraphHopper - GraphHopper Directions API

E.g. MMAP requires you to use lower Xmx settings to have off-heap memory available.

3

Thanks, I was able to complete the process after using your suggestion.

1 Like