I am using the gh 0.10 branch and I am not able to build my vehicle graph with world.pbf [smaler areas (like DACH) are fine]
java.lang.ArrayIndexOutOfBoundsException: -1262 at com.graphhopper.storage.RAMIntDataAccess.setInt(RAMIntDataAccess.java:200) at com.graphhopper.storage.BaseGraph.initNodeRefs(BaseGraph.java:269) at com.graphhopper.storage.BaseGraph.inPlaceNodeRemove(BaseGraph.java:725) at com.graphhopper.storage.GraphHopperStorage.optimize(GraphHopperStorage.java:242) at com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks.doWork(PrepareRoutingSubnetworks.java:99) at com.graphhopper.GraphHopper.cleanUp(GraphHopper.java:1286)
I have running multiple different tests & setups in order to try to find the root cause of this issue - Running original gh 0.10 branch with stock ‘car’ FlagEncoder is finish graph creation without any issues.
So in general this issue seams to be ‘self made’ - I have a written a custom Car FlagEncoder - in order to support additional features & functionality within our ors project.
I have added additional logging information inside PrepareRoutingSubnetworks & BaseGraph in order to find some evidence what could be the root cause of my issue.
The PrepareRoutingSubnetworks.doWork() generates the following output:
start finding subnetworks (min:200, min one way:0) totalMB:112640, usedMB:70303 1510204 subnetworks found for car-ors, totalMB:112640, usedMB:75761 optimize to remove subnetworks (1510204), unvisited-dead-end-nodes (0), maxEdges/node (31)
in the GraphHopperStorage.optimize() I have added additional logging in order to get some numbers
getNodes(): 153648072 baseGraph.getCapacity(): 17364418560 ((BitSet) removeNodesObject).length(): 153648072 removeNodesObject.getCardinality() / delNodes: 5015126
so then in the optimize() the inPlaceNodeRemove(delNodes) of the BaseGraph will be called - there I have added additional logging as well:
I was interested in the 'itemsToMove: object:
itemsToMove: 5015126 removeNodes.length: 153648072 getCardinality: 5015126
also I logged the ‘toMoveSet’:
toMoveSet.length: 153646465 getCardinality: 3968681
and then finally I logged the ‘nodeCount’ & ‘removeNodeCount’ shortly before the initNodeRefs(…) method will be called:
nodeCount: 153648072 / removeNodeCount: 5015126 / nodeEntryBytes: 20
then the final mthod in the stack trace will be called initNodeRefs(long oldCapacity, long newCapacity) where oldCapacity = (nodeCount-removeNodeCount) * nodeEntryBytes.
and looking at the numbers you might notice, that (153648072-5015126)*20 will give us a 2.972.658.920…
then in the initNodeRefs() itself a pointer will be generated - staring from oldCapacity + N_EDGE_REF
I have also included additional logging info in the initNodeRefs() - printing the loop count and the value of the pointer - and already in the first time the loop will be entered:
loopRun pointer: -1322308376
so oldCapacity (in my run ‘2972658920’) + N_EDGE_REF will result in -1322308376 - which looks like an overflow - right?
My question for now is: -> why I am not running into a similar problem with stock gh code - or with other words - what is the graph size for car FlagEncoder?
Are my numbers (nodeCount: 153648072) extremely high? And if ‘yes’ would it possible for you to give me hint’s where/what to check in my FlagEncoder implementation - or more simpler question - can be my FlagEncoder be the root of this problem - or are there other/additional this that have an influence on the final nodeCount ?!