CH failure with new OSM data (GH 0.5.3)

I’m running continous updates on an older GH version0.5.3 and with the latest OSM data it started to fail:

Caused by: java.lang.IllegalArgumentException: new capacity has to be strictly positive
at com.graphhopper.storage.RAMDataAccess.ensureCapacity(RAMDataAccess.java:99)
at com.graphhopper.storage.BaseGraph.ensureGeometry(BaseGraph.java:949)
at com.graphhopper.storage.BaseGraph.setWayGeometry_(BaseGraph.java:836)
at com.graphhopper.storage.BaseGraph.inPlaceNodeRemove(BaseGraph.java:757)
at com.graphhopper.storage.GraphHopperStorage.optimize(GraphHopperStorage.java:232)
at com.graphhopper.routing.util.PrepareRoutingSubnetworks.doWork(PrepareRoutingSubnetworks.java:92)

Any theories what could be the reason? Anything one can rule out? Given where it happens is it rather that total number of geometry bytes reached a limit (int size maybe?) Or does it rather look like a corrupt OSM data that caused it?

I fear like upgrading now to Graphhopper 0.11 is a long long way of code changes and testing for me so I’d rather try a quick-fix here before if possible, so we can keep the production system running with OSM updates.

Thanks!

Jan

This can be everything. So many bugs found and fixed since the last 3 years, even OSM data format changed since then etc. I recommend to upgrade to 0.10 as this is relative fresh version and should come with only a few refactorings. Moving to 0.11 is a bit harder, especially if you used our servlets, but with a proper decoupling this should be also no magic.

BTW: we never released 0.5.3 only 0.5.0 :smiley:

True, 0.5 it is!

I had a quick look and at least the FlagEncoder, EdgeIteratorState, Weighting, Osm[Way,Relation,Node] classes seemed to have changed or moved. Haven’t checked the rest of the low level API, yet.
Before I jump into trying to adjust that I would try to understand the issue better - to be sure an update would really fix it.

So the last successfully imported geometry file had a size of 8,584,691,812 - which is divided by 4 = 2,146,172,953 pretty close to the max int value. (I noticed there is some magic with the number 4 in the code).

Would be plausible that I hit a limit here!

–> Did you change the geometry storage between 0.5 and 0.10, e.g. do you remember there was a limitation in older versions that would be fixed in 0.10?

Yes, there was something. Please search the issues for geometry and 31 bits or similar.

Perfect, very helpful! https://github.com/graphhopper/graphhopper/issues/500 starts actually with you quoting me in 2015 - looks like I had the same issue back then.

I’ll try to reduce the way accuracy a bit, maybe that gives me a few months to do the migration to 0.10

Jan

1 Like