Hi, I want to create a Setup where I first create a the base graph, store it and then create new CHs step-by-step. (Parallelize in different threads, processes or even machines). I noticed that GraphHopperStorage requires me to define the future ch’s weightings before I import the base graph. This would make it impossible for me to add new CHs, later.
After reading the source code I noticed that it might not be necessary to have this restriction (might be wrong here, don’t understand all implications), however GraphHopperStorage is marked as final and the only way to instantiate the BaseGraph class which is package protected. Although access modifiers can easily be changed at runtime I wonder if there is a better way?
Doing multiple CH preparation via multiple threads is already possible. The use case of adding CH later is currently not covered although it should possible (at least from the design I had in mind). Still I’m unsure if that is required in your case, as I do not understand why you need multiple profiles but some of them are unknown.
But maybe you need to process multiple CH preparations on multiple machines and you want to avoid to redo the import multiple times? That is the only case where it could make sense, although something like the following could work
create the base graph
deploy this to multiple machines
do the CH preparation on every machine
copy the CH files back, probably modify the properties file “a bit” and load the base graph with multiple CH profiles
This should be already possible although quite hackish.
At base graph creation, I would have to define all future CHs and then I can load the graphs and add the shortcuts? Why is it necessary to define the CHs in advance?
It is not necessary, but it is simpler which was at least important for the initial implementation
And as you need to know how much memory you need and adding a new CH profile is resource intensive (RAM&CPU) I thought this assumption should hold for the most cases, especially RAM usage is required to know up front too. And for the same reason the normal user should not make this mistake but we can make it possible somehow for the advanced user if that is valueable.
Ok good to know. At least for me it would be helpful to to be able to create a GraphHopperStorage without requiring future Weightings to be declared at construction time and then just add more CHs if needed. I’ll hack around it for now.
GHDirectory directory = new GHDirectory(graphWorkDirectory.getAbsolutePath(), RAM_STORE);
GraphHopperStorage storage = new GraphHopperStorage(directory, encodingManager, true, new GraphExtension.NoOpExtension());
OSMReader reader = new OSMReader(storage)
.setOSMFile(osmFile)
.setEncodingManager(encodingManager)
.setWayPointMaxDistance(2);
reader.readGraph();
// optimize ...
storage.flush();
storage.close();
But how can I load it now into a new GraphHopperStorage so I can add a CH profile? I have to specify the CH profile in the constructor to be able to add it later but then it fails when calling #loadExisting():
java.lang.IllegalStateException: Configured graph.chWeightings: [MyNewCHWeighting] is not equal to loaded []
at com.graphhopper.storage.GraphHopperStorage.loadExisting(GraphHopperStorage.java:278)
Or without calling it it fails when adding the CHs:
CHGraph chGraph = storage.getGraph(CHGraph.class, weighting);
new PrepareContractionHierarchies(whatever, storage, chGraph, weighting.getFlagEncoder(), weighting, TraversalMode.NODE_BASED);
java.lang.IllegalStateException: Cannot find CHGraph for specified weighting [...]
What do I miss? How can I first create a base graph and later add a CH weighting?
GraphHopperStorage without requiring future Weightings to be declared at construction time
But you know all weightings in advance and you just want to do every CH preparation on a different machine?
What do I miss? How can I first create a base graph and later add a CH weighting?
Maybe it is not possible at the moment. I would init the GraphHopperStorage with the weightings up front, even if unused and then you should not get problems while loading them.
Or try adding a method BaseGraph.addWeighting which does the necessary thing chGraphs.add(new CHGraphImpl(w, dir, this.baseGraph)); … but not sure if this has other implications
For information, I have another use case in which such a feature would be useful.
I have a custom weighting, but before using it, i want to extend the weights through graph exploration (so i need a base graph), then I load the extended weights and do the CH.
So my use case would look like something like that:
Load base graph
Load weights
Extend weights by graph exploration using the base graph
Do CH using extended weights (and reusing the base graph)
This should already be possible. You could pass the weighting object(s) to the graph storage and e.g. inject/set the correct weights into the weighting object later on. E.g. import&load with correct weighting objects, traverse the graph, set the weights, do CH preparation
I am on GraphHopper version 10 and I am not sure how you accomplish this goal. I am using GraphHopperOSM class where I originally loaded an osm file with only “fastest” as the CH graph. Now I would like to re-use the GraphHopperOSM and just add “shortest” CH graph, how do you add it to the existing GraphHopperOSM? I attempted the following:
There are a few problems with this implementation. I get an exception thrown on the line “chGraph.create(100)” if the graph was already created, but if it isn’t created, I notice the correct files are created in the ghdir. Then when attempting to route on the chGraphs, I get a point out of bounds error. So I must be initializing something wrong.
No not really, but this will be added soon. The only thing that changed is that now we can add CHGraph to GraphHopperStorage after the constructor call but before the create call. But there is no real way of e.g. creating the storage, flushing to disk and at some later time adding more CH graphs.
