Can't load pre-imported graph on different machine

Hey there!

I have a problem when using a pre-imported graph on a different machine.
I’m trying to have a graphhopper instance run via docker on a kubernetes cluster.
It works fine if I do the graph imported for smaller maps in the pod itself.
Since the nodes of the cluster aren’t super powerful and the RAM for needed routing itself (not importing) is lower, I used a more powerful machine to compute the graph for whole europe, transferred the graph files to the cluster and then tried restarting. I made sure the configuration is matching, it seems to me like everything is the same.
The files all got transferred correctly I think, at least they result in the same md5 checksums on both machines.

When the graph gets loaded though, I get an “IllegalStateException: Cannot load CHGraph|fastest|car”.

Am I missing something? I do have elevation activated, are there any more files I need to transfer? Is it even possible to transfer a computed graph at all?

Help would be greatly appreciated.

Is it even possible to transfer a computed graph at all?

Yes, this should be no problem.

Which graphhopper versions were/are you using on the different machines ? Usually the graph files are not necessarily compatible when imported with one version and used with another.

Both times the latest Docker image

Can you open the properties file inside the graphhopper folder with a text editor and post the content here ?
The value of graph.ch.weightings in this file has to match with your configuration file. The important settings there are: graph.flag_encoders, prepare.ch.weightings and prepare.ch.edge_based, please post these as well (or if you are unsure the whole config.yml file.

config.yml:
config_yml.txt (5.7 KB)
content of properties:
properties.txt (632 Bytes)

prepare.ch.turn_costs: edge_or_node

should be

prepare.ch.edge_based: edge_or_node

in your config.yml. This will enable speed mode with turn costs for both shortest and fastest weighting.
However, I do not understand where the exception comes from. Did you use this config on both machines (precomputation and cluster) ?

Its probably best to test your setup with some smaller map first, since edge based CH for whole Europe will take some time.

I did use the same on both machines, yes… So I need a reimport?

To make use of turn costs with speed mode/CH, yes you need a re-import, because the false setting I mentioned was ignored and speed mode was only prepared without turn costs.

Regarding the error this could also be a low-level data issue, @karussell (also note that graph.dataaccess: MMAP_STORE) ?

So @DUlbrich it would be best to try your setup with a very small map first and if you do not need speed mode with turn costs it also might be better to use the current stable release.

@easbar I was thinking about low-level data issues too, I just tried out to change MMAP_STORE to RAM_STORE and it actually began loading the map, however it ran out of memory…

So I changed the config as you suggested and reimported, this time using only the map of berlin, now I’m getting this error:

The properties stayed the same only that now
graph.ch.weightings=[]
and
graph.ch.edge.weightings=[fastest|car, shortest|car]

Ok, now it looks like the prepare.ch.edge_based: edge_or_node setting was accepted and CH was prepared for turn_costs. However, I can reproduce this error if I do not apply the setting change when starting the server. Have you double checked this ? Can you start the server without problem on the machine you are doing the import ?

On another note I am quite confused which graphhopper version is used for the Docker image. Judging from the error message this looks like a commit from a feature branch (?!), so this could also be a bug that existed at this specific commit.

@DUlbrich you could build a Docker image from the current master, I just checked and I do not get this error.

@karussell this could be because the docker release is built automatically and the turn_costs_ch branch was merged instead of squash-merged ?

Is there an easy way to see the git commit hash for a given docker build ?

@easbar What exactly do you mean by setting change?

Yes, the server starts without a problem on the machine I’m using for the import.

Sorry for being unclear, I meant the fix prepare.ch.turn_costs->prepare.ch.edge_based. You need to do this on both machines.

No problem!
Yes, I did the change on both machines.

Are you using the ‘latest’ docker image from here: https://hub.docker.com/r/graphhopper/graphhopper/tags ?
I wonder how you managed to have graph.ch.edge.weightings=[fastest|car, shortest|car] in your properties file (in the gh folder), because as far as I can see the latest image does not support this yet ?

Yes I am.
Not too sure about the versioning, didn’t dig too deep into the git history, since it worked before when doing everything on the same machine…

Ok, sorry I do not think I can help here. It is unclear to me what code runs inside this docker image. I opened a shell and looked inside the image and the code there does not match the error message we see here. Also I have no explanation (and cannot reproduce) how the graph.ch.edge.weightings property can be set with this version. Apart from this there seems to be an issue when transferring the files from one machine to the other.

Maybe you can try current master via docker-compose as explained here:https://github.com/graphhopper/graphhopper/tree/master#docker

@easbar No Problem, thanks a lot for your help!

As I said, I checked all the file’s MD5 Sums, they get transferred fully.

I will try out building from master, if that doesn’t work, got a recommendation how I could change my settings to make it run? I don’t really care for speed benefits too much right now, I just need a Routing Machine for europe running for testing purposes…

Hmm, I do not know why transferring the files does not work. Could it be some line ending /encoding issue on different machines ? The other problems we were seeing should go away if you are using master and stick to the example config .yml file. Sorry I have no explanation for the behavior we saw.

Hey there!
Sorry, for dropping silent, I had other matters to attend to.
Also my first proplem was obviously a versioning issue, since after using the master I can run and start the graphhopper server again with my precomputed graph.
However, now I’m unable to route on that graph.

I know always get the following error for points that lay within the bounding box:

"Cannot find CH RoutingAlgorithmFactory for weighting map {instructions=true, way_point_max_distance=1.0, calc_points=true, weighting=fastest, vehicle=car} in entries fastest|car|edge, shortest|car|edge, ",

"Cannot find CH RoutingAlgorithmFactory for weighting map {instructions=true, way_point_max_distance=1.0, calc_points=true, weighting=fastest, vehicle=car} in entries fastest|car|edge, shortest|car|edge, "

I attached txt versions of the config and the properties files once again.

config.txt (5.2 KB)
properties.txt (32.1 KB)