GraphHopper.com | Forum | GitHub | Maps | Blog

Batch processing from command line


#1

Is there a doc/example for using GraphHopper for batch processing a large number of routes from command line, without web call overhead. Example use case: a large CSV file with start and end locations, for which time and distance estimates are required?


#2

There isn’t something like this but it should be easy to use a python client or even just bash via curl.


#3

You mean use python/bash to query via web API? This seems like a large overhead if you’d want to do millions of route calculation…


#4

Do you want to run it locally?

If so: You could just use the java implementation of graphhopper? Create Graphhopper graph folder (example in doc) and read in your csv and process all distance calculations.


#5

Yep, it would be completely local and offline. Could you point me to the doc on how to create graph folder?
This example https://github.com/graphhopper/graphhopper/blob/master/docs/core/routing.md still seems to be about creating the server etc.


#6

I meant that example yes.

We just use it without the “.forServer()” (look into the method is just sets some things for you, isn’t needed locally or at all. You don’t build a server. You can just create a Class with a constructor that’s doing:

GraphHopper hopper = new GraphHopperOSM().forServer();
hopper.setDataReaderFile(osmFile);
// where to store graphhopper files?
hopper.setGraphHopperLocation(graphFolder);
hopper.setEncodingManager(new EncodingManager("car"));

// now this can take minutes if it imports or a few seconds for loading
// of course this is dependent on the area you import
hopper.importOrLoad();

and a function distance which has 2 paramaeters for your call and returns for example:

rsp.getBest.getDistance()

This function includes:

GHRequest req = new GHRequest(latFrom, lonFrom, latTo, lonTo).
    setWeighting("fastest").
    setVehicle("car").
    setLocale(Locale.US);
GHResponse rsp = hopper.route(req);

// first check for errors
if(rsp.hasErrors()) {
   // handle them!
   // rsp.getErrors()
   return;
}

Just write a main for processing your csv and loop over all entrys and you will be fine. The graphfolder which is created just takes some time. (multiple hours if you need a big region, but it’s just a one time operation, loading an existing graph folder just takes seconds or minutes. It depends on your region size also)

The pbf file (called osmfile in the example) can you get from for example: https://download.geofabrik.de/ What regions do you need? Just a single country or more?

It all runs locally in your java environment. :slight_smile: With a graphfolder in your memory something like 100ms per request.

@karussell What’s the .forServer() method good for? Is it better to use it everytime you run it on a server? Or is it just to set something like memstore? I didn’t notice any downside not using it, so I’m just interested.


#7

Those are configurations per platform, like simplify response geometry or use in-memory vs memory-mapped graph. Though can be used wherever is need, e.g. mobile config on desktop too if low on memory.


#8

You could adapt the code for QueryTorture which does requests in parallel or adapt csv using this log format.


#9

Hi.

This is something I was looking for for ages. Have anyone already created the code? I so, would you share it? (I can’t code in Java.) Or what would be the overhead if the server was accessed from Python or R (which I can code in)?

And by the way, can GraphHopper find the shortest route (not only fastest route) too?

Many thanks and best wishes.
Michal