Hello, everyone.
I’d like to know if the current graph implementation supports multi-threaded work with it.
For example, if I want to perform Map Matching for different tracks from different threads on the same graph
Multiple reads should be allowed. For example the routing can handle a lot of routing requests in parallel. I would assume that Map Matching can do the same (probably a bit slower, since more complicated to calculate).
Best,
Robin
1 Like
We use Graphhopper (routing) in our production environment where multiple users (> 100 ?) are doing routing requests at the same time.
However, we notice that we get empty results of the routings in case of multiple use.
How many parallel routing requests can Graphhopper do ? And can you check or configure that somewhere ?
I can easily reproduce the problem.
Can you reproduce this with a pure GraphHopper? If yes, can you share details how to reproduce this.
I am not aware of any strict parallel processing limitations of the GraphHopper route calculation. At some point your server might be maxed out and responses will slow down, I haven’t seen empty responses.
If you are running a webserver in front of your GraphHopper instance, maybe that’s the reason for this behavior?
Yes, It is a pure GraphHopper engine (v0.13) that I access with PHP.
I uses curl to multi execute in php to run for example 150 calls in bulk with routings accross Europe.
When I ran the code in verbose mode, I sometimes get the Address already in use that gives me a empty result for that specified call.
- Closing connection 188 * Found bundle for host MYHOST: 0x1bb5c6aa740 [serially] * Hostname MYHOST was found in DNS cache * Trying MYIP… * TCP_NODELAY set * connect to MYIP port MYPORT failed: Address already in use * Failed to connect to MYHOST port MYPORT: Address already in use * Closing connection 189 * Connection timed out after 1156 milliseconds * stopped the pause stream! * Closing connection 0 * Connection timed out after 1156
$mh = curl_multi_init();
$chs = array();
foreach($urls as $index => $url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
//curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT_MS, 250);
$chs[$index] = $ch;
curl_multi_add_handle($mh, $ch);
}
$running = null;
do {
curl_multi_exec($mh, $running);
} while($running > 0);
foreach($chs as $id => $ch) {
$result = curl_multi_getcontent($ch);
//$result is sometimes emty
}
Are you sure this is not a Curl issue? When you search for Curl and “Address already in use”, there are quite some issues that mention Curl when opening a lot of connections. I have seen the recommendation to use the setting SO_REUSEADDR, I haven’t tested that though.
BTW: I used CURL for this a long time ago and had issues with CURL in PHP. You may want to have a look at Guzzle: https://github.com/guzzle/guzzle - I also moved away from PHP and Guzzle a long time ago, but that’s a different story 
I will take a look at your proposal and let you known.
Thanks for your reply.
I have tried all your ideas.
It seams not to be a curl issue. While setting SO_REUSEADDR the error Address already in use still occurs.
I also implemented guzzleHttp. Same problem there because guzzleHttp uses infact curl in the background.
I even have tried file_get_contents, it produces the same error.
So I tried multiple (2 to 10) instances that run ony one single curl / guzzleHttp or file_get_contents at the time.
Always gives the error somewhere between 30sec and 5min.
file get contents
$answer = file_get_contents($url);
curl
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT_MS, 250);
$answer = curl_exec($ch);
$curlInfo = curl_getinfo($ch);
curl_close($ch);
guzzleHttp
$client = new GuzzleHttp\Client();
$result = $client->request('GET', $url);
$statusCode = $result->getStatusCode();
$body = $result->getBody();
any ideas ?
It seams to work with the curl_option
curl_setopt($ch, CURLOPT_HTTPHEADER, array(‘Connection: close’));
Hi there, I noticed that parallel CH requests load only one processor core, is it possible to increase CH workers threads?
I guess prepare.ch.threads affect only to map importing.
You mean when sending requests to the GH server? Yes, prepare.ch.threads only affects the importing
Sorry, graphhopper works in multi-threaded mode there was a bug in my client code