Details (encoded_values) are missing when using Docker?

Harel · April 12, 2020, 8:26pm

Hi,
First and foremost thanks for this awesome library and endless giving!
I’m trying to get details on a route I’m requesting.
The following is the GET request:

http://localhost:8989/route?instructions=false&points_encoded=false&details=road_class&details=max_speed&elevation=true&point=31.971396577420734,35.24470233230383&point=31.926065209376176,35.00963707334776&vehicle=foot

The following is my config file:

github.com

IsraelHikingMap/Site/blob/master/gh-config.yml

graphhopper:

  # OpenStreetMap input file
  #datareader.file: ./israel-and-palestine-latest.osm.pbf

  ##### Vehicles #####


  # More options: foot,bike,bike2,mtb,racingbike,motorcycle (comma separated)
  # bike2 takes elevation data into account (like up-hill is slower than down-hill) and requires enabling graph.elevation.provider below.
  graph.flag_encoders: car4wd|block_fords=false|block_barriers=false,foot|block_fords=false,bike2|block_fords=false

  # Enable turn restrictions for car or motorcycle.
  # graph.flag_encoders: car|turn_costs=true
  graph.bytes_for_flags: 8
  # Add additional information to every edge. Used for path details.
  # If road_environment is added and elevation is enabled then also a tunnel and bridge interpolation is done, see #798.
  # More options are: surface,max_width,max_height,max_weight,max_axle_load,max_length,hazmat,hazmat_tunnel,hazmat_water,toll,track_type
  graph.encoded_values: road_class,road_class_link,road_environment,max_speed,road_access

This file has been truncated. show original

The following is the docker compose I’m using:

github.com

IsraelHikingMap/Site/blob/master/docker-compose.yml#L26


    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    volumes:
      - esdata:/usr/share/elasticsearch/data:rw
    ports:
      - 9200:9200
      - 9300:9300
    restart: always
  graphhopper:
    image: graphhopper/graphhopper
    container_name: graphhopper
    volumes:
      - ghdata:/data:rw
      - ./gh-config.yml:/data/gh-config.yml
    ports:
      - 8989:8989
    command: asia_israel-and-palestine.pbf -c /data/gh-config.yml -fd
    restart: always
volumes:

Seems like I’m defining the details (encoded_values) correctly yet the response I get is:

{"message":"You requested the details [road_class, max_speed] but we could only find []","hints":[{"message":"You requested the details [road_class, max_speed] but we could only find []","details":"java.lang.IllegalArgumentException"}]}

Am I missing something out? are details not supported for foot flag encoder?

Thanks ahead! any help will be appreciated.

karussell · April 13, 2020, 2:00pm

As you use the path details here there is likely some configuration problem. Once this is available you’ll get this information independent of the flag encoder.

Harel · April 13, 2020, 3:51pm

The configuration file is almost identical to the example-config.yml file (commented in some stuff).
What do you suggest I should do? The graph is building and working as expected and I get a route between any two points, I also get directions if I set the query to get it.
What should I try? remove flag encoders? remove encoded values? Which part of the configuration can cause this?

karussell · April 13, 2020, 4:29pm

Which GraphHopper version are you using?

Harel · April 13, 2020, 5:18pm

The one from dockerhub 0.12 I think it is.

karussell · April 13, 2020, 5:39pm

We do not publish GraphHopper to docker hub, only our docker containers. And as you said

The following is the docker compose I’m using:

you build your docker container on your own (if I understand this properly). So only you know which GraphHopper version you use. At start up it is also printed:

GraphHopperOSM - version 1.0|2020-03-30T09:22:02Z (5,15,4,3,5,5)

Make sure you use the latest stable 0.13 where the path details you tried where introduced:

github.com/graphhopper/graphhopper

Introduce RoadClass, RoadEnvironment, CarMaxSpeed, Surface, Toll and more EncodedValues

graphhopper:master ← graphhopper:shared_enc2

opened 11:34AM - 21 Feb 19 UTC

karussell

+2870 -1319

This PR shows how shared EncodedValues look like and how we populate them (via T…agParsers). The prefix "OSM" indicates that such parsers can look differently for a different data source. A TagParser has not necessarily a one-to-one relationship to an EncodedValue. With the new mechanism we can have multiple EncodedValues and just one TagParser like e.g. for height, width and weight. We likely need RoadClass, RoadEnvironment and a few more to be added by default. (As we could really need them for improving instructions, improving ETAs, finally fixing bugs like #713 for all vehicles, ...) ![image](https://user-images.githubusercontent.com/129644/53579021-18859500-3b79-11e9-9be7-63ee1dee1142.png) ![image](https://user-images.githubusercontent.com/129644/53954484-9fd18c00-40d6-11e9-9b43-2a4a3c3dc3c8.png) The following EncodedValues are available as path detail and could be used to influence the routing for all vehicle profiles (see [removed AvoidWeighting](https://github.com/graphhopper/graphhopper/pull/1548/commits/cb2c98301a2728c42f4eef096d3502808eb2535d)): * roundabout * road_class, road_class_link * road_environment * road_access * max_speed (earlier versions: car_max_speed) * surface * country * max_height, max_width, max_weight * toll Done: - [x] refactor DataFlagEncoder to use those new EncodedValues - [x] ~improve calculating instructions with e.g. roadclass~ -> separate issue - [x] refactor Tunnel&Bridge interpolator to use RoadEnvironment and enable it for all vehicles! - [x] The new path details classes show that as soon as there is an EncodedValue, we can show it along the path, e.g. replace AverageSpeedDetails (see #1551) - [x] added toll and car_max_speed EncodedValue and parser - [x] add a few of these EncodedValues per default - [x] introduced a simple AvoidWeighting that supports toll, motorway, ferry etc avoidance (or preference) - [x] remove AvoidWeighting (move to separate issue) - [x] implement a loading mechanism like we have for flag encoders - [x] make it possible to enable storing those new fields without the DataFlagEncoder (Should be possible with `graph.encoded_values=surface,road_class`!) - [x] new EncodedValueFactory and TagParserFactory to provide custom builds also loading from configuration - [x] new SpatialRuleParser to provide country-based decisions to other vehicle profiles - [x] It could be even good to introduce a dependency of TagParsers. E.g. the SpatialRuleParser requires the info from RoadAccessParser. We could enforce this via checking the EncodedValueLookup in createEncodedValue. Can be done with lookup.hasEncodedValue - [x] maxspeed returns 0 for default value (no sign) in Java API only, breaks backward compatibility => if fixed remove TODO NOW - [x] DataFlagEncoder requires e.g. road_environment - throw an exception if not added before. Breaks backward compatibility. - [x] remove all TODO NOW in src and test - [x] ~~should we introduce special rounding behaviour? E.g. for max_weight or max_height it can happen that the value increases due to rounding which might be a problem.~~ For now do not do this - [x] generic MappedStringEncodedValue -> different issue - [x] a real ObjectEncodedValue that uses e.g. MapDB under the hood and just stores the integer ID in the edge -> different issue - [x] move back to EnumEncodedValue and remove the IndexedBase stuff, see #1570, see [this branch and the resulting changes](https://github.com/graphhopper/graphhopper/compare/shared_enc2...shared_enc2_enum?expand=1) - [x] ~rename max_speed to speed_limit?~ no. let's just use max and average speed... - [x] default value is 0, but use a different value e.g. for max_speed (infinity) - [x] support "unlimited" on German autobahns: maybe we just use the maximum value. E.g. 5 bits and factor==5 => 5*(2⁵-1)=155. But then custom implementations couldn't easily change the precision of maxspeed as this changes the maximum value too. Maybe let's define 30*5=150 (can be precisely returned if precision is 5, 3, 2, 1). As the 140 limit exists several times and 160 is very rare (basically [2 ways world wide](http://overpass-turbo.eu/s/Hfw)) and everything above is a tagging mistake or something like [this](https://www.openstreetmap.org/way/38108321), [2](https://www.openstreetmap.org/way/40816550), [3](https://www.openstreetmap.org/way/227906554), [4](https://www.openstreetmap.org/way/39959598). [Valhalla uses 255 (8 bits)](https://github.com/valhalla/valhalla/blob/534b400a4cba4b1b4b643b69e2e3a549075dccfd/src/mjolnir/osmway.cc#L12). If we use 150 max speed, then the max average speed is `0.9*150=135` - [x] `other` should not be the first (default) access - [x] rename file back to map.geo.json - [x] improve error message of MappedDecimalEncodedValue; for now remove unused MappedDecimalEncodedValue? - [x] should int details also get returnMinus like decimals? removed this param for now. - [x] try returning `destination` access for tracks - [x] fix toll order handling (see comment) - [x] there should be an exception if only one direction is supported but setReverse is used. Also make the twoDirections variable accessible from outside. - [x] for JSON return -1 instead of the default "infinity" string value. Update: return `null` - [x] Forestry is not a highway (we would tag it via track and maybe add access=forestry) but add briddleway. later: - For the JSON API we should omit default entries if they are infinity and not return -1 - important follow up: #1269 and #1551

Harel · April 13, 2020, 5:51pm

I see, thanks for the clarification and quick response!
The version I’m using is: version 0.12|2019-01-31T16:58:02Z (5,14,4,3,3,2)
How can I make sure I’m using a stable version?
In any case, I’ll see what I can do in order to rebuild graphhopper image.

Harel · April 13, 2020, 5:58pm

I’m not sure I built the image locally. I’m using the following image as far as I understand which is a year old image:
https://hub.docker.com/r/graphhopper/graphhopper/
from docker image ls:
graphhopper/graphhopper latest 3c42f9a7011d 14 months ago 757MB

karussell · April 13, 2020, 7:13pm

Ah, I see. Looks like people build their docker image based on the current version they need.

Have created: https://github.com/graphhopper/graphhopper/issues/1985

Harel · April 13, 2020, 7:37pm

I see, I probably mixed the config example with the wrong version of graphhopper and so ended up with this issue - which is basically a compatibility issue as far as I understand.
It would be extremely helpful if you could push a new version to docker hub when creating a release in github. If there’s anything I can help with let me know.
Please, please, please, pretty please don’t choose the option to just add a deprecate comment instead. Even pre-release docker container are useful instead of rolling out my own…
Thanks again for all the hard work!

karussell · April 13, 2020, 8:01pm

Everything comes for the price of maintenance: Issues · graphhopper/graphhopper · GitHub

So it is rather unlikely that we include it again as our build process is already complicated and lengthy enough.

If there’s anything I can help with let me know.

Sure, you could set up a repository that fetches from our repo and publishes the docker container for every tag or commit and we could then point to this docker image as the official one.

Harel · April 13, 2020, 8:23pm

graphhopper/graphhopper is probably where people would look for a docker image, I don’t suppose you’ll give me the permission to update this image, right?
I don’t mind setting up a fork that will automatically gets updated and set some CI to upload it to dockerhub when a new tag is defined and see how this plays out, but I am concerned about the image name…
This is an interesting tool to keep a fork up-to-date:

So the only thing I need to implement is the CI which shouldn’t be too hard and make sure the dockerfile is up-to-date.
I’d love to help out. Been using graphhopper for a long time and I would really like to contribute back.

karussell · April 13, 2020, 8:34pm

From there we can easily link to your image (and remove the rest of the readme).

So the only thing I need to implement is the CI which shouldn’t be too hard and make sure the dockerfile is up-to-date.

You can use travis (have a look into our .travis.yml for an example) or github actions which is now also without costs.

Been using graphhopper for a long time and I would really like to contribute back.

Nice, great to hear that !

karussell · April 13, 2020, 8:41pm

I don’t suppose you’ll give me the permission to update this image, right?

Yeah, it is probably even better if we give you direct access to graphhopper/graphhopper at docker hub.

Harel · April 13, 2020, 8:51pm

I’m not sure if I understand you last comment - will it be possible to give me access to this dockerhub image settings so I can maintain it?
In any case, I should probably prove myself to be worthy of maintaining this dockerhub image first :-).
I’m comfortable with appveyor for CI which is what I’m currently using, but I can probably look at what Travis has to offer. I’ll look for a repo that builds docker images and see what are the required steps, shouldn’t be too hard as creating a docker image is pretty straight forward if you have a dockerfile, which graphhopper has :-).
Where would you like me to report my progress? Here or in the issue you have opened?

karussell · April 13, 2020, 9:33pm

Sorry, I was not clear enough. We will give you access to docker hub, will contact you via pm. The only thing we would require is that the code for building the image is done in an open source repo, so that everyone can see what is included and we are 100% transparent.

In any case, I should probably prove myself to be worthy of maintaining this dockerhub image first :-).

I trust you based on the previous contributions and once you pushed the image I can easily test and we improve from there.

I’m comfortable with appveyor for CI which is what I’m currently using

Sure, we also use appveyor to test our windows compatibility.

Where would you like me to report my progress? Here or in the issue you have opened?

We can keep discussion here and if code changes are required open a pull request. But as you are already in the contributors file there is probably not much we need

Harel · April 14, 2020, 9:27pm

Ok, seems like the docker build had finished successfully!
Can be seen here:
https://ci.appveyor.com/project/HarelM/graphhopper/builds/32176974
The repository I’ve used for my tests can be found here:

The only file I added was the appveyor-dockerhub-deploy.yml and used the appveyor settings to run it instead of the default one.
I haven’t tried to deploy it to dockerhub yet, It will probably be my next test.
It takes about 2 minutes to build the docker image and just a few build script commands (when using a linux environment and not windows as I think the default docker container settings for windows are windows containers, which do not work obviously…).
I’m not sure if I want to keep this as the official repository instead of using the more familiar IsraelHikingMap github account, but I already have a version there which is worth storing, I think, I’ll need to think about it and update where the “official” docker graphhopper github repository is planned to be, unless you are willing to reconsider pulling this in to your build script, which I have to say is pretty straight forward, and it shouldn’t complicate it much.
of course there are more tests to be made before I can call it a day…

Harel · April 14, 2020, 9:34pm

BTW, regarding the issue described in the begging of this thread, I was able to use another docker version while playing to make mine work: noamgal/graphhopper and it uses version 1.0…
I’m not able to see road_class and track_type but when I plan a route (Yay!!) using the foot flag encoder over a footway and a path (OSM highways tags) I’m unable to distinguish them - I get other for both track_type and road_class. Should I open an issue about it in Github or should I use another encoded_value? is this a know issue?

karussell · April 15, 2020, 12:30pm

over a footway and a path (OSM highways tags)

This was added only very recently: Extend RoadClass by otbutz · Pull Request #1907 · graphhopper/graphhopper · GitHub
I.e. it should work with master.

which I have to say is pretty straight forward, and it shouldn’t complicate it much

Ok. And if it fails could we assign the issue (to resolve it) to you ? The problem I see with this is really just the maintenance.

Harel · April 15, 2020, 5:18pm

Since I volunteered to create a repository to keep the docker image up to date, and I was planning on opening the issues there for docker issues I think it would only be fair to assign docker issues to me.
I don’t see a lot of maintenance once this is up and running, but I’m up for the challenge nevertheless :-).
Once I’ll finish all my tests and changes I’ll submit a pull request to resolve #1985 using either the appveyor or the travis CI files (Probably the appveyor as I’m more comfortable with it, later we might change it to travis since it should run on linux. BTW, appveyor can do both if you want to unify the build process to one file).