Flexible / Configurable Weighting Update

I invite you to try a planned feature for 1.0 which is now in a usable shape. It allows you to heavily customize the existing vehicle profiles without any Java knowledge. And not only per-request but also for the CH and LM preparations.

Cargo bike example:

base: bike

vehicle_height: 2.3
vehicle_width: 1.2

# let's assume cargo bikes are e-bikes and we should lower speed to prefer shorter and not faster routes automatically
max_speed:
  road_class:
    primary: 28

# if no other condition matches
max_speed_fallback: 25

priority:
  # exclude steps
  road_class:
    steps: 0

  surface:
    sand: 0.5
  # prefer better tracks
  track_type: { other: 1.1, grade1: 1.1 }
  # prefer official bike routes
  bike_network: { other: 0.5 }
  # avoid all situations where we have to get off the bike
  get_off_bike: 0.5

As you can see from the examples there are already several things implemented:

  • avoid, prefer or block roads with with a certain road_class (motorway, primary), surface, track_type, road_access (destination, …), road_environment (tunnel, bridge), toll, bike or hike network etc
  • define maximum vehicle properties like for speed, length, weight, height and width
  • lower the average speed via a speed_factor i.e. influence the time!
  • cap maximum speed (also influences the time!)
  • avoid, prefer or block areas via GeoJSON or also change speed or cap speed based on these areas.
  • avoid, prefer or block countries also works if issue 1875 is fixed (usage: add country encoded value and set border_directory)
  • change distance influence to prefer shorter but potentially slower routes

As there is no documentation except the PR itself and the tests I would appreciate your feedback about the usage and what is unclear or even your first custom profiles :slight_smile:.

Previously discussed here and here.

6 Likes

Have hacked together an interesting feature where we can avoiding certain turns like left_turns. Currently it still has some unexpected behaviour (for T-junctions) and is rather slow for longer routes, as the junction calculation stuff is done while the graph exploration, but it is really fun and interesting which routes are found:

link to animated gif

Update: this matter is a bit more tricky as there is sometimes no reason to avoid certain left turns, e.g. for a T-junction coming from bottom, going to left - this is a complete different situation compared with a normal 4-road junction where oncoming traffic needs to be “crossed”.

2 Likes

Cool :slight_smile: I tried it on my machine and got it up and running straight away. First I copied the cargo bike example into the flex box field in the left side pane. Then I tried this bike route (without customization)

it does not take the official bike route that we can see on open cycle map:

image

So I tried with this simplified custom bike:

base: bike

priority:
  bike_network: { other: 0.5 }

and the new route was then using the bike network as expected: :+1:

However, the new route is 5.8km instead of 3.4km long, so I tried playing with the distance_term_constant:

distance_term_constant: 0.10

And after trying a few different values I got the short route again. I am sure there are better examples where there are different ‘intermediate’ routes depending on the priority/distance trade-off, but in my quick example the calculated road was merely snapping back an forth between the two alternatives (depending on the parameters).

1 Like

That’s a very interesting feature.

How can I test this?

I’m running my own instance successfully, but I’m not seeing the flex input field on the left side.

routing.ch.disabling_allowed and routing.lm.disabling_allowed is set to true in config.yml

Tested with master and latest web service jar

I’ve opened a new topic for setting up flex mode: Setup for Flexible / Configurable Weighting Update

You probably need to recreate the js: https://github.com/graphhopper/graphhopper/blob/master/docs/core/quickstart-from-source.md#javascript

E.g. I’m using the following so that changes will be updated automatically:

cd graphhopper/web
$ BROWSERIFYSWAP_ENV='development' npm run watch

btw: you can test this without the UI too via sending the content via curl or similar.

This was now finally merged into master with additional documentation: https://github.com/graphhopper/graphhopper/pull/1841

The data format has slightly changed but is described in the docs.

Get Started with CustomWeighting

Have updated this small tutorial and published it as a blog post:

1 Like

This is a very cool feature! In light of your request on feedback about usage and examples I’ll briefly describe how I’m using it to provide routes for emergency vehicles in case it’s helpful for someone else to see:

I create a new FlagEncoder by copying the car flag encoder https://github.com/graphhopper/graphhopper/blob/master/core/src/main/java/com/graphhopper/routing/util/CarFlagEncoder.java and modifying the speeds like so

double emergencyCarSpeedIncrease = 1.5;
// autobahn
defaultSpeedMap.put("motorway", (int) emergencyCarSpeedIncrease * 100);
defaultSpeedMap.put("motorway_link", (int) emergencyCarSpeedIncrease * 70);
...

I reference this new encoder in DefalutFlagEncoderFactory.java and FlagEncoderFactory.java.

I then create a yaml profile with other specifics (vehicle size, etc.) and modify the config.yml like so:

line 11: graph.flag_encoders: car, emergencycar
line 19: graph.encoded_values: max_weight,max_height,max_width
line 43+:

- name: emergencycar
      vehicle: emergencycar
      weighting: custom
      custom_model_file: ./emergencycar.yml

I can now make requests to my server using the route-custom?profile=emergencycar endpoint!

2 Likes

Thanks, this is the feedback we need!

Indeed for increasing the speed values currently still editing the flag encoder is required. We could think about allowing this for the server-side profile that does not modify an existing profile.

I can now make requests to my server using the route-custom?profile=emergencycar endpoint!

If you do not need further customization you can also use the standard endpoint GET /route?profile=emergencycar or via POST /route.

BTW: the road access could be also something that is “tweakable” by the yaml file in a next version, e.g. to avoid excluding of one-ways and instead putting a slow speed or low priority on it. But we are currently not sure if this is wanted nor which problems we face when implementing this.

This is so excellent!!! I cannot wait!! Good work to you all!! I’m playing around with the pre-release now, its so flexible, and thank you for such straightforward blog documentation!

A few quick questions:

  1. Will version-1.0 allow you weight subtypes of highways?
    for example - a subtype of footways such as marked crosswalks which are tagged:
    • highway=footway
    • footway=marked <–can this tag be weighted?
  • or another example, sidewalks:
    • highway=footway
    • footway=sidewalk <–or can this tag be weighted?
  1. Is the bike_network in the documentation referring to the OpenCycleMap bicycle network? Can the Ways/Roads of that network be customized at all?

  2. I assume the Geojson will have a max file size, or number of feature limitation listed in the documentation? (the geojson feature might be my best work-around, if I can’t specifically “prefer” those subtypes of highways in Question #1)

Great work!! I’ll be eagerly awaiting the release and full documentation! =]

This is not yet possible. Still you could create a new parser (in Java) for the footway tag that would allow this. Support for sidewalks would be nice to have in core for sure.

Is the bike_network in the documentation referring to the OpenCycleMap bicycle network?

It refers to official bike routes that are easily visible with OpenCycleMap, but they are just tags in OSM data.

Can the Ways/Roads of that network be customized at all?

You can customize the route but of course not the ways itself.

I assume the Geojson will have a max file size, or number of feature limitation listed in the documentation?

There are no such limitations yet, but everyone who sets up GraphHopper will set a limit likely with a reverse proxy or dropwizard itself.

(the geojson feature might be my best work-around, if I can’t specifically “prefer” those subtypes of highways in Question #1)

What do you mean here?

Great work!! I’ll be eagerly awaiting the release and full documentation! =]

It is already “full” documentation. If something is unclear do not hesitate to ask and we’ll improve.

Still you could create a new parser (in Java) for the footway tag that would allow this.

Which .java file do I need to modify to add a footway parser? I assume it’s one of these? I’m sorry, I’m trying to find it myself, I don’t come from a java background!

(the geojson feature might be my best work-around, if I can’t specifically “prefer” those subtypes of highways in Question #1)

Sorry that was unclear - I meant to say if I cannot figure how to code a new footway parser (in Java), then I was going to use the my_area geojson (discussed in the documentation), to create weighted polygons around the footway=marked crosswalks of interest, but it sounds like I can just put a footway parser into Java!

Yes, see e.g. the OSMRoadClassParser for something similar.

Maybe this can be done even without doing any changes in Java code: For example by default the possible values for RoadClass are:

public enum RoadClass {
    OTHER("other"), MOTORWAY("motorway"),
    TRUNK("trunk"), PRIMARY("primary"), SECONDARY("secondary"),
    TERTIARY("tertiary"), RESIDENTIAL("residential"), UNCLASSIFIED("unclassified"),
    SERVICE("service"), ROAD("road"), TRACK("track"),
    BRIDLEWAY("bridleway"), STEPS("steps"), CYCLEWAY("cycleway"),
    PATH("path"), LIVING_STREET("living_street"), FOOTWAY("footway"),
    PEDESTRIAN("pedestrian"), PLATFORM("platform"), CORRIDOR("corridor");

A cheap and somewhat hacky way would be first checking if your OSM data even uses all these tags. If for example it does not use ‘platform’ at all you could simply tag all your cross-walks with highway=platform and then in your custom profile you can setup some priority/speed for road_class: {"platform": 0.6} or something. Of course this approach fails if there are roads that are actually tagged as platform and that you do not want to be treated like crosswalks. A slightly cleaner way would be modifying RoadClass.java and simply add CROSSWALK("crosswalk") and tag all your crosswalks with highway=crosswalk (I mean tagging on your local file, you should not push these changes to openstreetmap.org unless this is something that is appreciated by the OSM community). Do not forget to re-build GraphHopper after modifying RoadClass.java though.

What I am trying to say is that if you do custom modifications of your OSM file it might be easier to use some of the existing tags (like highway) than writing another parser for another tag. That being said like @karussell pointed out adding a parser for the footway tag would be a nice addition for GraphHopper.

2 Likes

A cheap and somewhat hacky way would be first checking if your OSM data even uses all these tags. If for example it does not use ‘platform’ at all you could simply tag all your cross-walks with highway=platform and then in your custom profile you can setup some priority/speed for road_class: {"platform": 0.6} or something. Of course this approach fails if there are roads that are actually tagged as platform and that you do not want to be treated like crosswalks. A slightly cleaner way would be modifying RoadClass.java and simply add CROSSWALK("crosswalk") and tag all your crosswalks with highway=crosswalk

  • THAT’S IT!!! That’s a perfect work around!!!

you should not push these changes to openstreetmap.org

  • Yes, totally agree, all these modifications will only happen locally.

it might be easier to use some of the existing tags (like highway) than writing another parser for another tag. That being said like @karussell pointed out adding a parser for the footway tag would be a nice addition for GraphHopper.

  • I agree with that too! I would love to learn how to write the parser myself (and might still try in the coming weeks), but that would be cool if your team eventually developed that into version-1.0. =]

I can’t seem to get the custom profiles to take effect! I’m experimenting with a “roadbike” profile by changing the priority for different surface types. It works OK when I do test the profile on your demo server (it successfully avoids routing on a path with an unknown surface type) but not on our own set up. These are the relevant bits from the config file:

graph.flag_encoders: bike|block_fords=false

graph.encoded_values: surface

profiles:
- name: roadbike
vehicle: bike
weighting: shortest

profiles_ch:
- profile: bike
- profile: roadbike

I generate the graph then make the requests as follows:

route?profile=roadbike&point=
51.260894,-0.430938&point=51.261874,-0.431773

I’ve also tried it without a roadbike CH and added &ch.disable=true to the request, but that just came up with the same result.

Any idea what I’m missing? I’m sure it’s something simple I’m doing wrong!

Sorry, I missed some lines off on my previous post, the profiles section is as follows:

profiles:
- name: bike
vehicle: bike
weighting: shortest
- name: roadbike
vehicle: bike
weighting: shortest
custom_model_file: roadbike.yml

The roadbike.yml is as follows:

priority:
surface: {
paving_stones: 0.0,
cobblestone: 0.0,
unpaved: 0.0,
gravel: 0.0,
fine_gravel: 0.0,
dirt: 0.0,
grass: 0.0,
sand: 0.0,
ground: 0.0,
other: 0.0
}

The usage of custom_model_file and weighting==shortest can’t work and should throw an exception. You have to use weighting: custom. Can you post your full config.yml?

Thanks, I’ll try that.

This is the full config file:

graphhopper:

  # OpenStreetMap input file PBF or XML, can be changed via command line -Ddw.graphhopper.datareader.file=some.pbf
  datareader.file: ""
  # Local folder used by graphhopper to store its data
  # graph.location: graph-cache

  ##### Vehicles #####


  # More options: foot,hike,bike,bike2,mtb,racingbike,motorcycle,car4wd,wheelchair (comma separated)
  # bike2 takes elevation data into account (like up-hill is slower than down-hill) and requires enabling graph.elevation.provider below.
  graph.flag_encoders: bike|block_fords=false

  # Enable turn restrictions for car or motorcycle.
  # graph.flag_encoders: car|turn_costs=true

  # Add additional information to every edge. Used for path details (#1548), better instructions (#1844) and tunnel/bridge interpolation (#798).
  # Default values are: road_class,road_class_link,road_environment,max_speed,road_access (since #1805)
  # More are: surface,max_width,max_height,max_weight,max_axle_load,max_length,hazmat,hazmat_tunnel,hazmat_water,toll,track_type
  # graph.encoded_values: surface,toll,track_type
  graph.encoded_values: surface

  ##### Routing Profiles ####

  # Routing can be done for the following list of profiles. Note that it is required to specify all the profiles you
  # would like to use here. The fields of each profile are as follows:
  # - name (required): a unique string identifier for the profile
  # - vehicle (required): refers to the `graph.flag_encoders` used for this profile
  # - weighting (required): the weighting used for this profile, e.g. fastest,shortest or short_fastest
  # - turn_costs (true/false, default: false): whether or not turn restrictions should be applied for this profile.
  #   this will only work if the `graph.flag_encoders` for the given `vehicle` is configured with `|turn_costs=true`.
  #
  # Depending on the above fields there are other properties that can be used, e.g.
  # - distance_factor: 0.1 (can be used to fine tune the time/distance trade-off of short_fastest weighting)
  # - u_turn_costs: 60 (time-penalty for doing a u-turn in seconds (only possible when `turn_costs: true`)).
  #   Note that since the u-turn costs are given in seconds the weighting you use should also calculate the weight
  #   in seconds, so for example it does not work with shortest weighting.
  # - custom_model_file: when you specified "weighting: custom" you need to set a yaml file that defines the custom_model.
  #   If you want an empty model you can also set "custom_model_file: empty".
  #
  #   For more information about profiles and especially custom profiles have a look into the documentation
  #   at docs/core/profiles.md or the examples under web/src/test/resources/com/graphhopper/http/resources/ or
  #   the CustomWeighting class for the raw details.
  #
  # To prevent long running routing queries you should usually enable either speed or hybrid mode for all the given
  # profiles (see below). Otherwise you should at least limit the number of `routing.max_visited_nodes`.
  profiles:
      vehicle: bike
      weighting: shortest
    - name: roadbike
      vehicle: bike
      weighting: shortest
      custom_model_file: roadbike.yml
    - name: offroadbike
      vehicle: bike
      weighting: shortest
      custom_model_file: offroadbike.yml

  #  - name: car_with_turn_costs
  #    vehicle: car
  #    weighting: short_fastest
  #    distance_factor: 0.1
  #    turn_costs: true
  #    u_turn_costs: 60

  # Speed mode:
  # Its possible to speed up routing by doing a special graph preparation (Contraction Hierarchies, CH). This requires
  # more RAM/disk space for holding the prepared graph but also means less memory usage per request. Using the following
  # list you can define for which of the above routing profiles such preparation shall be performed. Note that to support
  # profiles with `turn_costs: true` a more elaborate preparation is required (longer preparation time and more memory
  # usage) and the routing will also be slower than with `turn_costs: false`.
  profiles_ch:
    - profile: bike
    - profile: roadbike
    - profile: offroadbike
  #   - profile: car_with_turn_costs

  # Hybrid mode:
  # Similar to speed mode, the hybrid mode (Landmarks, LM) also speeds up routing by doing calculating auxiliary data
  # in advance. Its not as fast as speed mode, but more flexible.
  #
  # Advanced usage: It is possible to use the same preparation for multiple profiles which saves memory and preparation
  # time. To do this use e.g. `preparation_profile: my_other_profile` where `my_other_profile` is the name of another
  # profile for which an LM profile exists. Important: This only will give correct routing results if the weights
  # calculated for the profile are equal or larger (for every edge) than those calculated for the profile that was used
  # for the preparation (`my_other_profile`)
  profiles_lm: []

  ##### Elevation #####


  # To populate your graph with elevation data use SRTM, default is noop (no elevation). Read more about it in docs/core/elevation.md
  # graph.elevation.provider: srtm


  # default location for cache is /tmp/srtm
  # graph.elevation.cache_dir: ./srtmprovider/


  # If you have a slow disk or plenty of RAM change the default MMAP to:
  # graph.elevation.dataaccess: RAM_STORE



  #### Speed, hybrid and flexible mode ####


  # To make CH preparation faster for multiple profiles you can increase the default threads if you have enough RAM.
  # Change this setting only if you know what you are doing and if the default worked for you.
  # prepare.ch.threads: 1

  # To tune the performance vs. memory usage for the hybrid mode use
  # prepare.lm.landmarks: 16

  # Make landmark preparation parallel if you have enough RAM. Change this only if you know what you are doing and if
  # the default worked for you.
  # prepare.lm.threads: 1

  # In many cases the road network consists of independent components without any routes going in between. In
  # the most simple case you can imagine an island without a bridge or ferry connection. The following parameter
  # allows setting a minimum size (number of nodes) for such detached components. This can be used to reduce the number
  # of cases where a connection between locations might not be found.
  prepare.min_network_size: 200


  ##### Routing #####


  # You can define the maximum visited nodes when routing. This may result in not found connections if there is no
  # connection between two points within the given visited nodes. The default is Integer.MAX_VALUE. Useful for flexibility mode
  routing.max_visited_nodes: 1000000


  # If enabled, allows a user to run flexibility requests even if speed mode is enabled. Every request then has to include a hint ch.disable=true.
  # Attention, non-CH route calculations take way more time and resources, compared to CH routing.
  # A possible attacker might exploit this to slow down your service. Only enable it if you need it and with routing.maxVisitedNodes
  # routing.ch.disabling_allowed: true


  # If enabled, allows a user to run flexible mode requests even if the hybrid mode is enabled. Every such request then has to include a hint routing.lm.disable=true.
  # routing.lm.disabling_allowed: true

  # Control how many active landmarks are picked per default, this can improve query performance
  # routing.lm.active_landmarks: 4


  # You can limit the max distance between two consecutive waypoints of flexible routing requests to be less or equal
  # the given distance in meter. Default is set to 1000km.
  routing.non_ch.max_waypoint_distance: 1000000


  ##### Storage #####


  # configure the memory access, use RAM_STORE for well equipped servers (default and recommended)
  graph.dataaccess: RAM_STORE


  # will write way names in the preferred language (language code as defined in ISO 639-1 or ISO 639-2):
  # datareader.preferred_language: en


  # Sort the graph after import to make requests roughly ~10% faster. Note that this requires significantly more RAM on import.
  # graph.do_sort: true



  ##### Spatial Rules #####
  # Spatial Rules require some configuration and only work with the DataFlagEncoder.


  # Spatial Rules require you to provide Polygons in which the rules are enforced
  # The line below contains the default location for the files which define these borders
  # spatial_rules.borders_directory: core/files/spatialrules

  # You can define the maximum BBox for which spatial rules are loaded.
  # You might want to do this if you are only importing a small area and don't need rules for other countries.
  # Having less rules, might result in a smaller graph. The line below contains the world-wide bounding box, uncomment and adapt to your need.
  # spatial_rules.max_bbox: -180,180,-90,90


# Uncomment the following to point /maps to the source directory in the filesystem instead of
# the Java resource path. Helpful for development of the web client.
# Assumes that the web module is the working directory.
#
# assets:
#  overrides:
#    /maps: web/target/classes/assets/

# Dropwizard server configuration
server:
  application_connectors:
  - type: http
    port: 8989
    # for security reasons bind to localhost
    bind_host: localhost
  request_log:
      appenders: []
  admin_connectors:
  - type: http
    port: 8990
    bind_host: localhost
# See https://www.dropwizard.io/1.3.8/docs/manual/configuration.html#logging
logging:
  appenders:
  - type: file
    time_zone: UTC
    current_log_filename: logs/graphhopper.log
    log_format: "%d{YYYY-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
    archive: true
    archived_log_filename_pattern: ./logs/graphhopper-%d.log.gz
    archived_file_count: 30
    never_block: true
  - type: console
    time_zone: UTC
    log_format: "%d{YYYY-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"

Sorry for the formatting - the hash characters seem to make the text go large and bold!