Hi!
I’m trying to add new FlagEncoder and custom profiles.
Here is my code:
Flagencoder
@Component
public class SmallTruckEncoder extends CarFlagEncoder {
public SmallTruckEncoder() {
this(new PMap());
}
public SmallTruckEncoder(int speedBits, double speedFactor, int maxTurnCosts) {
this(new PMap().putObject("speed_bits", speedBits).putObject("speed_factor", speedFactor).
putObject("max_turn_costs", maxTurnCosts));
}
public SmallTruckEncoder(PMap properties) {
super(properties.getInt("speed_bits", 5),
properties.getDouble("speed_factor", 5),
properties.getInt("max_turn_costs", properties.getBool("turn_costs", false) ? 1 : 0));
intendedValues.add("delivery");
restrictedValues.remove("delivery");
maxPossibleSpeed = 110;
}
@Override
public int getVersion() {
return 1;
}
@Override
public TransportationMode getTransportationMode() {
return TransportationMode.HGV;
}
@Override
public String toString() {
return VehicleType.SMALL_TRUCK.getValue();
}
}
config.yml
graph.flag_encoders: small_truck|turn_costs=true
graph.dataaccess: RAM_STORE
graph.encoded_values: max_weight,max_height,max_width
profiles:
- name: small_truck_short_fastest
vehicle: small_truck
turn_costs: true
weighting: custom
custom_model_file: ./small_truck.yml
small_truck.yml
priority:
- if: max_weight < 3.5
multiply by: 0
I’m getting exception
Caused by: java.lang.IllegalArgumentException: Could not create weighting for profile: ‘small_truck_short_fastest’.
Profile: name=small_truck_short_fastest|vehicle=car|weighting=custom|turnCosts=true|hints={}
Error: custom weighting requires a CustomProfile but was profile=small_truck_short_fastest
How I can fix it?
I used informations from this sources:
graphhopper:master
← graphhopper:issue_1776
opened 06:19PM - 30 Dec 19 UTC
fixes #1776
fixes #223
## Note
**Disclaimer: custom profiles were an al… pha feature in version 1.0 and 2.0.**
You can try this feature in [the linked small tutorial](https://www.graphhopper.com/blog/2020/05/06/get-started-with-customizable-routing/). Please note that the format changed in version 3.0 - see [here](https://github.com/graphhopper/graphhopper/pull/2209) for the pull request and [the blog post with examples](https://www.graphhopper.com/blog/2020/05/31/examples-for-customizable-routing/) which is always updated.
### Usage
First you need to define a custom profile:
```
profiles:
- name: car
vehicle: car
weighting: custom
```
Then you can use this profile in a custom request. You query a json towards the new endpoint `/route-custom`. In the following section we use yaml to allow comments (you can convert it with this [node js tool](https://github.com/karussell/snake_case/blob/master/yaml2json.js) or [this online tool](https://www.json2yaml.com/convert-yaml-to-json)):
```yml
# query parameters:
points: [[10.373926, 52.042989], [10.384043, 52.042289]]
details: [road_class, street_name]
# pick the previously defined custom profile
profile: car
# The formula is defined in CustomWeighting, where the speed_factor can be used to decrease speed.
# The minimum value is 0 and the maximum is 1. If two conditions are satisfied the values are
# multiplied e.g. if road_class==motorway and road_environment==tunnel, then the resulting speed is
# average_speed*0.85*0.9.
speed_factor:
road_environment:
tunnel: 0.85
# trucks should be a bit slower on certain road classes compared to the 'base' car
# a more compact JSON-way to list the properties, fully YAML-compatible:
road_class: { motorway: 0.85, primary: 0.9 }
# You can lower the average speed for certain conditions via this max_speed entry. The speed_factor is applied before
# this operation. See the following example that sets the maximum speed to 95km/h for motorways.
max_speed:
road_class:
motorway: 95
residential: 30
road_environment:
bridge: 85
# Trucks are slower so limit all speed to this value. In km/h
# If none of the conditions in the map above apply use the fallback, if specified:
max_speed_fallback: 100
# This term changes the influence of the distance. I.e. longer roads get a higher cost.
# The distance_influence is independent of the edge properties and does not influence the ETA. The default is 70 (seconds/1km).
# Let's assume a route that takes 1000sec and is 10km long, then a value of 30 means that I would like to drive maximum 11km
# to reduce the travel time to 970sec or 12km to reduce it to 940sec.
distance_influence: 90
# Now we want to set a preference without changing the taken time. The default priority is 1 and does not change something.
# Higher than 1 is not possible directly, only via the catch-all key "*", see road_class where motorways are preferred. Lower than 1 will avoid it.
# If two conditions are met the values will be multiplied. The minimum value is 0 (block access) the maximum value is 100.
priority:
road_class:
motorway: 1.0
residential: 0.4
"*": 0.9
# and if you do not want that tracks go over tracks just exclude them via
# track: 0
# let's assume we transport gas: so NEVER go on restricted roads with hazmat==no
hazmat: { no: 0 }
# avoid destination-only roads
road_access: { destination: 0.1 }
# avoid turns if possible and links are one simple indication for that
road_class_link:
true: 0.5
# avoid toll roads
toll: { no: 1, "*": 0.5 }
# avoid a certain area
area_custom1: 0.5
max_weight: { "<4.5": 0 }
max_height: { "<3.8": 0 }
max_width: { "<2.5": 0 }
areas:
custom1:
type: "Feature"
geometry: { type: "Polygon", coordinates: [[[13.722, 51.053], [13.722, 51.055], [13.731, 51.055], [13.731, 51.053], [13.722, 51.053]]] }
```
Or a cargo bike:
```yml
profile: bike
# let's assume cargo bikes are e-bikes and we should lower speed to prefer shorter and not faster routes automatically
max_speed:
road_class:
primary: 28
# if no other condition matches
max_speed_fallback: 25
priority:
# exclude steps
road_class:
steps: 0
surface:
sand: 0.5
# prefer better tracks
track_type: { other: 1.0, grade1: 1.0, "*": 0.9 }
# prefer official bike routes
bike_network: { other: 0.5 }
# avoid all situations where we have to get off the bike
get_off_bike:
"true": 0.5
max_height:
"<2.3": 0
max_width:
"<1.2": 0
```
Now the interesting part is that you can try this out in the demo UI and when you are happy you can use the exact same content to create a file (e.g. truck.yml) and use it for the LM or CH preparation to make the query much faster, like so:
```yaml
profiles:
- name: truck
vehicle: car
weighting: custom
custom_model_file: path/to/truck.yml
- name: cargo_bike
vehicle: bike
weighting: custom
custom_model_file: path/to/cargo_bike.yml
```
To query such a prepared custom profile you specify e.g. `profile=truck`. If you want to customize your request further you use A* or LM and the `/route-custom` endpoint. If no further customization is required or you want to use a CH-prepared custom profile you use the `/route` endpoint.
When you try this out in the UI you currently have to include custom weighting and potentially disabling turn cost: http://localhost:8989/maps/?point=51.060063%2C13.714714&point=51.064378%2C13.756599&weighting=custom&turn_costs=false
- [x] basic changes of the Weighting already work
- [x] examples and description is available in truck.yml, RouteResourceTest and CustomWeightingRouteResourceTest. Other examples could be: wheelchair (base:foot), electric_bike, cargo_bike (base:electric_bike -> which is not a FlagEncoder!), electric_car (base:car), bike that prefers official bike routes much more, fire_truck (base:truck), scooter (speed limited car?)
- [x] query via json or yaml
- [x] no need for a jackson dependency in core
- [x] ~~implement DelayFlexConfig~~ see below
- [x] for now remove `delay` as there are similar problems like for elevation (needs to be splitted for virtual edges) otherwise it sums up if there are several query points on a single edge.
- [x] `-d` for curl does not work for yaml as it strips line breaks. Instead use `--data-binary`
- [x] make FlexModelWeighting possible for CH and LM preparation too, without introducing a jackson dependency to core (due to Android)
- [x] priority is implemented wrong => it should be `priority=1/factor`
- [x] support for weight, height, width limitations
- [x] more "FlexModelWeighting" features like shown [here](https://gist.github.com/karussell/7a38acf13b2aa3a1bc8344d14d4da038#file-v4-yml) should follow in separate PRs
- [x] merge FlexResource into RouteResource but only support it for POST
- [x] simple UI support like [here](https://github.com/graphhopper/graphhopper/compare/flex_vehicleprofile2#diff-9941521f8b7f449ab4c126d163e4d94aR74)
- [x] throw illegal arg exception if we use max_height but this is not supported from the storage.
- [x] documentation in config-example.yml and docs/ maybe
- [x] rename weight -> max_vehicle_weight etc
- [x] Yaml is much more standard & "better", but even our simplistic gscript implementation had a much better error reporting. Maybe also allow ScriptedWeighting via some interfaces and bring it other languages. For now yaml is ok. Later we might add a ScriptedWeighting.
- [x] ~~if speed value is too slow => rounding to zero => is blocking desired?~~ should not happen as we do not store the weight again into an EncodedValue
- [x] ~~different/separate formula for calcWeight vs. calcTime?~~ probably not now
- [x] add difference of average_speed handling (first wins) vs. speed_factor handling (all multiplied) to docs
- [x] throw error if value in average_speed is higher than max_speed
- [x] add custom request to Measurement
- [x] extend from AbstractWeighting
- [x] A very significant route change and query speed difference between normal "car" requests and a custom weighting with `base: car` is the default of `distance_factor` being 1 instead of 0. I.e. a [normal car request](http://localhost:8989/maps/?point=48.458352%2C10.546875&point=49.410973%2C11.392822) takes 1100ms and the same as custom request with just "base: car" takes only 350ms. For now changed this default to 0.07 (copied from ShortFastestWeighting)
- [x] replace `average_speed` with a "max speed" variable or even remove it completely? The `average_speed` is not that useful. The speed value could be from traffic sources and we would completely overwrite it by a rather simple condition like road_class==motorway. Also it should be probably more a `max_average_speed` (instead of max speed)
- [x] before merging we need #729
- [x] CustomerWeighting should always increase weight (normalize priority and factor)
- [x] CustomWeighting does not work with LM (not in Measurement and not in RouteResource)
- [x] See issue #1708. Instead of `base` should we always pick the vehicle from the request? (this won't work for CH.) -> Do not use `vehicle` as it will collide with GHRequest.vehicle.
- [x] avoid nesting the `model` in the request because using the profile in a request or a request for a profile is ugly and only possible via editing. But then a separate endpoint is required?
- [x] throw exception if vehicle and weighting is specified in URL but does not match -> not needed if good documentation
- [x] introduce distance_factor map
- [x] web UI: keep flex config between requests (update: this only happens when setting weighting=flex as url parameter)
- [x] "sum" of terms instead of "multiplication" would make more sense, but then "preferring" a certain attribute is no longer as easy and often a requirement if we want to keep the yaml expressions simple (i.e. no complex boolean algebra)
- [x] check again about A* correctness (min weight -> priority and speed range)
- [x] is distance_factor map really necessary or too complicated too understand the nuances to "inverse priority"? -> for now remove it
- [x] allow boolean encoded values too like road_class_link or get_off_bike. Would be great to use this for "avoiding".
- [x] support block_area in the CustomModel
- [x] simple turn cost support for query -> already works via root-level fields populated GHRequest.hints (e.g. disabling works via `edge_based: false`)
- [x] switch to snake case is a bit ugly as current config.ymls stop working
- [x] Avoid error: "You can try disabling LM by setting lm.disable=true"
- [x] try using country enum
- [x] remove snake case as it is already in PR #1918
- [x] for fun try "avoid left turns" via per-request geometry evaluation https://github.com/graphhopper/graphhopper/commit/38b62f069f5f2f48e84b3d50cf4bbf85286cb4d2
- [x] remove newly introduced name parameter from AbstractWeighting
- [x] change formula to either `(const_distance_term + 1/v/prio)*d` (no multiplication of prio with const_distance_term) or `(const_distance_term + k/v + sum(some_map))*d` (instead of priority multiplication use summation)
- [x] replace mixins with SNAKE-enabled jackson parser
- [x] if a CustomModel `abc` is known to GraphHopper and then there is a CustomRequest with `profile=abc` but a few parameters have changed, then it should be possible to merge them. Maybe in `CustomWeighting.prepareRequest`?
- [x] is it ugly that we do `profile=custommodel` against `/route` but if it is a CustomRequest it is against `/route-custom`. Especially if it is a CustomRequest with `profile=custommodel` (see point above) it gets strange.
- [x] turn cost support for CH/LM preparation -> should be already done via profiles
- [x] CustomModel.merge should multiply (or pick maximum) to keep LM-correctness
- [x] solve #1708
- [x] change endpoint to `/route-custom`
- [x] add JSON support for file import and remove yaml endpoint
- [x] formula change #2021
- [x] merge different ranges of DecimalEncodedValue
- [ ] update docs
- [x] revert FlagEncoder changes
Later:
- [ ] solve #1835
- [ ] CustomRequest for Isochrone & SPT endpoint (POST request)
- [ ] preliminary & simple tests show at least 20% slower CH preparation even for a very simple FlexModel. -> minor improvements to 10% - but for the simple case only! Can we somehow cache calls to edge.get(roadClass) so that at least if only road_class is used for the different FlexConfigs it does not get slower?
- [ ] Cache EncodedValues in a later issue
- [ ] try to get car turn restrictions working for a bike model (`car.turn_cost`)
- [ ] web UI: show more useful error messages?
- [ ] allow DoubleEncodedValue as value on the "right side" e.g. to directly use an EncodedValue as speed_factor. This is a bit complicated as we do not know the maximum value and cannot ensure correctness for LM.
- [ ] allow != and &&
- [ ] later allow a special shorter notation ala
`speed_factor: { road_class: { motorway||trunk: 1.1 }}` instead of
`speed_factor: { road_class: { motorway: 1.1, trunk: 1.1 }}`?
- [ ] elevation. Let us assume we store the up and down elevation in EncodedValues then this is more involved for a QueryGraph as we need to split this for virtual edges like it is currently done also for the distance. Keep in mind that there are different use cases for elevation: one for weight-only (electric car) and one where weight and time is changed (bike that avoid hills). Also do delay (time offset) later as same problems like for elevation #1898
- [ ] avoid changing a country
- [x] turn cost support in Weighting class (see #1820) and also for importFlexModels
- [ ] ScriptedWeighting
graphhopper:master
← graphhopper:janino_scripting
opened 09:36AM - 10 Dec 20 UTC
This change is intended to make custom profiles more powerful and also more read… able. It breaks backward compatibility. It uses a very powerful and fast library called [Janino](https://janino-compiler.github.io/janino/) that allows on-the-fly compilation of partial expressions, blocks or whole classes written in Java.
With Janino scripting we can satisfy much more use cases. As soon as custom profiles were released there was often the need to combine expressions like "use factor=0.5 only if road_class is primary AND in a certain area" or "surface is unpaved AND road_class is not unclassified". This is now possible. Still I do not think we should use more power of Janino and keep the scripting simple (expressions only).
The big advantage with Janino scripting is now that we do not have the burden in the future to extend or improve our own interpreter. Performance-wise the routing with Janino scripting is at least of the same speed (for preparation it is even faster!). For query time this sounds a bit strange as the compilation takes some time. But when we assume that a custom profile is re-used for future requests of the same profile the class cache helps (see ExpressionBuilder, but the cache can be disabled). Now the most surprising thing for me was that even without the cache the slow down is not much (20% for LM queries? precise numbers needs to be re-done for the most recent commit). Janino is able to compile many expressions and a whole class in only a few milliseconds. And currently we create a cached version automatically for all server-side profiles due to GraphHopper.checkProfilesConsistency, i.e. for them there is no compilation overhead. btw: our cache is not that bad or complex as the cached object for key X is always the same (unlike for DBs with user data).
We have to add Janino as dependency to the core (1MB). I find this acceptable for what we get.
A custom profile in the current master looks like:
```yml
priority:
max_height: {"<3.8": 0}
speed_factor:
road_class:
motorway: 0.85
primary: 0.9
max_speed:
road_class:
living_street: 2
max_speed_fallback: 80
```
with this branch this gets more readable and you do not need to study the docs and still know what it does:
```yml
priority:
- if: max_height < 3.80
multiply by: 0
speed:
- if: road_class == MOTORWAY:
multiply by: 0.85
- if: road_class == PRIMARY
multiply by: 0.9
# if you wish to keep compatibility you apply max_speed after all multiplications, but in general you can use "limit to" at any time
- if: road_class == LIVING_STREET
limit to: 2
# replacement for max_speed_fallback which is now implicit
# it could be also done via the else statement of the last if ... or via:
- if: true
limit to: 80
```
It is slightly more verbose for the enum case, but also more clear.
And now new things like inequality `!=` or AND-expressions `&&` are possible and all other comparison operators (`<=`, `>`, ...). See this example:
```yml
priority:
- if: road_class == MOTORWAY && max_speed >= 70
multiply by: 0.85
- if: road_class != PRIMARY
multiply by: 0.9
```
We could even use (whitelisted) methods in the expression like `road.class.ordinal() < 3`.
The `multiply by` operation can now be `>1` for server-side profiles. Note that the custom profile in the **query** still has the limitation of `factor<=1`.
Also areas are still possible and can be combined with other expressions:
```yml
expressionA && in_area_custom1: 0.5
areas:
- custom1: <geojson>
```
The area check is done only if expressionA is true and should be avoided to be called multiple times.
For some use cases (and sometimes for performance) you can use else-if and else:
```yml
priority:
- if: <expressionA>
multiply by: 0.85
- else if: <expressionB>
multiply by: 0.6
- else:
multiply by: 0.8
```
Such a block is also what happens for a single encoded value in master branch so you can use this for migration.
---
Technically Janino compiles CustomWeightingHelper at runtime and adds user expressions to it before (in a safe manner).
Most code changes and tests are necessary to make the janino compilation safe regarding attacks from user expressions. Of course it should not be possible to call System.exit, mine crypto currencies or read the file system on the routing servers that could be exposed to the public. Additionally to this work we could think about enabling the SecurityManager (see [here](https://github.com/graphhopper/graphhopper/tree/security_manager)) but this only further increases security and would not be the main protection. The main protection comes from the fact that we allow only "simple" expressions like "if" clauses and enforce that there are no assignments, no loops, no lambdas or other stuff happening in user expressions and have a whitelist of variable names and methods. We do this with the help of the Scanner API from Janino and traverse the code (ExpressionVisitor).
Also we want simple and readable user expressions. For that we need to declare and assign variables before like `road_class` and create the class variable `EncodedValue<RoadClass> roadClassEnc` and also make constants like `NO` more specific (convert it into `Toll.NO` if the `toll` encoded value is used). The user should not need to do that. With the nice API of Janino I found a safe way to mix arbitrary code with the user expressions. First I create Java objects (`List<Java.BlockStatement>`) from all the user expressions (it is ensured that they are simple) and inject them into the parsed class CustomWeightingHelper (`Java.CompilationUnit`) containing arbitrary Java code like assignments and the compile this into byte code, load the class and finally instantiate an object of it.
To my knowledge and experiments Janino does not use the Metaspace (unlike javaassist) and instead it creates the classes on the heap.
I’m trying to create a custom profile following the instructions here: https://github.com/graphhopper/graphhopper/pull/1841
but I get the following error:
java.lang.IllegalArgumentException: Could not create weighting for profile: 'emergencycar'. │serve: /route?point=36.04682,-75.6827&point=35.90204,-75.671464&vehicle=emergencytruck&calc_points=true&points_encoded=false&ch.di
Profile: name=emergencycar|vehicle=emergencycar|weighting=custom|turnCosts=false|hints…
Thanks in advance!
You should return HGV here.
How I can fix it?
Can you paste your config in quotes (```) with the correct white space?
1 Like
You should return HGV here.
Many thanks. Updated code
Can you paste your config in quotes (```) with the correct white space?
Sure.
graph.flag_encoders: small_truck|turn_costs=true
graph.dataaccess: RAM_STORE
graph.encoded_values: max_weight,max_height,max_width
profiles:
- name: small_truck_short_fastest
vehicle: small_truck
turn_costs: true
weighting: custom
custom_model_file: ./small_truck.yml
small_truck.yml
priority:
- if: max_weight < 3.5
multiply by: 0
I tried to reproduce this problem with the latest version but wasn’t able to. You need to make this new FlagEncoder available in DefaultFlagEncoderFactory:
if (name.equals("small_truck")) return new TruckFlagEncoder(configuration);
Ensure that “small_truck” is also used in toString of TruckFlagEncoder.
Then set this factory to the GraphHopper class.
Please make sure you use the latest version 4.0 too. E.g. the function is called multiply_by
and no longer multiply by
.
btw: you could directly embed the small_truck custom model:
graphhopper:
...
profiles:
- name: small_truck_short_fastest
custom_model:
priority:
- if: max_weight < 3.5
multiply_by: 0
In recent version also no path entry is allowed in custom_model_file and you would need:
graphhopper:
...
custom_model_folder: ./custom_models
profiles:
- name: small_truck_short_fastest
vehicle: small_truck
turn_costs: true
weighting: custom
custom_model_file: small_truck.yml
If you still cannot make it working you would need to send a minimal project where this problem occurs so that I can debug it.
1 Like
Hi!
Thank you so much for the quick response.
I tried to reproduce this problem with the latest version but wasn’t able to. You need to make this new FlagEncoder available in DefaultFlagEncoderFactory
Sure. Here is my code:
@Component
public class CustomFlagEncoderFactory extends DefaultFlagEncoderFactory {
public CustomFlagEncoderFactory() {
super();
}
@Override
public FlagEncoder createFlagEncoder(String name, PMap configuration) {
if (name.equals(VehicleType.MOFA.getValue())) {
return new MofaFlagEncoder(configuration);
} else if (name.equals(VehicleType.SMALL_TRUCK.getValue())) {
return new SmallTruckEncoder(configuration);
} else if (name.equals(VehicleType.TRUCK.getValue())) {
return new TruckEncoder(configuration);
}
return super.createFlagEncoder(name, configuration);
}
}
Ensure that “small_truck” is also used in toString of TruckFlagEncoder.
Done.
Then set this factory to the GraphHopper class.
hopper.setEncodingManager(EncodingManager.create(smallTruckEncoder));
hopper.setFlagEncoderFactory(customFlagEncoderFactory);
Please make sure you use the latest version 4.0 too. E.g. the function is called multiply_by
and no longer multiply by
.
I’m using graphopper-reader-osm 3.0-pre3 (not graphhopper-core). In mvnrepository the latest version of this library is 3.0-pre3, not 4.0.
https://mvnrepository.com/artifact/com.graphhopper/graphhopper-reader-osm
Do I need to build the latest version manually?
Thanks in advance!
You should only use releases like 3.0 or 4.0. Not pre-releases.
Do I need to build the latest version manually?
No. You can use graphhopper-core - see here: Move the reader-osm module into core by easbar · Pull Request #2298 · graphhopper/graphhopper · GitHub
1 Like
So I have updated graphhopper to 4.0. But the exception not gone away.
If you still cannot make it working you would need to send a minimal project where this problem occurs so that I can debug it.
Here is the minimal project:
https://github.com/hasanli-orkhan/gh-demo
Exception from DefaultWeightingFactory
Caused by: java.lang.IllegalArgumentException: Could not create weighting for profile: 'small_truck_short_fastest'.
Profile: name=small_truck_short_fastest|vehicle=small_truck|weighting=custom|turnCosts=true|hints={}
Error: custom weighting requires a CustomProfile but was profile=small_truck_short_fastest
Thank you so much! I have resolved this issue by adding CustomProfile in Java configuration.
1 Like
system
Closed
March 13, 2022, 7:20pm
11
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.