Error indexing graphhopper maps

Hello,

I’ve been trying to index a planet pbf with three profiles, car, truck, and big-truck (custom).

I’m trying to activate ch profile for car and truck, and lm profile for all of them.

But, when compiling the planet pbf, I received the following error.

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.UnsupportedOperationException: java.lang.UnsupportedOperationException: 
java.lang.UnsupportedOperationException: landmark weight factor cannot be bigger than Integer.MAX_VALUE 2.6281334681504593E9
	at com.graphhopper.util.GHUtility.runConcurrently(GHUtility.java:510)
	at com.graphhopper.routing.lm.LMPreparationHandler.prepare(LMPreparationHandler.java:192)
	at com.graphhopper.GraphHopper.prepareLM(GraphHopper.java:1384)
	at com.graphhopper.GraphHopper.loadOrPrepareLM(GraphHopper.java:1360)
	at com.graphhopper.GraphHopper.postProcessing(GraphHopper.java:1175)
	at com.graphhopper.GraphHopper.process(GraphHopper.java:819)
	at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:763)
	at com.graphhopper.http.GraphHopperManaged.start(GraphHopperManaged.java:44)
	at io.dropwizard.lifecycle.JettyManaged.doStart(JettyManaged.java:27)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.server.Server.start(Server.java:423)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.server.Server.doStart(Server.java:387)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
	at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:52)
	at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:68)
	at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
	at io.dropwizard.cli.Cli.run(Cli.java:78)
	at io.dropwizard.Application.run(Application.java:94)
	at com.graphhopper.application.GraphHopperApplication.main(GraphHopperApplication.java:38)
Caused by: java.util.concurrent.ExecutionException: java.lang.UnsupportedOperationException: java.lang.UnsupportedOperationException: java.lang.UnsupportedOperationException: landmark weight factor cannot be bigger than Integer.MAX_VALUE 2.6281334681504593E9
	at java.base/java.util.concurrent.ForkJoinTask.reportExecutionException(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.get(Unknown Source)
	at com.graphhopper.util.GHUtility.runConcurrently(GHUtility.java:508)
	... 21 more
Caused by: java.lang.UnsupportedOperationException: java.lang.UnsupportedOperationException: java.lang.UnsupportedO… [message truncated due to size]

I’m running compilation in a virtual machine with 60 vCPUs and 480 GB RAM. So I believe the problem was not about compute resources.

Here my config.yml

    graphhopper:
      datareader.file: ""
      datareader.worker_threads: 16
      custom_models.directory: /graphhopper/profiles
      profiles:
        - name: car
          custom_model_files: [ car.json ]
        - name: truck
          custom_model_files: [ truck.json ]
        - name: big_truck
          weighting: custom
          custom_model_files: [ big_truck.json ]
      profiles_ch:
        - profile: car
        - profile: truck
      profiles_lm:
        - profile: car
        - profile: truck
        - profile: big_truck
      prepare.min_network_size: 1000
      prepare.subnetworks.threads: 16
      prepare.ch.threads: 2
      prepare.lm.threads: 3
      routing.non_ch.max_waypoint_distance: 100000000 #100.000.000
      routing.max_visited_nodes: 15000000 #15.000.000
      import.osm.ignored_highways: footway,cycleway,path,pedestrian,steps # typically useful for motorized-only routing
      index.max_region_search: 30
      graph.location: graph-cache
      graph.dataaccess.default_type: RAM_STORE
      graph.encoded_values: hgv,max_weight,max_height,max_width,toll,car_access,car_average_speed,road_access,road_class
    server:
      application_connectors:
        - type: http
          port: 8989
          bind_host: localhost
          max_request_header_size: 50k
      request_log:
        appenders: []
      admin_connectors:
        - type: http
          port: 8990
          bind_host: localhost
    logging:
      appenders:
        - type: file
          time_zone: UTC
          current_log_filename: logs/graphhopper.log
          log_format: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
          archive: true
          archived_log_filename_pattern: ./logs/graphhopper-%d.log.gz
          archived_file_count: 30
          never_block: true
        - type: console
          time_zone: UTC
          log_format: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
      loggers:
        "com.graphhopper.osm_warnings":
          level: DEBUG
          additive: false
          appenders:
            - type: file
              currentLogFilename: logs/osm_warnings.log
              archive: false
              logFormat: '[%level] %msg%n'

Any idea?

Please identify for which of the 3 vehicle profiles this happens.

What is the actual config of this vehicle?

btw:

routing.max_visited_nodes: 15000000 #15.000.000

it should be possible to directly use:

routing.max_visited_nodes: 15_000_000

@karussell

I have the following log right before the error stacktrace:

2025-06-30 20:45:10.410 [main] INFO com.graphhopper.reader.osm.OSMReader - Finished reading OSM file: /data/pbf/planet.osm.pbf, nodes: 336,778,946, edges: 419,707,115, zero distance edges: 6,162,184
2025-06-30 20:45:10.413 [main] INFO c.g.r.s.PrepareRoutingSubnetworks - Start marking subnetworks, prepare.min_network_size: 1000, threads: 16, nodes: 336,778,946, edges: 419,707,115, jobs: [car_subnetwork|com.graphhopper.routing.weighting.custom.CustomWeighting@2ecf5915, truck_subnetwork|com.graphhopper.routing.weighting.custom.CustomWeighting@53e76c11, big_truck_subnetwork|com.graphhopper.routing.weighting.custom.CustomWeighting@42cc183e], totalMB:127789, usedMB:97794
2025-06-30 20:49:24.939 [ForkJoinPool-1-worker-2] INFO c.g.r.s.PrepareRoutingSubnetworks - car - Found 103677357 subnetworks (102159595 single edges and 1517762 components with more than one edge, total nodes: 839414230), took: 254.5125s
2025-06-30 20:49:29.263 [ForkJoinPool-1-worker-2] INFO c.g.r.s.PrepareRoutingSubnetworks - car - Marked 103677153 subnetworks (biggest: 1978 edges) -> 204 components(s) remain (smallest: 2002, biggest: 470281303 edges), total marked edges: 5122189, took: 4.3210773s
2025-06-30 20:50:10.880 [ForkJoinPool-1-worker-1] INFO c.g.r.s.PrepareRoutingSubnetworks - truck - Found 105918536 subnetworks (104226982 single edges and 1691554 components with more than one edge, total nodes: 839414230), took: 300.45685s
2025-06-30 20:50:17.923 [ForkJoinPool-1-worker-1] INFO c.g.r.s.PrepareRoutingSubnetworks - truck - Marked 105918315 subnetworks (biggest: 1978 edges) -> 221 components(s) remain (smallest: 2002, biggest: 467866400 edges), total marked edges: 5855567, took: 7.042985s
2025-06-30 20:50:48.596 [ForkJoinPool-1-worker-3] INFO c.g.r.s.PrepareRoutingSubnetworks - big_truck - Found 1345781 subnetworks (0 single edges and 1345781 components with more than one edge, total nodes: 839414230), took: 338.17303s
2025-06-30 20:50:49.299 [ForkJoinPool-1-worker-3] INFO c.g.r.s.PrepareRoutingSubnetworks - big_truck - Marked 1345607 subnetworks (biggest: 1980 edges) -> 174 components(s) remain (smallest: 2022, biggest: 530435538 edges), total marked edges: 3624141, took: 0.70254093s
2025-06-30 20:50:59.980 [main] INFO c.g.r.s.PrepareRoutingSubnetworks - Finished finding and marking subnetworks for 3 jobs, took: 349.56754s, totalMB:128796, usedMB:91607
2025-06-30 20:50:59.981 [main] INFO com.graphhopper.GraphHopper - nodes: 336,778,946, edges: 419,707,115
2025-06-30 20:51:09.106 [main] INFO c.g.routing.lm.LMPreparationHandler - Creating LM preparations, totalMB:128796, usedMB:94379
2025-06-30 20:51:09.108 [ForkJoinPool-3-worker-2] INFO c.g.routing.lm.LMPreparationHandler - 1/3 calling LM prepare.doWork for car ... (totalMB:128796, usedMB:94431)
2025-06-30 20:51:09.108 [ForkJoinPool-3-worker-1] INFO c.g.routing.lm.LMPreparationHandler - 2/3 calling LM prepare.doWork for truck ... (totalMB:128796, usedMB:94405)
2025-06-30 20:51:09.108 [ForkJoinPool-3-worker-3] INFO c.g.routing.lm.LMPreparationHandler - 3/3 calling LM prepare.doWork for big_truck ... (totalMB:128796, usedMB:94457)
2025-06-30 20:51:09.109 [big_truck] INFO c.g.routing.lm.PrepareLandmarks - Start calculating 16 landmarks, weighting:LM_BFS|custom|com.graphhopper.routing.weighting.custom.CustomWeighting@1242bb93, totalMB:128796, usedMB:94457
2025-06-30 20:51:09.109 [truck] INFO c.g.routing.lm.PrepareLandmarks - Start calculating 16 landmarks, weighting:LM_BFS|custom|com.graphhopper.routing.weighting.custom.CustomWeighting@4836af90, totalMB:128796, usedMB:94457
2025-06-30 20:51:09.109 [car] INFO c.g.routing.lm.PrepareLandmarks - Start calculating 16 landmarks, weighting:LM_BFS|custom|com.graphhopper.routing.weighting.custom.CustomWeighting@109ddb22, totalMB:128796, usedMB:94457
2025-07-01 00:40:07.832 [car] INFO c.g.routing.lm.PrepareLandmarks - Calculated landmarks for 13 subnetworks, took:13738s => { "type": "FeatureCollection", "features": [{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [34.0480371, 44.41588]}, "properties":{"node_index":264614864,"subnetwork":1,"lm_index":0}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [-6.2456927, 58.499446]}, "properties":{"node_index":263306874,"subnetwork":1,"lm_index":1}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [13.5639766, 47.8276236]}, "properties":{"node_index":315215877,"subnetwork":1,"lm_index":2}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [29.5268002, 63.26639]}, "properties":{"node_index":253183626,"subnetwork":1,"lm_index":3}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [28.7520652, 51.4850322]}, "properties":{"node_index":116074112,"subnetwork":1,"lm_index":4}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [-13.4592346, 29.2017568]}, "properties":{"node_index":265573900,"subnetwork":1,"lm_index":5}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [-10.1954776, 53.9673225]}, "properties":{"node_index":264928418,"subnetwork":1,"lm_index":6}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [27.908026, 42.1198936]}, "properties":{"node_index":269291487,"subnetwork":1,"lm_index":7}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [7.4677856, 61.7626002]}, "properties":{"node_index":278688308,"subnetwork":1,"lm_index":8}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [6.7304825, 46.379515]}, "properties":{"node_index":239049039,"subnetwork":1,"lm_index":9}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [20.5872305, 49.4140653]}, "properties":{"node_index":78602074,"subnetwork":1,"lm_index":10}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [7.8585236, 51.3330262]}, "properties":{"node_index":257000026,"subnetwork":1,"lm_index":11}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [14.1911915, 36.0687684]}, "properties":{"node_index":264175120,"subnetwork":1,"lm_index":12}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [19.4986457, 70.2397474]}, "properties":{"node_index":276094877,"subnetwork":1,"lm_index":13}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [1.7417662, 52.…
2025-07-01 00:40:07.832 [car] INFO c.g.routing.lm.LMPreparationHandler - LM car finished totalMB:245897, usedMB:194126
2025-07-01 00:57:44.742 [truck] INFO c.g.routing.lm.PrepareLandmarks - Calculated landmarks for 13 subnetworks, took:14795s => { "type": "FeatureCollection", "features": [{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [34.0480371, 44.41588]}, "properties":{"node_index":264614864,"subnetwork":1,"lm_index":0}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [-6.2447172, 58.500053]}, "properties":{"node_index":263306870,"subnetwork":1,"lm_index":1}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [29.5268002, 63.26639]}, "properties":{"node_index":253183626,"subnetwork":1,"lm_index":2}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [12.4915045, 47.3644691]}, "properties":{"node_index":255863312,"subnetwork":1,"lm_index":3}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [20.5872305, 49.4140653]}, "properties":{"node_index":78602074,"subnetwork":1,"lm_index":4}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [-13.4811686, 29.2136407]}, "properties":{"node_index":76812398,"subnetwork":1,"lm_index":5}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [-10.1954776, 53.9673225]}, "properties":{"node_index":264928418,"subnetwork":1,"lm_index":6}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [8.3267662, 61.0196268]}, "properties":{"node_index":320627917,"subnetwork":1,"lm_index":7}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [7.5001531, 46.4315853]}, "properties":{"node_index":261241971,"subnetwork":1,"lm_index":8}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [27.7371802, 42.6588252]}, "properties":{"node_index":288719774,"subnetwork":1,"lm_index":9}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [1.7417662, 52.506824]}, "properties":{"node_index":318176730,"subnetwork":1,"lm_index":10}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [10.4521956, 51.2246053]}, "properties":{"node_index":257124992,"subnetwork":1,"lm_index":11}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [19.4986457, 70.2397474]}, "properties":{"node_index":276094877,"subnetwork":1,"lm_index":12}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [14.1911915, 36.0687684]}, "properties":{"node_index":264175120,"subnetwork":1,"lm_index":13}},{ "type": "Feature", "geometry": {"type": "Point", "coordinates": [27.6349365, …
2025-07-01 00:57:44.742 [truck] INFO c.g.routing.lm.LMPreparationHandler - LM truck finished totalMB:246741, usedMB:148076
2025-07-01 01:30:13.659 [main] ERROR io.dropwizard.cli.ServerCommand - Unable to start server, shutting down

Before that, I only have logs of pbf parsing status.

big_truck seems to be the problem (as it is started but not finished). What config and custom model were you using for it?

big-truck.json

    {
        "priority": [
            {
                "if": "road_class == MOTORWAY",
                "multiply_by": "1"
            },
            {
                "if": "road_class == TRUNK",
                "multiply_by": "0.8"
            },
            {
                "if": "road_class == PRIMARY",
                "multiply_by": "0.7"
            },
            {
                "if": "max_width < 3 || max_height < 4",
                "multiply_by": "0.01"
            },
            {
                "if": "max_weight < 18",
                "multiply_by": "0.01"
            },
            {
                "else": "",
                "multiply_by": "1.0"
            }
        ],
        "speed": [
            {
                "if": "true",
                "limit_to": "car_average_speed * 0.9"
            },
            {
                "if": "max_height < 4.5",
                "multiply_by": "0.5"
            },
            {
                "if": "max_weight < 40",
                "multiply_by": "0.5"
            },
            {
                "else": "",
                "multiply_by": "1.0"
            }
        ]
    }

config.yml

graphhopper:
      datareader.file: ""
      datareader.worker_threads: 16
      custom_models.directory: /graphhopper/profiles
      profiles:
        - name: car
          custom_model_files: [ car.json ]
        - name: truck
          custom_model_files: [ truck.json ]
        - name: big_truck
          weighting: custom
          custom_model_files: [ big_truck.json ]
      profiles_ch:
        - profile: car
        - profile: truck
      profiles_lm:
        - profile: car
        - profile: truck
        - profile: big_truck
      prepare.min_network_size: 1000
      prepare.subnetworks.threads: 16
      prepare.ch.threads: 2
      prepare.lm.threads: 3
      routing.non_ch.max_waypoint_distance: 100000000 #100.000.000
      routing.max_visited_nodes: 15000000 #15.000.000
      import.osm.ignored_highways: footway,cycleway,path,pedestrian,steps # typically useful for motorized-only routing
      index.max_region_search: 30
      graph.location: graph-cache
      graph.dataaccess.default_type: RAM_STORE
      graph.encoded_values: hgv,max_weight,max_height,max_width,toll,car_access,car_average_speed,road_access,road_class
    server:
      application_connectors:
        - type: http
          port: 8989
          bind_host: localhost
          max_request_header_size: 50k
      request_log:
        appenders: []
      admin_connectors:
        - type: http
          port: 8990
          bind_host: localhost
    logging:
      appenders:
        - type: file
          time_zone: UTC
          current_log_filename: logs/graphhopper.log
          log_format: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
          archive: true
          archived_log_filename_pattern: ./logs/graphhopper-%d.log.gz
          archived_file_count: 30
          never_block: true
        - type: console
          time_zone: UTC
          log_format: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
      loggers:
        "com.graphhopper.osm_warnings":
          level: DEBUG
          additive: false
          appenders:
            - type: file
              currentLogFilename: logs/osm_warnings.log
              archive: false
              logFormat: '[%level] %msg%n'

Try instead of 0.01 a smallest factor of only 0.1 (or 0.05).

Or better completely exclude those roads via a factor of 0. Because if you have a big truck and the road has a known maximum width or weight property, why should the big truck ever use this road if it can’t pass?

Hello @karussell

Sorry for the delay, but I was trying to validate what you suggested, and It worked!!! Thank you!!!

But I have a consideration. My compilation process used to take about 9 hours, and now it took almost 16 hours.

Considering the vm resources I have available, is there something I could do to improve the compilation process??

Those “threads” mentioned in the config.yml (datareader.worker_threads, prepare.subnetworks.threads, prepare.ch.threads, prepare.lm.threads) improve the compilation process if I increase the value?

Any suggestion to improve this process are too welcome.

By compilation process you mean the import time? And it went up when you changed one of your factors from 0.01 to a bigger value like 0.1 or to zero? That would be surprising as I would expect the opposite. Can you check the log to see which step exactly took longer than before, and give more details what you changed? And since you are using vCPUs can you rule out this was only by coincidence or does this happen consistently?

Increasing the threads value only speeds up the process until there is one thread for each profile. At this point all profiles will be prepared in parallel. Increasing the value further won’t make a difference.

Hey @easbar ,

By compilation process you mean the import time?

No, I mean the time graphhopper takes to read pbf and save into default-gh folder.

And it went up when you changed one of your factors from 0.01 to a bigger value like 0.1 or to zero?

Yes, and No! I believe the time increasing would be expected, because I have 3 different profiles. All of them with LM, and two of them with CH. So the time problem, is not exatcly about the factor by itself.

But the point is if there’s anything I could “activate” to speed up the “compilation” process from 16 hours, to 9 hours. You understand me?

I’ll try to get the logs, but I think It’d be not possible

Ok this is what I call the ‘import’ :slight_smile:

Ok, if you enable LM and CH a (much) longer import time is expected.

Take a look at this article: Host Your Own Worldwide Route Calculator With GraphHopper - GraphHopper Directions API Maybe there is something you did not consider, yet. Also note that the import for CH takes much longer if turn costs are enabled.

1 Like