Modifying Dockerfile, config-example.yml, and setting up Graphhopper via Docker with Your Own Data

Can someone review my current understanding of the localhost install process? (windows 10 computer, using Docker-Toolbox VirtualBox). **I am trying to get graphhopper to run on my localhost:8989 using my regions PBF and the bike vehicle profiles

  • I need to switch berlin-latest.osm.pbf to the arizona-latest.osm.pbf
  • and switch the car profile to bike, bike2, mtb, racingbike

I have successfully run graphhopper on localhost:8989 with the default germany-berlin data, and the default car profile - so I know this should work.

To setup a custom install of graphhopper (on windows using Docker-Toolbox):

  1. Use cd command to navigation into my Projects folder
  2. git clone https://github.com/graphhopper/graphhopper.git
  3. create an arizona_data folder, download arizona-latest.osm.pbf into that folder downloaded from here

I now have this folder structure

Project
 |
 +-- graphhopper (git cloned)
 |    
 +-- arizona_data
    |  
    +-- arizona-latest.osm.pbf (downloaded)
  1. Open graphhopper/Dockerfile and edit line 33 of the docker file:
  • From: COPY ./graphhopper.sh ./pom.xml ./config-example.yml ./
  • To: COPY ./graphhopper.sh ./pom.xml ./config.yml ./
  1. Open graphhopper/Dockerfile and edit line 41 of the docker file:
  • From: CMD [ "/data/europe_germany_berlin.pbf" ]
  • To: CMD [ "/data/north-america_us_arizona.pbf" ]
  1. Create a config.yml file by copy and pasting the config-example.yml file
  2. Open the config.yml file, and make the following edits, shown below
graphhopper:

  # OpenStreetMap input file PBF or XML, can be changed via command line -Ddw.graphhopper.datareader.file=some.pbf
  datareader.file: "../arizona_data/arizona-latest.osm.pbf"
  # Local folder used by graphhopper to store its data
  graph.location: graph-cache

  ##### Vehicles #####s


  # More options: foot,hike,bike,bike2,mtb,racingbike,motorcycle,car4wd,wheelchair (comma separated)
  # bike2 takes elevation data into account (like up-hill is slower than down-hill) and requires enabling graph.elevation.provider below.
  graph.flag_encoders: bike, bike2, mtb, racingbike

  # Enable turn restrictions for car or motorcycle.
  # graph.flag_encoders: car|turn_costs=true

  # Add additional information to every edge. Used for path details (#1548), better instructions (#1844) and tunnel/bridge interpolation (#798).
  # Default values are: road_class,road_class_link,road_environment,max_speed,road_access (since #1805)
  # More are: surface,max_width,max_height,max_weight,max_axle_load,max_length,hazmat,hazmat_tunnel,hazmat_water,toll,track_type
  # graph.encoded_values: surface,toll,track_type

  ##### Routing Profiles ####

  # Routing can be done for the following list of profiles. Note that it is required to specify all the profiles you
  # would like to use here. The fields of each profile are as follows:
  # - name (required): a unique string identifier for the profile
  # - vehicle (required): refers to the `graph.flag_encoders` used for this profile
  # - weighting (required): the weighting used for this profile, e.g. fastest,shortest or short_fastest
  # - turn_costs (true/false, default: false): whether or not turn restrictions should be applied for this profile.
  #   this will only work if the `graph.flag_encoders` for the given `vehicle` is configured with `|turn_costs=true`.
  #
  # Depending on the above fields there are other properties that can be used, e.g.
  # - distance_factor: 0.1 (can be used to fine tune the time/distance trade-off of short_fastest weighting)
  # - u_turn_costs: 60 (time-penalty for doing a u-turn in seconds (only possible when `turn_costs: true`)).
  #   Note that since the u-turn costs are given in seconds the weighting you use should also calculate the weight
  #   in seconds, so for example it does not work with shortest weighting.
  #
  # To prevent long running routing queries you should usually enable either speed or hybrid mode for all the given
  # profiles (see below). Otherwise you should at least limit the number of `routing.max_visited_nodes`.
  profiles:
    - name: bike
      vehicle: bike
      weighting: fastest

    - name: bike2
      vehicle: bike2
      weighting: fastest

    - name: mtb
      vehicle: mtb
      weighting: fastest

    - name: racingbike
      vehicle: racingbike
      weighting: fastest

  #  - name: car_with_turn_costs
  #    vehicle: car
  #    weighting: short_fastest
  #    distance_factor: 0.1
  #    turn_costs: true
  #    u_turn_costs: 60

  # Speed mode:
  # Its possible to speed up routing by doing a special graph preparation (Contraction Hierarchies, CH). This requires
  # more RAM/disk space for holding the prepared graph but also means less memory usage per request. Using the following
  # list you can define for which of the above routing profiles such preparation shall be performed. Note that to support
  # profiles with `turn_costs: true` a more elaborate preparation is required (longer preparation time and more memory
  # usage) and the routing will also be slower than with `turn_costs: false`.
  profiles_ch:
    - profile: bike
    - profile: bike2
    - profile: mtb
    - profile: racingbike
  #   - profile: car_with_turn_costs

  # Hybrid mode:
  # Similar to speed mode, the hybrid mode (Landmarks, LM) also speeds up routing by doing calculating auxiliary data
  # in advance. Its not as fast as speed mode, but more flexible.
  #
  # Advanced usage: It is possible to use the same preparation for multiple profiles which saves memory and preparation
  # time. To do this use e.g. `preparation_profile: my_other_profile` where `my_other_profile` is the name of another
  # profile for which an LM profile exists. Important: This only will give correct routing results if the weights
  # calculated for the profile are equal or larger (for every edge) than those calculated for the profile that was used
  # for the preparation (`my_other_profile`)
  profiles_lm: []

  ##### Elevation #####


  # To populate your graph with elevation data use SRTM, default is noop (no elevation). Read more about it in docs/core/elevation.md
  # graph.elevation.provider: srtm


  # default location for cache is /tmp/srtm
  # graph.elevation.cache_dir: ./srtmprovider/


  # If you have a slow disk or plenty of RAM change the default MMAP to:
  # graph.elevation.dataaccess: RAM_STORE



  #### Speed, hybrid and flexible mode ####


  # To make CH preparation faster for multiple profiles you can increase the default threads if you have enough RAM.
  # Change this setting only if you know what you are doing and if the default worked for you.
  # prepare.ch.threads: 1

  # To tune the performance vs. memory usage for the hybrid mode use
  # prepare.lm.landmarks: 16

  # Make landmark preparation parallel if you have enough RAM. Change this only if you know what you are doing and if
  # the default worked for you.
  # prepare.lm.threads: 1


  # avoid being stuck in a (oneway) subnetwork, see https://discuss.graphhopper.com/t/93
  prepare.min_network_size: 200
  prepare.min_one_way_network_size: 200



  ##### Routing #####


  # You can define the maximum visited nodes when routing. This may result in not found connections if there is no
  # connection between two points within the given visited nodes. The default is Integer.MAX_VALUE. Useful for flexibility mode
  # routing.max_visited_nodes: 1000000


  # If enabled, allows a user to run flexibility requests even if speed mode is enabled. Every request then has to include a hint ch.disable=true.
  # Attention, non-CH route calculations take way more time and resources, compared to CH routing.
  # A possible attacker might exploit this to slow down your service. Only enable it if you need it and with routing.maxVisitedNodes
  # routing.ch.disabling_allowed: true


  # If enabled, allows a user to run flexible mode requests even if the hybrid mode is enabled. Every such request then has to include a hint routing.lm.disable=true.
  # routing.lm.disabling_allowed: true

  # Control how many active landmarks are picked per default, this can improve query performance
  # routing.lm.active_landmarks: 4


  # You can limit the max distance between two consecutive waypoints of flexible routing requests to be less or equal
  # the given distance in meter. Default is set to 1000km.
  routing.non_ch.max_waypoint_distance: 1000000


  ##### Storage #####


  # configure the memory access, use RAM_STORE for well equipped servers (default and recommended)
  graph.dataaccess: RAM_STORE


  # will write way names in the preferred language (language code as defined in ISO 639-1 or ISO 639-2):
  # datareader.preferred_language: en


  # Sort the graph after import to make requests roughly ~10% faster. Note that this requires significantly more RAM on import.
  # graph.do_sort: true



  ##### Spatial Rules #####
  # Spatial Rules require some configuration and only work with the DataFlagEncoder.


  # Spatial Rules require you to provide Polygons in which the rules are enforced
  # The line below contains the default location for the files which define these borders
  # spatial_rules.borders_directory: core/files/spatialrules

  # You can define the maximum BBox for which spatial rules are loaded.
  # You might want to do this if you are only importing a small area and don't need rules for other countries.
  # Having less rules, might result in a smaller graph. The line below contains the world-wide bounding box, uncomment and adapt to your need.
  # spatial_rules.max_bbox: -180,180,-90,90


# Uncomment the following to point /maps to the source directory in the filesystem instead of
# the Java resource path. Helpful for development of the web client.
# Assumes that the web module is the working directory.
#
# assets:
#  overrides:
#    /maps: web/src/main/resources/assets/

# Dropwizard server configuration
server:
  application_connectors:
  - type: http
    port: 8989
    # for security reasons bind to localhost
    bind_host: localhost
  request_log:
      appenders: []
  admin_connectors:
  - type: http
    port: 8990
    bind_host: localhost
# See https://www.dropwizard.io/1.3.8/docs/manual/configuration.html#logging
logging:
  appenders:
  - type: file
    time_zone: UTC
    current_log_filename: logs/graphhopper.log
    log_format: "%d{YYYY-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
    archive: true
    archived_log_filename_pattern: ./logs/graphhopper-%d.log.gz
    archived_file_count: 30
    never_block: true
  - type: console
    time_zone: UTC
    log_format: "%d{YYYY-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
  1. Open Docker Quickstart Terminal (installed with Docker-Toolbox)
  2. In Docker terminal, use cd command to navigate to Projects/graphhopper
  3. Build the Docker Image from the Dockerfile by running docker build -t graphhopper:master . (Wait 5-10 minutes for it to complete)
  4. docker run --name graphhopper -v //C/Users/user/Projects/arizona_data:/data -p 8989:8989 graphhopper:master

Questions - Please Answer

  • Can I modify my the config.yml after I create the Docker Image? But before I I create the Docker Container?
    • If not, do I need to recreate a Docker Image every time I modify my config.yml file?
  • Was I suppose to modify the dockerfile?
  • If yes, did I correctly modify the dockerfile?
  • Did I correctly modify the config.yml file?
  • Can the Dockerfile read parent folders ?(
    • i.e. When I specify in the config.yml file datareader.file: "../arizona_data/arizona-latest.osm.pbf" Can docker read the ../ portion of the filepath?
  • Yes or No - Is the graph-cache folder that stores the graphs data is stored after graphhopper runs?
  • Where/when does the graph-cache suppose to show up?
    • Inside the Projects/graphhopper folder?
    • Inside the Docker Container itself?

For completeness

  1. My log file for docker build -t graphhopper:master .
$ docker build -t graphhopper:master .
Sending build context to Docker daemon  31.99MB
Step 1/19 : FROM maven:3.6.3-jdk-8 as build
 ---> c77985f27eaf
Step 2/19 : ENV JAVA_OPTS "-server -Xconcurrentio -Xmx1g -Xms1g -XX:+UseG1GC -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989"
 ---> Running in e171057cbb2d
Removing intermediate container e171057cbb2d
 ---> 9962955f6215
Step 3/19 : RUN apt-get install -y wget        && curl -sL https://deb.nodesource.com/setup_13.x | bash -        && apt-get install -y nodejs
 ---> Running in a6cc5922383e
Reading package lists...
Building dependency tree...
Reading state information...
wget is already the newest version (1.20.1-1.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

## Installing the NodeSource Node.js 13.x repo...


## Populating apt-get cache...

+ apt-get update
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://deb.debian.org/debian buster InRelease [122 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [49.3 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [187 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [7380 B]
Fetched 8337 kB in 10s (852 kB/s)
Reading package lists...

## Installing packages required for setup: lsb-release...

+ apt-get install -y lsb-release > /dev/null 2>&1

## Confirming "buster" is supported...

+ curl -sLf -o /dev/null 'https://deb.nodesource.com/node_13.x/dists/buster/Release'

## Adding the NodeSource signing key to your keyring...

+ curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK

## Creating apt sources list file for the NodeSource Node.js 13.x repo...

+ echo 'deb https://deb.nodesource.com/node_13.x buster main' > /etc/apt/sources.list.d/nodesource.list
+ echo 'deb-src https://deb.nodesource.com/node_13.x buster main' >> /etc/apt/sources.list.d/nodesource.list

## Running `apt-get update` for you...

+ apt-get update
Hit:1 http://security.debian.org/debian-security buster/updates InRelease
Hit:2 http://deb.debian.org/debian buster InRelease
Hit:3 http://deb.debian.org/debian buster-updates InRelease
Get:4 https://deb.nodesource.com/node_13.x buster InRelease [4584 B]
Get:5 https://deb.nodesource.com/node_13.x buster/main amd64 Packages [764 B]
Fetched 5348 B in 1s (6395 B/s)
Reading package lists...

## Run `sudo apt-get install -y nodejs` to install Node.js 13.x and npm
## You may also need development tools to build native addons:
     sudo apt-get install gcc g++ make
## To install the Yarn package manager, run:
     curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
     echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
     sudo apt-get update && sudo apt-get install yarn


Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  nodejs
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 24.3 MB of archives.
After this operation, 117 MB of additional disk space will be used.
Get:1 https://deb.nodesource.com/node_13.x buster/main amd64 nodejs amd64 13.13.0-1nodesource1 [24.3 MB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 24.3 MB in 8s (3087 kB/s)
Selecting previously unselected package nodejs.
(Reading database ... 13259 files and directories currently installed.)
Preparing to unpack .../nodejs_13.13.0-1nodesource1_amd64.deb ...
Unpacking nodejs (13.13.0-1nodesource1) ...
Setting up nodejs (13.13.0-1nodesource1) ...
Removing intermediate container a6cc5922383e
 ---> 52e953a58242
Step 4/19 : WORKDIR /graphhopper
 ---> Running in 890a33bf1e15
Removing intermediate container 890a33bf1e15
 ---> 8e1eca1bc004
Step 5/19 : COPY . .
 ---> 1af1fd42ed8c
Step 6/19 : WORKDIR /graphhopper/web
 ---> Running in e57444c18aa7
Removing intermediate container e57444c18aa7
 ---> b5fabd626f22
Step 7/19 : RUN npm install && npm run bundleProduction
 ---> Running in 6eb2be507c07
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated vector-tile@1.3.0: This module has moved: please install @mapbox/vector-tile instead
npm WARN deprecated fsevents@1.2.12: fsevents 1 will break on node v14+. Upgrade to fsevents 2 with massive improvements.
npm WARN deprecated point-geometry@0.0.0: This module has moved: please install @mapbox/point-geometry instead
npm WARN deprecated minimatch@0.3.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.12: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN graphhopper-js-ui@1.0.0 No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: abbrev@1.1.1 (node_modules/fsevents/node_modules/abbrev):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/abbrev' -> '/graphhopper/web/node_modules/fsevents/node_modules/.abbrev.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: ansi-regex@2.1.1 (node_modules/fsevents/node_modules/ansi-regex):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/ansi-regex' -> '/graphhopper/web/node_modules/fsevents/node_modules/.ansi-regex.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: aproba@1.2.0 (node_modules/fsevents/node_modules/aproba):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/aproba' -> '/graphhopper/web/node_modules/fsevents/node_modules/.aproba.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: balanced-match@1.0.0 (node_modules/fsevents/node_modules/balanced-match):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/balanced-match' -> '/graphhopper/web/node_modules/fsevents/node_modules/.balanced-match.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: chownr@1.1.4 (node_modules/fsevents/node_modules/chownr):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/chownr' -> '/graphhopper/web/node_modules/fsevents/node_modules/.chownr.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: code-point-at@1.1.0 (node_modules/fsevents/node_modules/code-point-at):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/code-point-at' -> '/graphhopper/web/node_modules/fsevents/node_modules/.code-point-at.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: concat-map@0.0.1 (node_modules/fsevents/node_modules/concat-map):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/concat-map' -> '/graphhopper/web/node_modules/fsevents/node_modules/.concat-map.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: console-control-strings@1.1.0 (node_modules/fsevents/node_modules/console-control-strings):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/console-control-strings' -> '/graphhopper/web/node_modules/fsevents/node_modules/.console-control-strings.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: core-util-is@1.0.2 (node_modules/fsevents/node_modules/core-util-is):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/core-util-is' -> '/graphhopper/web/node_modules/fsevents/node_modules/.core-util-is.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: deep-extend@0.6.0 (node_modules/fsevents/node_modules/deep-extend):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/deep-extend' -> '/graphhopper/web/node_modules/fsevents/node_modules/.deep-extend.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: delegates@1.0.0 (node_modules/fsevents/node_modules/delegates):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/delegates' -> '/graphhopper/web/node_modules/fsevents/node_modules/.delegates.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: detect-libc@1.0.3 (node_modules/fsevents/node_modules/detect-libc):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/detect-libc' -> '/graphhopper/web/node_modules/fsevents/node_modules/.detect-libc.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fs.realpath@1.0.0 (node_modules/fsevents/node_modules/fs.realpath):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/fs.realpath' -> '/graphhopper/web/node_modules/fsevents/node_modules/.fs.realpath.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: has-unicode@2.0.1 (node_modules/fsevents/node_modules/has-unicode):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/has-unicode' -> '/graphhopper/web/node_modules/fsevents/node_modules/.has-unicode.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: inherits@2.0.4 (node_modules/fsevents/node_modules/inherits):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/inherits' -> '/graphhopper/web/node_modules/fsevents/node_modules/.inherits.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: ini@1.3.5 (node_modules/fsevents/node_modules/ini):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/ini' -> '/graphhopper/web/node_modules/fsevents/node_modules/.ini.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: isarray@1.0.0 (node_modules/fsevents/node_modules/isarray):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/isarray' -> '/graphhopper/web/node_modules/fsevents/node_modules/.isarray.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: minimist@1.2.5 (node_modules/fsevents/node_modules/minimist):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/minimist' -> '/graphhopper/web/node_modules/fsevents/node_modules/.minimist.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: ms@2.1.2 (node_modules/fsevents/node_modules/ms):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/ms' -> '/graphhopper/web/node_modules/fsevents/node_modules/.ms.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm-normalize-package-bin@1.0.1 (node_modules/fsevents/node_modules/npm-normalize-package-bin):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/npm-normalize-package-bin' -> '/graphhopper/web/node_modules/fsevents/node_modules/.npm-normalize-package-bin.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: number-is-nan@1.0.1 (node_modules/fsevents/node_modules/number-is-nan):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/number-is-nan' -> '/graphhopper/web/node_modules/fsevents/node_modules/.number-is-nan.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: object-assign@4.1.1 (node_modules/fsevents/node_modules/object-assign):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/object-assign' -> '/graphhopper/web/node_modules/fsevents/node_modules/.object-assign.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: os-homedir@1.0.2 (node_modules/fsevents/node_modules/os-homedir):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: signal-exit@3.0.2 (node_modules/fsevents/node_modules/signal-exit):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/signal-exit' -> '/graphhopper/web/node_modules/fsevents/node_modules/.signal-exit.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: strip-json-comments@2.0.1 (node_modules/fsevents/node_modules/strip-json-comments):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/strip-json-comments' -> '/graphhopper/web/node_modules/fsevents/node_modules/.strip-json-comments.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: util-deprecate@1.0.2 (node_modules/fsevents/node_modules/util-deprecate):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/util-deprecate' -> '/graphhopper/web/node_modules/fsevents/node_modules/.util-deprecate.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: wrappy@1.0.2 (node_modules/fsevents/node_modules/wrappy):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/wrappy' -> '/graphhopper/web/node_modules/fsevents/node_modules/.wrappy.DELETE'
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: yallist@3.1.1 (node_modules/fsevents/node_modules/yallist):
npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '/graphhopper/web/node_modules/fsevents/node_modules/yallist' -> '/graphhopper/web/node_modules/fsevents/node_modules/.yallist.DELETE'

added 357 packages from 214 contributors and audited 5108 packages in 27.611s

2 packages are looking for funding
  run `npm fund` for details

found 2 vulnerabilities (1 low, 1 high)
  run `npm audit fix` to fix them, or `npm audit` for details

> graphhopper-js-ui@1.0.0 bundleProduction /graphhopper/web
> browserify -g uglifyify src/main/resources/assets/js/main-template.js -o src/main/resources/assets/js/main.js

Removing intermediate container 6eb2be507c07
 ---> 4585cf669f0f
Step 8/19 : WORKDIR /graphhopper
 ---> Running in 1cf70c50f6de
Removing intermediate container 1cf70c50f6de
 ---> de9ae5330a34
Step 9/19 : RUN ./graphhopper.sh build
 ---> Running in f5fcd3119990
/bin/sh: 1: ./graphhopper.sh: not found
The command '/bin/sh -c ./graphhopper.sh build' returned a non-zero code: 127

Dylan@Surface MINGW64 ~/5_RoutingEngines/graphhopper
$ docker build -t graphhopper:master .
Sending build context to Docker daemon     32MB
Step 1/19 : FROM maven:3.6.3-jdk-8 as build
 ---> c77985f27eaf
Step 2/19 : ENV JAVA_OPTS "-server -Xconcurrentio -Xmx1g -Xms1g -XX:+UseG1GC -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989"
 ---> Using cache
 ---> 9962955f6215
Step 3/19 : RUN apt-get install -y wget        && curl -sL https://deb.nodesource.com/setup_13.x | bash -        && apt-get install -y nodejs
 ---> Using cache
 ---> 52e953a58242
Step 4/19 : WORKDIR /graphhopper
 ---> Using cache
 ---> 8e1eca1bc004
Step 5/19 : COPY . .
 ---> cacdc0c3ac8e
Step 6/19 : WORKDIR /graphhopper/web
 ---> Running in dec776c1dc3e
Removing intermediate container dec776c1dc3e
 ---> 76b54e19538d
Step 7/19 : RUN npm install && npm run bundleProduction
 ---> Running in e1cfb3aa48cb
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated vector-tile@1.3.0: This module has moved: please install @mapbox/vector-tile instead
npm WARN deprecated point-geometry@0.0.0: This module has moved: please install @mapbox/point-geometry instead
npm WARN deprecated fsevents@1.2.12: fsevents 1 will break on node v14+. Upgrade to fsevents 2 with massive improvements.
npm WARN deprecated minimatch@0.3.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/chokidar/node_modules/fsevents):

added 357 packages from 214 contributors and audited 5108 packages in 22.96s

2 packages are looking for funding
  run `npm fund` for details

found 2 vulnerabilities (1 low, 1 high)
  run `npm audit fix` to fix them, or `npm audit` for details

> graphhopper-js-ui@1.0.0 bundleProduction /graphhopper/web
> browserify -g uglifyify src/main/resources/assets/js/main-template.js -o src/main/resources/assets/js/main.js

Removing intermediate container e1cfb3aa48cb
 ---> e24c5c5d18db
Step 8/19 : WORKDIR /graphhopper
 ---> Running in f5a99baff69d
Removing intermediate container f5a99baff69d
 ---> ee5cfe356482
Step 9/19 : RUN ./graphhopper.sh build
 ---> Running in 8d0ac63aea23
## using java 1.8.0_252 (64bit) from /usr/local/openjdk-8
copying non-default config file: config.yml
cp: 'config.yml' and 'config.yml' are the same file
## building graphhopper jar:
## using maven at /usr/share/maven
Removing intermediate container 8d0ac63aea23
 ---> d692f53c4bf7
Step 10/19 : FROM openjdk:11.0-jre
 ---> 786611991d70
Step 11/19 : ENV JAVA_OPTS "-Xmx1g -Xms1g -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989"
 ---> Running in 28966ec1fab2
Removing intermediate container 28966ec1fab2
 ---> dcdaf0ea6d26
Step 12/19 : RUN mkdir -p /data
 ---> Running in 8162c98a7490
Removing intermediate container 8162c98a7490
 ---> 70f7a47a90a6
Step 13/19 : WORKDIR /graphhopper
 ---> Running in 4b566f2272f2
Removing intermediate container 4b566f2272f2
 ---> 975341bf8d5a
Step 14/19 : COPY --from=build /graphhopper/web/target/*.jar ./web/target/
 ---> 694d232c4976
Step 15/19 : COPY ./graphhopper.sh ./pom.xml ./config.yml ./
 ---> bd492f0702d8
Step 16/19 : VOLUME [ "/data" ]
 ---> Running in c51ce6d1535b
Removing intermediate container c51ce6d1535b
 ---> 68d73d5a4e3d
Step 17/19 : EXPOSE 8989
 ---> Running in 2a2967c2c191
Removing intermediate container 2a2967c2c191
 ---> a1e19133432f
Step 18/19 : ENTRYPOINT [ "./graphhopper.sh", "web" ]
 ---> Running in 54fcc1319baf
Removing intermediate container 54fcc1319baf
 ---> 8822dbfa4b80
Step 19/19 : CMD [ "/data/north-america_us_arizona.pbf" ]
 ---> Running in 7c10d42b30fc
Removing intermediate container 7c10d42b30fc
 ---> 67a5351cf951
Successfully built 67a5351cf951
Successfully tagged graphhopper:master
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
  1. docker run --name graphhopper -v //C/Users/user/5_RoutingEngines/1_data:/data -p 8989:8989 graphhopper:master
docker run --name graphhopper -v //C/Users/user/5_RoutingEngines/1_data:/data -p 8989:8989 graphhopper:master
## using java 11.0.7 (64bit) from /usr/local/openjdk-11
copying non-default config file: config.yml
cp: 'config.yml' and 'config.yml' are the same file
File not found '/data/north-america_us_arizona.pbf'. Press ENTER to get it from: http://download.geofabrik.de/north-america/us/arizona-latest.osm.pbf
Press CTRL+C if you do not have enough disc space or you don't want to download several MB.
## now downloading OSM file from http://download.geofabrik.de/north-america/us/arizona-latest.osm.pbf and extracting to /data/north-america_us_arizona.pbf
  HTTP/1.1 200 OK
  Date: Tue, 21 Apr 2020 06:30:53 GMT
  Server: Apache/2.4.29 (Ubuntu)
  Last-Modified: Mon, 20 Apr 2020 23:20:47 GMT
  ETag: "88fb7a2-5a3c1261cf1c0"
  Accept-Ranges: bytes
  Content-Length: 143636386
  Content-Type: application/octet-stream
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
2020-04-21 06:36:13 URL:http://download.geofabrik.de/north-america/us/arizona-latest.osm.pbf [143636386/143636386] -> "/data/north-america_us_arizona.pbf" [1]
## existing jar found web/target/graphhopper-web-1.0-SNAPSHOT.jar
## now web. JAVA_OPTS=-Xmx1g -Xms1g -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989
2020-04-21 06:36:17.667 [main] INFO  i.d.server.DefaultServerFactory - Registering jersey handler with root path prefix: /
2020-04-21 06:36:17.675 [main] INFO  i.d.server.DefaultServerFactory - Registering admin handler with root path prefix: /
2020-04-21 06:36:18.952 [main] INFO  i.d.b.assets.ConfiguredAssetsBundle - Registering ConfiguredAssetBundle with name: assets for path /webjars/*
2020-04-21 06:36:18.954 [main] INFO  i.d.b.assets.ConfiguredAssetsBundle - Registering ConfiguredAssetBundle with name: assets for path /maps/*
2020-04-21 06:36:18.959 [main] WARN  i.d.jetty.setup.ServletEnvironment - Overriding the existing servlet registered with the name: assets
2020-04-21 06:36:19.065 [main] INFO  io.dropwizard.server.ServerFactory - Starting GraphHopperApplication
                        _     _
   __ _ _ __ __ _ _ __ | |__ | |__   ___  _ __  _ __   ___ _ __
  / _` | '__/ _` | '_ \| '_ \| '_ \ / _ \| '_ \| '_ \ / _ \ '__|
 | (_| | | | (_| | |_) | | | | | | | (_) | |_) | |_) |  __/ |
  \__, |_|  \__,_| .__/|_| |_|_| |_|\___/| .__/| .__/ \___|_|
  |___/          |_|                     |_|   |_|


... DELETED LINES - NEEDED TO SAVE ON CHARACTER LIMIT ...

2020-04-21 06:40:49.351 [main] INFO  o.e.j.server.handler.ContextHandler - Started i.d.j.MutableServletContextHandler@586cc15d{/,null,AVAILABLE}
2020-04-21 06:40:49.381 [main] INFO  o.e.jetty.server.AbstractConnector - Started application@4c7e978c{HTTP/1.1, (http/1.1)}{0.0.0.0:8989}
2020-04-21 06:40:49.385 [main] INFO  o.e.jetty.server.AbstractConnector - Started admin@354e7004{HTTP/1.1, (http/1.1)}{localhost:8990}
2020-04-21 06:40:49.387 [main] INFO  org.eclipse.jetty.server.Server - Started @275800ms

Not sure if I can help since I never used GH with Docker (btw why do you want to use Docker?). But few things that might help you:

  • GH reads the OSM file (arizona-latest.osm.pbf) and then stores its internal data structures in the folder referenced as graph.location in config.yml. Sometimes this folder is also referred to as the graph-cache. After this import has been done it will not run again when you restart the GH server, so (most) changes you do in config.yml after this will either be ignored or yield an error message. For example you cannot add another vehicle like this but you could change for example routing.ch.disabling_allowed. However, as a general rule of thumb lets just say after the first import you do not change config.yml again. You can of course delete the graph-cache folder, then change config.yml and do the import again.

  • Docker does not store anything ‘inside’ the container. If you shut down docker and restart it again it will be in the same state as you started it the first time. It does not store anything. But Docker can for example use a ‘volume’, which is a dedicated folder on your hard-disk and it can read/write data from/to it. This should be used to store the graph-cache data and the location of this folder needs to be configured with Docker. This way its visible for Docker and you can also use it like any other folder with your file manager.

Hey There,

You will be able to change the config file once your image is built and the service is running. My preference is using docker-compose and defining the services. then you can run docker-compose up --build -d to run the service in the background.

Once every thing is running, you can check the running containers with either docker ps -a or docker container ls -a to get the container name or id.

Next, you can run the terminal command: docker exec -it <id|container_name> bash

From there you can navigate to your file, or if you’re not sure where it would be (most likely the working dir you set in the Dockerfile), type where config.yml to bring up any files matching that filename.

Using Vim/Nano, you can make your changes to the file. Once those are made, and you are exited out of the container, you can copy the changes to your local directory via docker cp <id|container_name>:path/to/file path/to/local

You can also make Changes to the config.yml prior to building and using the COPY command in your Dockerfile to copy it to your container working directory.

Hope this helps!

Best,

Matt

@Matt_Frey Thanks Matt, yes this does help! I’m trying to mentally glue together my understanding. I think I’m getting there, but yes, while quite powerful, Docker is not intuitive. So much appreciate from over here!

You will be able to change the config file once your image is built and the service is running. My preference is using docker-compose and defining the services. then you can run docker-compose up --build -d to run the service in the background.

Okay, I think that makes sense about the docker-compose.yml file! What I’m mainly confused with is: If I want to modify my Docker-Container, Do I need to modify my config.yml file, then regenerate the Docker-Image, then regenerate the Docker-Container? Or can I “step inside” the Docker-Image with the docker exec command as discussed, and modify the config.yml in there, and generate a second Docker-Container from that newly Modified Image? (Maybe that’s a strange question, but conceptually it’s helping understand how these things files, relate to the images, and relate to the containers).

And one final question regarding the Docker graphhopper command:
docker run -d --name graphhopper -v <path_data_directory>/data:/data -p 8989:8989 graphhopper:master

When you copy the local <path_data_directory>/data into a Docker-Volume at /data does that /data folder “live” inside the Image? or Container? or some third thing? I don’t really understand the use case for Volumes, and not just having the data stored locally, or in a server… Why is the data “compartmentalized” somewhere in Docker like that?

If this is too much to answer, don’t worry about it, either way I’m going to keep sleuthing through Youtube Videos! Haha

@Dylansc I agree, docker is not very intuitive and their docs could use an overhaul haha.

Like you mentioned, You could step into the container and edit it to the way you want it to be, or you do it before building the container. If you edit it inside the container, you’ll want to use the docker cp command to copy it out of the container and then rebuild.

Another option, would be to put the config in a directory and share it as a volume. Then you can make your changes and they will persist. Also, updating the path to the config in your dockerfile or docker command so that the build can locate it.

Volumes/-v
volume or -v lets you have an open communication (or shared directory) with the VM docker container and your local PC. So when you make updates in your local pc to that drive/directory, you can see the changes in both container and pc.

So if you were to add files into a directory you added as a volume, you could see the files in the docker container too. Also, if you make changes to files in the container, the changes will persist.

Powered by Discourse