Thanks for the help. Both planet imports (Xmx=45G and Xmx=80g) completed successfully in about 35 hours. I think the performance discrepancy is just due to slow cpus on linode/digitalocean. The output was ~23GB.
I tested importing a smaller 1GB osm.pbf extract on 3 vps’s to compare performance:
- digitalocean “optimized” droplet (8GB RAM, 4 dedicated vCPUs) finished in 34m
- digitalocean “regular” droplet (8GB RAM ,4 shared vCPUs) finished in 58m
- linode memory-optimized droplet finished in 64m
So I think if I ran the planet import on an optimized droplet, the import time would be closer to ~20 hours you get on bare metal. I’d be curious if there’s a way to parallelize work on PrepareContractionHierarchies.contractNodes() to take advantage of more CPUs though.
Just to clarify before I burn too much more money running tests…
- if the foot profile finished with 45GB RAM, how much more would you expect 3 profiles to use with parallel vs sequential contraction hierarchy generation?
- What part of the import process do datareader.dataaccess=MMAP and graph.dataacess reduce memory requirements for? Does the entire dataset need to get loaded into RAM at some point regardless of MMAP/RAM_STORE?