Readonly DataAccess for InputStreams instead of Files

Developers
1

For our routing runtime we wanted to load the graph data from an URL / Inputstream rather than file. (Graph data was on external server and boot time already long enough).

I want to share the implementation with you, maybe it is of interest for others. It also uses off-heap storage via lxarray but you can combine it with other storage techniques, of course: https://gist.github.com/jtheuer/a709ef97aad4eb63bc87c57744cccc19

It is a bit reverse-engineered to remove the random access logic but works without a problem in production for us.

The reading part might be interesting if you want to support any URL as graph folder.
I noticed for instance that the graphhopper unit tests that load graphs only work correctly if the workdir is set up correctly in my IntelliJ project. Since the test data is in the classpath, an URL could be used instead of a File. It would then work always and on all systems.

HTH,

Jan

2 Likes
2

Thanks a lot! So you can stream the data directly into off-heap memory … interesting :slight_smile: !

Do you have a maven snippet how to include LByteArray or where to get it?

ups, already found it: https://github.com/xerial/larray