Elevation data from local files

Open Source Routing Engine
17

Since SRTM does support more than one resolution (new SRTM is 1 arc while old is 3 arc) I think the code should be in SRTMProvider.

Looking at the code you are setting the width of the tile to be 1201 samples.
My assumptions is that the data is in a square format (meaning there will be same number of smaples both for height and width).
Using this assumption I can read the total number of bytes in the file, divide it by 2 in order to get the total number of samples and use sqrt to get the number of samples for both dimentions.

This is the relevant line of code:
int samples = (int)(Math.Sqrt(byteArray.Length / 2) + 0.5);
instead of using WIDTH = 1201

I think that all you need to do in the code is create the instance of HeightTile after you unzip the file so that you know how many bytes are in the file.

Again, I would love to do it but I think that setting up the environment will require too much effort from my side.

Do you want me to create an issue in github so I’ll be able to test it once it’s implemented?

18

Thanks, would you try if such a code change works for you? You just need to modify the sources, call ./graphhopper.sh clean and it’ll use the modification.

Again, I would love to do it but I think that setting up the environment will require too much effort from my side.

it should not be that complex :slight_smile:

Yes, feel free to create an issue for this

19

Took me 3 hours to set everything up, not sure it was really worth it…
Never mind me, just blowing off some steams.
I have change the code and ran the tests to make sure I didn’t broke any thing.
Unfortunately it looks like the DataAccess object that is used to keep the heights does not tell me the number of bytes in it - which I need in order to be resolution independent.
Should I be using a different data structure or save the total size/number of samples in the first byte?
Not sure how to continue here… I’m also not that experienced with Java judging from the google time I’ve just spent…
Also not sure that what I did is backwards compatible but you’ll have to review it anyway before you pull it…

20

The DataAccess concrete class is MMAPDataAccess, I was hoping to get the number of bytes from the capacity but since the segments size is a power of two the capacity is bigger than the original buffer that was used.
Any thoughts?

21

I can use the header, right? to place the size of the byte array.
I’ll submit a pull request tonight and you’ll let me know if what I did was ok or not…

22

Hey,

yes, the header seems a good solution for this

Also not sure that what I did is backwards compatible but you’ll have to review it anyway before you pull it…

It is not important for now, also when using the header and there is no we can just use 1201

Took me 3 hours to set everything up, not sure it was really worth it…

That is ugly, we need to improve this. What exactly took you so long? Maybe you should have choosen the IDE way e.g. via NetBeans, where you normally only need to open the project, instead of the command line fiddling?

Also it is really important that you do not see this as waste of time, I hope, as many changes are only possible via a good and vibrant community :slight_smile:

So I really appreciate any inputs and improvements on all sides!

I have change the code and ran the tests to make sure I didn’t broke any thing.

Yes, in case your changes break things, do not worry, then our tests coverage is broken and needs to be fixed!

23

Yes trying with Eclipse and Intellij it works by just opening the root pom.xml file.

Emux

1 Like
25

I chose NetBeans, most of the time was to install it and other tools. The only real problem was with Maven, which I managed to solve quite fast.
Are you planning a release soon? I would prefer to use a version built by you guys and not locally if possible.
I managed to change the code and the tests pass this morning :slight_smile: will push it tonight.
Your project and responsiveness are amazing, keep up the good work!

26

Ok, the maven problem you have is something which relates to the command line script but this is unrelated to NetBeans as NetBeans has it embeded. So you can also run the web UI via NetBeans:
Press F6, choose GHServer and specify config=../config.properties osmreader.osm=youosm.pbf as command line

Are you planning a release soon? I would prefer to use a version built by you guys and not locally if possible.

Not in this month. But we release snapshots over here. See the dev docs how you include a pom.xml snippet to specify

Also do not fear to run master in production (although I do not recommend it) but we also run it for our Directions API and if something critical is broken we have to fix it VERY fast :slight_smile: ! And we make sure nothing hopefully breaks via unit, integration, performance and manual tests

Your project and responsiveness are amazing, keep up the good work!

Thanks for the kind words!

27

Done: https://github.com/graphhopper/graphhopper/pull/545
Enjoy!

1 Like
28

Hi Karussell

Harel referred me here, maybe I can help you with the aster elevation data

what kind of problems do you have?

Ohad

29

In case we get an agreement for Aster data - would you be able to ‘clean’ this elevation data for the whole world? Or was your effort manually or specific to the one from Harel?

And would this always give better results compared to CGIAR what we currently have?

30

it will be interesting to try.

We actually extract the relevant elevation data from it and not actually clean it.

The result are not always better than CGIAR, and sometimes we have to “merge” data from there.

but first do you familiar with this data: ftp://topex.ucsd.edu/pub/ ?

Ohad

31

We actually extract the relevant elevation data from it and not actually clean it.

What rough procedure did you try to do that?

The result are not always better than CGIAR, and sometimes we have to “merge” data from there.

okay

but first do you familiar with this data: ftp://topex.ucsd.edu/pub/ ?

No. Looks like many data sources including the new srtm (?) Can you explain a bit about this and under which license is this? E.g. srtm plus seems also to need not only attribution but also an agreement (for commercial usage)

32

We order the data in ascii format, than rectifying the coordinate and than extract the Z value from it.

I never check if the data has any licensing, I got that link in the astr forum as a download referance. I know that the site belong to the us goverment but let me check it tommarow.

33

Thanks, I tried today to search a bit around but it is all a bit confusing. To include this in GraphHopper for all (be it in tests or in source) I have to make the licensing clear, e.g. if allowed for commercial usage or not and what kind of attribution.

34

I notice that in the ftp://topex.ucsd.edu/pub/srtm15_plus/SRTM_commercial_permission_letter.doc document there is all you need for the licensing, if you don’t use for commercial use it’s fine to use.

I think it is the same as astr licensing.

Ohad

35

This is a problem, then we cannot include it nor use it. Additionally one would have to put an attribution everywhere which is not possible as we e.g. distribute GH on maven and elsewhere: I hereby agree to include attribution as described above in websites, products, and publications that contain SRTM30_PLUS Global Bathymetry content

36

I’m not fully understand the needs, in what format do you use to calculate the sloops?

if the data is good enough for you I can supply it from astr data (I think they convert it in the same procedure).

Ohad

37

Thanks for your support. Currently we need to solve the licensing problem, be it for these data or for ASTER.

The improved elevation data that you refer to, is probably better compared to CGIAR which is better than normal SRTM, at least that was the impression from @Harel . So it would be interesting but firstly we need to solve or figure out the licensing stuff.