Continued Discussion of #1163
My last post:
We can discuss here
thanks for posting here :).
I think your proposed solution sounds very interesting and in general I would like to learn more (or maybe test it, if it’s already runnable?). I think @karussell is right, we should set this up decoupled from GraphHopper and make it even possible for other projects to use it as well?
okay I didn’t set anything up yet. I just read your issue and your discussion with @karussell and Jenkins fits very well in there. You can think of jenkins as a selfhosted alternative to travis It works with github webhooks, it has build queues, you can set up multiple slaves with multiple OS for testing, but you have to set it up yourself.
If you want to try jenkins, just install the jenkins docker image here and create a new freestyle job
There are two different jobtypes at the moment:
The hardest or more intense part would be to set up the log file analyses. As I can see here you have something like that already, so you could reuse. I could imagine a comparison to master branch performance and a report back into the pull request of jenkins with comparisons.
To isolate noise and it depends on the jenkins workload, you can rerun a test X times, create a median or something like that to just eliminate noise. Just some sorts for later here
All in all jenkins just is a tool to run the command you gave it like a shell script, but the infrastructure behind it supports all kind of things and also there is a big plugin support. If you need some special things, there is a big possibility that there is already a jenkins plugin for it.
If I can help you with something, just ask
thanks for the explanation, it’s nice to see that jenkins evolved, I haven’t heard about the freestyle jobs so far.
I used jenkins in a couple of projects several years ago.
It sounded like you already had a setup running and already a tweaked config for this.
no sorry. If I have time, I can start to work on some things in my freetime, but I just hasn’t created anything yet. After reading your discussion, it was just a quick thought and I just wanted to charge my thoughts.
How are the measurements are done right now?
I also don’t have the server capacity right now at home or as a vps to do something like that, but for testing I can create a docker on my machine.
you can have a look at the Measurement class and also at
graphhopper.sh. Currently you can run the measurements by executing
graphhopper.sh measurement osm.pbf. The result of the measurement is a log file containing numbers for the different cases that have been run.
I think we can host this if it’s in a runnable state.
Help for this issue would be definitely appreciated, if you happen to have some time and interest in doing it :).