Insert traffic data into graphhopper

Hi, I’m having some trouble assigning traffic data into the edges of the graph and a couple of questions. Two things that I wish to accomplish is alter speed values of a specific edge and completely block an edge on the fly.

For the first case I followed the demo published here https://github.com/karussell/graphhopper-traffic-data-integration. Most of the functionality works but periodically. The data is being set manually by getting the coordinates of a street from Google Maps and assigning it a speed value.

My problem was that sometimes a point will not influence the route even if it’s on the road being taken. I then came across that the problem is that maybe it is being assigned to another edge, so I should use map matching. Wanted to confirm this?

Another problem was when I set a really low speed like 3 or lower where it will throw this error:
java.lang.IllegalStateException: Calculating time should not require to read speed from edge in wrong direction. Reverse:false, fwd:false, bad:false

I found that when trying to block a road so that no vehicle would consider it, like an accident, by setting speed 0.

Final mention is that I have to use Google Maps for my school project so I’m not sure if map matching is the right option?

My questions are:

  • Is the map matching approach the correct one even though the update is done every few minutes?
  • Why is the speed reading exception thrown on really low speed values?
  • What is the best practice to completely block roads on the fly?

Thanks you for any help.

Hard to judge from the distance. Are you blocking in the correct direction? As the edges in the graph are bidirectional.

Another problem was when I set a really low speed like 3 or lower where it will throw this error

This indicates a bug, but it is strange that both directions are blocked i.e. false → fwd and bwd (not bad ;))

Final mention is that I have to use Google Maps

What do you mean here? BTW: Using/extracting data from there is not allowed IMO

Is the map matching approach the correct one even though the update is done every few minutes?

It depends on the data you have. If you have just points you cannot use map matching. If you have lines you can improve the matching through map matching, yes.

What is the best practice to completely block roads on the fly?

If you block a road, instead of reducing speed, try setting the access properties explicitly to false. This is hopefully a good workaround for this bug. Also you can try to upgrade the demo to the latest GH version, where this issue should be fixed (e.g. here or here)

As advised I updated my graphhopper version to the latest one.

Hard to judge from the distance. Are you blocking in the correct direction? As the edges in the graph are bidirectional.

I’m using the points that give the data point for each one of them I get the closest edge with QueryResult and that is the one whose speed value I set. How can I confirm it is the right direction, and/or enforce it?

This indicates a bug, but it is strange that both directions are blocked i.e. false → fwd and bwd (not bad :wink:)

Sorry, autocorrect
Is there a way currently around this bug, I’m still getting it?

It depends on the data you have. If you have just points you cannot use map matching. If you have lines you can improve the matching through map matching, yes.

What I’m trying to accomplish with traffic points is that if I place a marker with a speed value on the map, its coordinates will get the edge under it and modify its speed value. But it still fails sometimes to get the right edge, at least that is my theory why it’s not always working. Any suggestions?

About Google Maps, I meant that I’m using it as the UI for my project but on the backend I’m using graphhopper and sprit.

Update:
Tried to use set access on the road I tried to block but it gives me this same error.
java.lang.IllegalStateException: Calculating time should not require to read speed from edge in wrong direction. Reverse:false, fwd:false, bwd:false

See recent discussion here

There should be no bug in a recent version >= 0.7. If it still happens (your update seems to indicate this) it would be nice to have a small unit test reproducing the issue. A small speed value will block the access of the road too and therefore avoid this issue. To make it possible that e.g. 1km/h is not rounded downwards to 0, you will have to increase the resolution of the speed value e.g. do not pass a factor of 5 but use just 1 and increase the bits to lets say 7 and not 5. (done in the CarFlagEncoder constructor)

Again: without code and the actual problem I cannot give you hints what is incorrect. But yes, it should work :slight_smile:

I managed to make work the traffic most of the time. The problem now is that GH is incrementing correctly the time that the path should take, but even with the heavily increased time, it still decides to take the same path as before.

The configuration I am currently trying to use is:

Config

graph.flag_encoders=car|turn_costs=true

graph.dataaccess=RAM_STORE

# currently required for traffic data, otherwise you would need to regularly re-import
prepare.ch.weightings=no

Java

public final class CZRoutingService {

    private static CZRoutingService instance;

    @Getter
    private final GraphHopper hopper;
    private final ReadWriteLock lock = new ReentrantReadWriteLock();
    private CZRoutingDataUpdater dataUpdater;

    private CZRoutingService() {
        CmdArgs args;
        try {
            args = CmdArgs.readFromConfig("config.properties", "graphhopper.config");
        } catch (Exception e) {
            args = new CmdArgs();
        }
        args.put("osmreader.osm", "input/NewYork.osm");
        args.put("graph.location", "output/graph.location");

        this.hopper = new GraphHopper() {

            @Override
            public GHResponse route(GHRequest request) {
                lock.readLock().lock();
                try {
                    return super.route(request);
                } finally {
                    lock.readLock().unlock();
                }
            }

        }
                .forServer()
                .init(args)
//            .setOSMFile("input/planet.osm.pbf")
//            .setGraphHopperLocation("output/graph.location")
//            .setEncodingManager(new EncodingManager("car"))


            ;
        this.hopper.getCHFactoryDecorator().setWeightingsAsStrings("fastest");
        this.dataUpdater = new CZRoutingDataUpdater(this.hopper, lock.writeLock());
    }

    public void run() {
        this.hopper.importOrLoad();
        this.dataUpdater.start();
    }

    public static CZRoutingService getInstance() {
        if (instance == null) {
            synchronized (CZRoutingService.class) {
                if (instance == null) {
                    instance = new CZRoutingService();
                }
            }
        }
        return instance;
    }
}

public class CZRoutingDataUpdater {

    private final GraphHopper hopper;
    private final Lock writeLock;
    private final long seconds = 150;
    private CZRoutingData currentRoads;

    public CZRoutingDataUpdater( GraphHopper hopper, Lock writeLock ) {
        this.hopper = hopper;
        this.writeLock = writeLock;
    }

    public void feed( CZRoutingData data ) {
        writeLock.lock();
        try {
            lockedFeed(data);
        } finally {
            writeLock.unlock();
        }
    }

    private void lockedFeed( CZRoutingData data ) {
        currentRoads = data;
        Graph graph = hopper.getGraphHopperStorage().getBaseGraph();
        FlagEncoder carEncoder = hopper.getEncodingManager().getEncoder("car");
        LocationIndex locationIndex = hopper.getLocationIndex();

        int errors = 0;
        int updates = 0;
        TIntHashSet edgeIds = new TIntHashSet(data.size());
        System.out.println(data.size());
        for (CZRoutingEntry entry : data) {
            System.out.println(entry);

            // TODO get more than one point -> our map matching component
            CZCoordinate point = entry.getPoints().get(entry.getPoints().size() / 2);
            QueryResult qr = locationIndex.findClosest(point.getLatitude(), point.getLongitude(), EdgeFilter.ALL_EDGES);
            if (!qr.isValid()) {
                // logger.info("no matching road found for entry " + entry.getId() + " at " + point);
                errors++;
                continue;
            }

            int edgeId = qr.getClosestEdge().getEdge();
            if (edgeIds.contains(edgeId)) {
                // TODO this wouldn't happen with our map matching component
                errors++;
                continue;
            }

            edgeIds.add(edgeId);
            EdgeIteratorState edge = graph.getEdgeIteratorState(edgeId, Integer.MIN_VALUE);
            double value = entry.getValue();

            if ("block".equalsIgnoreCase(entry.getValueType())) {
                edge.setFlags(carEncoder.setAccess(edge.getFlags(), false, false));
                continue;
            }
            if ("replace".equalsIgnoreCase(entry.getMode())) {
                if ("speed".equalsIgnoreCase(entry.getValueType())) {
                    double oldSpeed = carEncoder.getSpeed(edge.getFlags());
                    if (oldSpeed != value) {
                        updates++;
                        // TODO use different speed for the different directions (see e.g. Bike2WeightFlagEncoder)
                        System.out.println("Speed change at " + entry.getId() + " (" + point + "). Old: " + oldSpeed + ", new:" + value);
                        edge.setFlags(carEncoder.setSpeed(edge.getFlags(), value));
                    }
                } else {
                    throw new IllegalStateException("currently no other value type than 'speed' is supported");
                }
            } else {
                throw new IllegalStateException("currently no other mode than 'replace' is supported");
            }
        }

        System.out.println("Updated " + updates + " street elements of " + data.size() + ". Unchanged:" + (data.size() - updates) + ", errors:" + errors);
    }

    protected String fetchJSONString(String url) throws UnirestException {
        String string = Unirest.post(url)
                .header("accept", "application/json").asString().getBody();
        return string;
    }

    public CZRoutingData fetch(String url) throws UnirestException {
        JSONArray arr = new JSONArray(fetchJSONString(url));
        CZRoutingData data = new CZRoutingData();

        for (int i = 0; i < arr.length(); i++) {
            JSONObject obj = arr.getJSONObject(i);
            double speed = obj.getDouble("speed");
            String idStr = obj.getString("road_entry_id");

            JSONArray paths = obj.getJSONArray("points");
            List<CZCoordinate> points = new ArrayList<CZCoordinate>();
            for (int pointIndex = 0; pointIndex < paths.length(); pointIndex++) {
                JSONObject point = paths.getJSONObject(pointIndex);
                points.add(new CZCoordinate(point.getDouble("latitude"), point.getDouble("longitude")));
            }

            if (!points.isEmpty()) {
                data.add(new CZRoutingEntry(idStr + "_", speed, "speed", "replace", points));
            }
        }
//        List<CZCoordinate> points = new ArrayList();
//        points.add(new CZCoordinate(40.752603, -73.985755));
//        CZRoutingEntry lowSpeedEntry = new CZRoutingEntry("some_id", 1, "speed", "replace",points);
//        data.add(lowSpeedEntry);

//        List<CZCoordinate> points = new ArrayList();
//        points.add(new CZCoordinate(40.752603, -73.985755));
//        CZRoutingEntry blockEntry = new CZRoutingEntry("block", 0, "block", "",points);
//        data.add(blockEntry);

        return data;
    }

    private final AtomicBoolean running = new AtomicBoolean(false);

    public void start() {
        if (running.get()) {
            return;
        }

        running.set(true);
        new Thread("DataUpdater" + seconds) {
            @Override
            public void run() {
                System.out.println("fetch new data every " + seconds + " seconds");
                while (running.get()) {
                    try {
                        System.out.println("fetch new data");
                        CZRoutingData data = fetch("http://localhost:3000/traffic");
                        feed(data);
                        try {
                            Thread.sleep(seconds * 1000);
                        } catch (InterruptedException ex) {
                            System.out.println("update thread stopped");
                            break;
                        }
                    } catch (Exception ex) {
                        System.out.println(ex);
                        System.out.println("Problem while fetching data");
                    }
                }
            }
        }.start();
    }

    public void stop() {
        running.set(false);
    }

    public CZRoutingData getAll() {
        if (currentRoads == null) {
            return new CZRoutingData();
        }

        return currentRoads;
    }
}

Maven

<dependency>
    <groupId>com.graphhopper</groupId>
    <artifactId>graphhopper</artifactId>
    <version>0.8-SNAPSHOT</version>
</dependency>

<repository>
    <id>sonatype-oss-public</id>
    <url>https://oss.sonatype.org/content/groups/public/</url>
    <releases>
        <enabled>true</enabled>
    </releases>
    <snapshots>
        <enabled>true</enabled>
    </snapshots>
</repository>

The osm file I’m using is of New York City from http://download.bbbike.org/osm/bbbike/NewYork/

The error only pops up if the path crosses through that specific edge. In the fetch function, if I enable any of the commented entries it throws.

Sorry, I tried the last minutes to get it working but there is too much fiddelling with non existing classes and dependencies to make it working. Can you please create a small unit test (or integration test using new york) using just GraphHopper classes reproducing the issue?