Failure to read PBF at startup, not supporting MESSAGE_INFO

Open Source Routing Engine Map Matching
1

I am trying to use graphhopper map matching in my client application but it fails at import of the pbf file. Stacktrace is below. I have tracked it down into some generated class in osmosisbin(Fileformat.java) that does not seem to support dynamicMethod(MethodToInvoke.BUILD_MESSAGE_INFO);

Exception in thread "PBF Reader" java.lang.RuntimeException: Unable to read PBF file.
at com.graphhopper.reader.osm.pbf.PbfReader.run(PbfReader.java:50)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: Unable to get message info for org.openstreetmap.osmosis.osmbinary.Fileformat$BlobHeader
at com.google.protobuf.GeneratedMessageInfoFactory.messageInfoFor(GeneratedMessageInfoFactory.java:62)
at com.google.protobuf.ManifestSchemaFactory$CompositeMessageInfoFactory.messageInfoFor(ManifestSchemaFactory.java:143)
at com.google.protobuf.ManifestSchemaFactory.createSchema(ManifestSchemaFactory.java:55)
at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:93)
at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:107)
at com.google.protobuf.GeneratedMessageLite.parsePartialFrom(GeneratedMessageLite.java:1554)
at com.google.protobuf.GeneratedMessageLite.parseFrom(GeneratedMessageLite.java:1654)
at org.openstreetmap.osmosis.osmbinary.Fileformat$BlobHeader.parseFrom(Fileformat.java:1127)
at com.graphhopper.reader.osm.pbf.PbfStreamSplitter.readHeader(PbfStreamSplitter.java:43)
at com.graphhopper.reader.osm.pbf.PbfStreamSplitter.getNextBlob(PbfStreamSplitter.java:72)
at com.graphhopper.reader.osm.pbf.PbfStreamSplitter.hasNext(PbfStreamSplitter.java:89)
at com.graphhopper.reader.osm.pbf.PbfDecoder.processBlobs(PbfDecoder.java:106)
at com.graphhopper.reader.osm.pbf.PbfDecoder.run(PbfDecoder.java:162)
at com.graphhopper.reader.osm.pbf.PbfReader.run(PbfReader.java:47)
... 1 more
Caused by: java.lang.UnsupportedOperationException
at org.openstreetmap.osmosis.osmbinary.Fileformat$BlobHeader.dynamicMethod(Fileformat.java:1430)
at com.google.protobuf.GeneratedMessageLite.dynamicMethod(GeneratedMessageLite.java:251)
at com.google.protobuf.GeneratedMessageLite.buildMessageInfo(GeneratedMessageLite.java:279)
at com.google.protobuf.GeneratedMessageInfoFactory.messageInfoFor(GeneratedMessageInfoFactory.java:60)
... 14 more
2020-04-21 13:21:05,459 INFO  [main] OSMReader: creating graph. Found nodes (pillar+tower):0, totalMB:502, usedMB:19
Exception in thread "PBF Reader" java.lang.RuntimeException: Unable to read PBF file.
at com.graphhopper.reader.osm.pbf.PbfReader.run(PbfReader.java:50)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: Unable to get message info for org.openstreetmap.osmosis.osmbinary.Fileformat$BlobHeader
at com.google.protobuf.GeneratedMessageInfoFactory.messageInfoFor(GeneratedMessageInfoFactory.java:62)
at com.google.protobuf.ManifestSchemaFactory$CompositeMessageInfoFactory.messageInfoFor(ManifestSchemaFactory.java:143)
at com.google.protobuf.ManifestSchemaFactory.createSchema(ManifestSchemaFactory.java:55)
at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:93)
at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:107)
at com.google.protobuf.GeneratedMessageLite.parsePartialFrom(GeneratedMessageLite.java:1554)
at com.google.protobuf.GeneratedMessageLite.parseFrom(GeneratedMessageLite.java:1654)
at org.openstreetmap.osmosis.osmbinary.Fileformat$BlobHeader.parseFrom(Fileformat.java:1127)
at com.graphhopper.reader.osm.pbf.PbfStreamSplitter.readHeader(PbfStreamSplitter.java:43)
at com.graphhopper.reader.osm.pbf.PbfStreamSplitter.getNextBlob(PbfStreamSplitter.java:72)
at com.graphhopper.reader.osm.pbf.PbfStreamSplitter.hasNext(PbfStreamSplitter.java:89)
at com.graphhopper.reader.osm.pbf.PbfDecoder.processBlobs(PbfDecoder.java:106)
at com.graphhopper.reader.osm.pbf.PbfDecoder.run(PbfDecoder.java:162)
at com.graphhopper.reader.osm.pbf.PbfReader.run(PbfReader.java:47)
... 1 more
Caused by: java.lang.UnsupportedOperationException
at org.openstreetmap.osmosis.osmbinary.Fileformat$BlobHeader.dynamicMethod(Fileformat.java:1430)
at com.google.protobuf.GeneratedMessageLite.dynamicMethod(GeneratedMessageLite.java:251)
at com.google.protobuf.GeneratedMessageLite.buildMessageInfo(GeneratedMessageLite.java:279)
at com.google.protobuf.GeneratedMessageInfoFactory.messageInfoFor(GeneratedMessageInfoFactory.java:60)
... 14 more
2

Where did you source your Pbf file from?
Can you provide a link for review?

3

Yes, it’s this one albeit some weeks old: https://download.geofabrik.de/europe/monaco-latest.osm.pbf

4

And my loading code looks like this:

File ghDirectory  = new File(graphHopperFolder);
if(ghDirectory.exists() && ghDirectory.isDirectory()){
    try {
        FileUtils.deleteDirectory(ghDirectory);
    } catch (IOException e) {
        e.printStackTrace();
    }
}
GraphHopperConfig config = new GraphHopperConfig();
config.putObject("graph.location", graphHopperFolder);
graphHopper = new MappedOSMGraphHopper();
graphHopper.init(config);
graphHopper.setDataReaderFile(filePath);
graphHopper.forDesktop();
graphHopper.setEncodingManager(EncodingManager.create("car"));
graphHopper.setProfiles(
  new ProfileConfig("my_car").setVehicle("car").setWeighting("shortest"));
graphHopper.importOrLoad();

Where the MappedOSMGraphHopper is a variant of this.

I get the same error running on GraphHopperOSM though.

I am on com.graphhopper:graphhopper-map-matching-core:1.0-pre33

5

I’ll check with my setup when I have time, probably later today though.

6

I didn’t have an issue with the pbf

7

I had exactly the same error in my application last week. It’s related to the version of the protobuf library that Maven pulls in as a dependency. I’m not satisfied I’ve truly gotten to the bottom of it, but it looked like osmosis-osm-binary depended upon an old version of protobuf, and some other dependency of mine forced a newer version of protobuf to be used.

I resolved it by explicitly adding a dependency to a newer osmosis-osm-binary in my pom.xml (GH played nicely with this, and this new version depended upon a more recent protobuf library, so all worked well).

    <dependency>
        <groupId>org.openstreetmap.osmosis</groupId>
        <artifactId>osmosis-osm-binary</artifactId>
        <version>0.47.3</version>
    </dependency>

FWIW, it looks like the dependency that caused it was on Google’s “tink” encryption library - this requires a newer version of protobuf (3.11.4, whilst the osmosis-osm-binary version that GH uses requires 3.6.1)

3 Likes
8

Can you explain this in more detail? Is there a new osmosis-osm-binary we could upgrade to? And would this be compatible with the OSM pbf files?

9

I’m not near my IDE right now to reproduce it, but my understanding is as follows:

GraphHopper has a dependency on osmos-osm-binary with version 0.47, which in turn depends on Google’s protobuf at 3.6.1 (see https://mvnrepository.com/artifact/org.openstreetmap.osmosis/osmosis-osm-binary/0.47)

Google Tink depended up protobuf 3.11.4, so my IDE dutifully upgraded to protobuf to 3.11.4. So I was running osmos-osm-binary:0.47 with protobuf:3.11.4. It compiled fine, but at runtime it’d crash out with the error that the OP reported.

I found there was a newer version of osmosis-osm-binary (0.47.3), which has updated its protobuf dependency to 3.11.4. It’s commented on in https://github.com/openstreetmap/osmosis/pull/58

Once I upgraded to osmosis-osm-binary version 0.47.3,everything worked fine. It parsed my PBF files just fine (I’ve successfully parsed the planet.pbf on it today in fact).

Anyway, if you wanted to reproduce the issue, then it should be as simple as including protobuf:3.11.4 in your pom.xml. And for me, the resolution was to upgrade to osmosis-osm-binary:0.47.3

10

Ah, thanks for the insights! Have created: https://github.com/graphhopper/graphhopper/issues/2019

2 Likes
11

Excellent! This solved my problem!

12

I am experiencing he same issue, I have checked my POM.xml and it has that version 0.47.3 specified.

Exception in thread “PBF Reader” java.lang.RuntimeException: Unable to read PBF file.
at com.graphhopper.reader.osm.pbf.PbfReader.run(PbfReader.java:50)
at java.base/java.lang.Thread.run(Unknown Source)

The pbf file is downloaded from http://download.geofabrik.de/australia-oceania/australia-latest.osm.pbf

13

Well it was a PICNIC error, I needed to read the error message more thoroughly and see that I needed to increase the Java Heapsize.

Maybe this is a hint for fellow traveller on this page anyway.