New OSM file format: 30% smaller than PBF, 5x faster to import

  • I was curious about the spec of this new GOB format, there's a comment down thread explaining that there isn't a spec yet but discussing some of the details of the format: https://community.openstreetmap.org/t/new-osm-file-format-30...

    Aside from OSM specifics, performance friendly formats for spatial data that support spatial indexing can make huge impact on usability and productivity of applications. e.g. trying to view a large dataset in QGIS that has been saved as KMZ (zipped XML) can make QGIS basically hang for minutes, while the same dataset saved as something like flatgeobuf [1] can be loaded instantly.

    [1] https://flatgeobuf.org/

  • Tangentially related question for any of you GIS people who might be lurking in this thread:

    Can anyone recommend me a method of meshing LIDAR point clouds? The sparseness of the data on building walls & other near-vertical surfaces combined with a lack of point normals leads to degenerate solutions with all the common approaches (poisson/ball pivot/vcg in meshlab) not to mention extremely slow perf. Tree canopies and overhanging parapets make a simple heightmap approach less-than desirable (though ultimately acceptable if I can't find anything better). I'm trying to turn 90 billion lidar points into maybe 30-50 million triangles, hopefully without spending months developing a custom pipeline.

  • Does this use the new OSM data model?

    https://media.jochentopf.com/media/2022-08-15-study-evolutio...

    https://github.com/osmlab/osm-data-model

    https://blog.openstreetmap.org/2023/01/04/reminder-call-for-...

    Resolving the coordinates to node references in current data model is such a nuisance as it's slow and requires lots of RAM.

  • My opinion: Without support in libosmium and GDAL, this will remain a marginal phenomenon.

  • Does it work with osmium?

  • does the tiling approach have any trade-offs on random access or is lookup performance comparable to PBF once loaded?