OSM PBF Loader¶

OSMPbfLoader can really quickly parse full OSM extract in the form of *.osm.pbf file.

It can download and parse a lot of features much faster than the OSMOnlineLoader, but it's much more useful when a lot of different features are required at once (like when using predefined filters).

When only a single or few features are needed, OSMOnlineLoader might be a better choice, since OSMPbfLoader will use a full extract of all features in a given region and will have to iterate over all of them.

Run this notebook in Google Colab:

Remember to install the srai library before running the notebook:

%pip install srai[all]

In [1]:

Copied!





import geopandas as gpd
from shapely.geometry import Point, box

from srai.constants import REGIONS_INDEX, WGS84_CRS
from srai.geometry import buffer_geometry
from srai.loaders.osm_loaders import OSMPbfLoader
from srai.loaders.osm_loaders.filters import GEOFABRIK_LAYERS, HEX2VEC_FILTER
from srai.loaders.osm_loaders.filters.popular import get_popular_tags
from srai.regionalizers import geocode_to_region_gdf
import geopandas as gpd
from shapely.geometry import Point, box

from srai.constants import REGIONS_INDEX, WGS84_CRS
from srai.geometry import buffer_geometry
from srai.loaders.osm_loaders import OSMPbfLoader
from srai.loaders.osm_loaders.filters import GEOFABRIK_LAYERS, HEX2VEC_FILTER
from srai.loaders.osm_loaders.filters.popular import get_popular_tags
from srai.regionalizers import geocode_to_region_gdf

Using OSMPbfLoader to download data for a specific area¶

Download all features from `HEX2VEC_FILTER` in Warsaw, Poland¶

In [2]:

Copied!





loader = OSMPbfLoader()
warsaw_gdf = geocode_to_region_gdf("Warsaw, Poland")
warsaw_features_gdf = loader.load(warsaw_gdf, HEX2VEC_FILTER)
warsaw_features_gdf
loader = OSMPbfLoader()
warsaw_gdf = geocode_to_region_gdf("Warsaw, Poland")
warsaw_features_gdf = loader.load(warsaw_gdf, HEX2VEC_FILTER)
warsaw_features_gdf

Finished operation in 0:00:53

Out[2]:

	geometry	aeroway	amenity	building	healthcare	historic	landuse	leisure	military	natural	office	shop	sport	tourism	water	waterway
feature_id
node/1269618045	POINT (21.08914 52.24265)	None	restaurant	None	None	None	None	None	None	None	None	None	None	None	None	None
node/1269640653	POINT (21.03589 52.20203)	None	fast_food	None	None	None	None	None	None	None	None	None	None	None	None	None
node/1269640664	POINT (21.03967 52.20219)	None	None	None	None	None	None	None	None	None	None	hearing_aids	None	None	None	None
node/1270633772	POINT (20.9571 52.32017)	None	None	None	None	None	None	playground	None	None	None	None	None	None	None	None
node/1270689059	POINT (21.06246 52.13417)	None	None	None	None	None	None	None	None	None	None	chemist	None	None	None	None
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
way/1358329361	POLYGON ((21.06142 52.24099, 21.06143 52.24099...	None	toilets	toilets	None	None	None	None	None	None	None	None	None	None	None	None
way/1358329962	POLYGON ((21.03105 52.25706, 21.03107 52.25703...	None	None	None	None	None	garages	None	None	None	None	None	None	None	None	None
way/1358332342	POLYGON ((20.92473 52.20373, 20.9247 52.20367,...	None	parking	None	None	None	None	None	None	None	None	None	None	None	None	None
way/1358332343	POLYGON ((20.92505 52.20357, 20.92513 52.20359...	None	parking	None	None	None	None	None	None	None	None	None	None	None	None	None
way/1358376631	POLYGON ((20.96831 52.25629, 20.96831 52.2563,...	None	parking	None	None	None	None	None	None	None	None	None	None	None	None	None

331712 rows × 16 columns

Plot features¶

Inspired by prettymaps

In [3]:

Copied!

clipped_features_gdf = warsaw_features_gdf.clip(warsaw_gdf.geometry.union_all())
clipped_features_gdf = warsaw_features_gdf.clip(warsaw_gdf.geometry.union_all())

In [4]:

Copied!





ax = warsaw_gdf.plot(color="lavender", figsize=(16, 16))

# plot water
clipped_features_gdf.dropna(subset=["water", "waterway"], how="all").plot(
    ax=ax, color="deepskyblue"
)

# plot greenery
clipped_features_gdf[
    clipped_features_gdf["landuse"].isin(
        ["grass", "orchard", "flowerbed", "forest", "greenfield", "meadow"]
    )
].plot(ax=ax, color="mediumseagreen")

# plot buildings
clipped_features_gdf.dropna(subset=["building"], how="all").plot(
    ax=ax, color="dimgray", markersize=0.1
)

xmin, ymin, xmax, ymax = warsaw_gdf.total_bounds
ax.set_xlim(xmin, xmax)
ax.set_ylim(ymin, ymax)

ax.set_axis_off()
ax = warsaw_gdf.plot(color="lavender", figsize=(16, 16))

# plot water
clipped_features_gdf.dropna(subset=["water", "waterway"], how="all").plot(
    ax=ax, color="deepskyblue"
)

# plot greenery
clipped_features_gdf[
    clipped_features_gdf["landuse"].isin(
        ["grass", "orchard", "flowerbed", "forest", "greenfield", "meadow"]
    )
].plot(ax=ax, color="mediumseagreen")

# plot buildings
clipped_features_gdf.dropna(subset=["building"], how="all").plot(
    ax=ax, color="dimgray", markersize=0.1
)

xmin, ymin, xmax, ymax = warsaw_gdf.total_bounds
ax.set_xlim(xmin, xmax)
ax.set_ylim(ymin, ymax)

ax.set_axis_off()

No description has been provided for this image

Download all features from popular tags based on OSMTagInfo in Vienna, Austria¶

In [5]:

Copied!

popular_tags = get_popular_tags(in_wiki_only=True)

num_keys = len(popular_tags)
f"Unique keys: {num_keys}."
popular_tags = get_popular_tags(in_wiki_only=True)

num_keys = len(popular_tags)
f"Unique keys: {num_keys}."

Out[5]:

'Unique keys: 357.'

In [6]:

Copied!

{k: popular_tags[k] for k in list(popular_tags)[:10]}
{k: popular_tags[k] for k in list(popular_tags)[:10]}

Out[6]:

{'4wd_only': ['yes'],
 'LandPro08:reviewed': ['no'],
 'abandoned': ['yes'],
 'abandoned:railway': ['rail'],
 'abutters': ['residential'],
 'access': ['agricultural',
  'customers',
  'delivery',
  'designated',
  'destination',
  'forestry',
  'no',
  'permissive',
  'permit',
  'private',
  'unknown',
  'yes'],
 'addr:TW:dataset': ['137998'],
 'addr:country': ['CZ', 'DE', 'RU', 'US'],
 'addr:state': ['AZ',
  'CA',
  'CO',
  'CT',
  'FL',
  'IN',
  'KY',
  'MD',
  'ME',
  'NC',
  'NJ',
  'NY',
  'PA',
  'TX'],
 'admin_level': ['10', '11', '2', '4', '5', '6', '7', '8', '9']}

In [7]:

Copied!





vienna_center_circle = buffer_geometry(Point(16.37009, 48.20931), meters=1000)
vienna_center_circle_gdf = gpd.GeoDataFrame(
    geometry=[vienna_center_circle],
    crs=WGS84_CRS,
    index=gpd.pd.Index(data=["Vienna"], name=REGIONS_INDEX),
)
vienna_center_circle = buffer_geometry(Point(16.37009, 48.20931), meters=1000)
vienna_center_circle_gdf = gpd.GeoDataFrame(
    geometry=[vienna_center_circle],
    crs=WGS84_CRS,
    index=gpd.pd.Index(data=["Vienna"], name=REGIONS_INDEX),
)

In [8]:

Copied!

loader = OSMPbfLoader()
vienna_features_gdf = loader.load(vienna_center_circle_gdf, popular_tags)
vienna_features_gdf
loader = OSMPbfLoader()
vienna_features_gdf = loader.load(vienna_center_circle_gdf, popular_tags)
vienna_features_gdf

/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/quackosm/pb
f_file_reader.py:2614: UserWarning: Select clause contains more than 100 columns
(found 357 columns). Query might fail with insufficient memory resources. 
Consider applying more restrictive OsmTagsFilter for parsing.
  warnings.warn(

Finished operation in 0:00:29

Out[8]:

	geometry	abandoned	access	admin_level	advertising	amenity	area:highway	artwork_type	atm	barrier	...	tunnel	type	usage	vehicle	vending	waste	water	water_source	waterway	wheelchair
feature_id
node/1076021076	POINT (16.38161 48.20756)	None	None	None	None	restaurant	None	None	None	None	...	None	None	None	None	None	None	None	None	None	no
node/1167275147	POINT (16.35999 48.21305)	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
node/1167275155	POINT (16.36003 48.2132)	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
node/1167275172	POINT (16.35906 48.21284)	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
node/1167275175	POINT (16.3597 48.21325)	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
way/298000297	POLYGON ((16.37222 48.20149, 16.37216 48.20137...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/298000298	POLYGON ((16.37202 48.20108, 16.37208 48.20121...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/298780844	LINESTRING (16.37487 48.20095, 16.37468 48.201...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/298780989	LINESTRING (16.37459 48.20124, 16.37457 48.201...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/298780990	LINESTRING (16.37458 48.20149, 16.37457 48.201...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None

22728 rows × 189 columns

Plot features¶

Uses default preset colours from prettymaps

In [9]:

Copied!

clipped_vienna_features_gdf = vienna_features_gdf.clip(vienna_center_circle)
clipped_vienna_features_gdf = vienna_features_gdf.clip(vienna_center_circle)

In [10]:

Copied!





ax = vienna_center_circle_gdf.plot(color="#F2F4CB", figsize=(16, 16))

# plot water
clipped_vienna_features_gdf.dropna(subset=["water", "waterway"], how="all").plot(
    ax=ax, color="#a8e1e6"
)

# plot streets
clipped_vienna_features_gdf.dropna(subset=["highway"], how="all").plot(
    ax=ax, color="#475657", markersize=0.1
)

# plot buildings
clipped_vienna_features_gdf.dropna(subset=["building"], how="all").plot(ax=ax, color="#FF5E5B")

# plot parkings
clipped_vienna_features_gdf[
    (clipped_vienna_features_gdf["amenity"] == "parking")
    | (clipped_vienna_features_gdf["highway"] == "pedestrian")
].plot(ax=ax, color="#2F3737", markersize=0.1)

# plot greenery
clipped_vienna_features_gdf[
    clipped_vienna_features_gdf["landuse"].isin(
        ["grass", "orchard", "flowerbed", "forest", "greenfield", "meadow"]
    )
].plot(ax=ax, color="#8BB174")

xmin, ymin, xmax, ymax = vienna_center_circle_gdf.total_bounds
ax.set_xlim(xmin, xmax)
ax.set_ylim(ymin, ymax)

ax.set_axis_off()
ax = vienna_center_circle_gdf.plot(color="#F2F4CB", figsize=(16, 16))

# plot water
clipped_vienna_features_gdf.dropna(subset=["water", "waterway"], how="all").plot(
    ax=ax, color="#a8e1e6"
)

# plot streets
clipped_vienna_features_gdf.dropna(subset=["highway"], how="all").plot(
    ax=ax, color="#475657", markersize=0.1
)

# plot buildings
clipped_vienna_features_gdf.dropna(subset=["building"], how="all").plot(ax=ax, color="#FF5E5B")

# plot parkings
clipped_vienna_features_gdf[
    (clipped_vienna_features_gdf["amenity"] == "parking")
    | (clipped_vienna_features_gdf["highway"] == "pedestrian")
].plot(ax=ax, color="#2F3737", markersize=0.1)

# plot greenery
clipped_vienna_features_gdf[
    clipped_vienna_features_gdf["landuse"].isin(
        ["grass", "orchard", "flowerbed", "forest", "greenfield", "meadow"]
    )
].plot(ax=ax, color="#8BB174")

xmin, ymin, xmax, ymax = vienna_center_circle_gdf.total_bounds
ax.set_xlim(xmin, xmax)
ax.set_ylim(ymin, ymax)

ax.set_axis_off()

$No description has been provided for this image$

Download all grouped features based on Geofabrik layers in New York, USA¶

In [11]:

Copied!





manhattan_bbox = box(-73.994551, 40.762396, -73.936872, 40.804239)
manhattan_bbox_gdf = gpd.GeoDataFrame(
    geometry=[manhattan_bbox],
    crs=WGS84_CRS,
    index=gpd.pd.Index(data=["New York"], name=REGIONS_INDEX),
)
manhattan_bbox = box(-73.994551, 40.762396, -73.936872, 40.804239)
manhattan_bbox_gdf = gpd.GeoDataFrame(
    geometry=[manhattan_bbox],
    crs=WGS84_CRS,
    index=gpd.pd.Index(data=["New York"], name=REGIONS_INDEX),
)

In [12]:

Copied!

loader = OSMPbfLoader()
new_york_features_gdf = loader.load(manhattan_bbox_gdf, GEOFABRIK_LAYERS)
new_york_features_gdf
loader = OSMPbfLoader()
new_york_features_gdf = loader.load(manhattan_bbox_gdf, GEOFABRIK_LAYERS)
new_york_features_gdf

/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/quackosm/osm_extracts/__init__.py:602: GeometryNotCoveredWarning: Skipping extract because of low IoU value (bbbike_newyork, 0.000187).
  warnings.warn(

Finished operation in 0:00:34

Out[12]:

	geometry	accommodation	buildings	catering	education	fuel_parking	health	highway_links	landuse	leisure	...	public	railways	shopping	tourism	traffic	transport	very_small_roads	water	water_traffic	waterways
feature_id
node/1906721425	POINT (-73.98569 40.76828)	None	None	amenity=fast_food	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
node/1912138232	POINT (-73.9398 40.80423)	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	public_transport=stop_position	None	None	None	None
node/1913191317	POINT (-73.96733 40.79874)	None	None	amenity=restaurant	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
node/2300369399	POINT (-73.97672 40.78568)	None	None	amenity=restaurant	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
node/2301162810	POINT (-73.97865 40.78515)	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
way/1354419959	LINESTRING (-73.95965 40.80116, -73.95962 40.8...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/1354419960	LINESTRING (-73.95909 40.80099, -73.95945 40.8...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/1354419961	LINESTRING (-73.95968 40.80111, -73.95965 40.8...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/1354419962	LINESTRING (-73.95968 40.80111, -73.95958 40.8...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None
way/1354419963	LINESTRING (-73.96385 40.80296, -73.96373 40.8...	None	None	None	None	None	None	None	None	None	...	None	None	None	None	None	None	None	None	None	None

49405 rows × 27 columns

Plot features¶

Inspired by https://snazzymaps.com/style/14889/flat-pale

In [13]:

Copied!





ax = manhattan_bbox_gdf.plot(color="#e7e7df", figsize=(16, 16))

# plot greenery
new_york_features_gdf[new_york_features_gdf["leisure"] == "leisure=park"].plot(
    ax=ax, color="#bae5ce"
)

# plot water
new_york_features_gdf.dropna(subset=["water", "waterways"], how="all").plot(ax=ax, color="#c7eced")

# plot streets
new_york_features_gdf.dropna(subset=["paths_unsuitable_for_cars"], how="all").plot(
    ax=ax, color="#e7e7df", linewidth=1
)
new_york_features_gdf.dropna(
    subset=["very_small_roads", "highway_links", "minor_roads"], how="all"
).plot(ax=ax, color="#fff", linewidth=2)
new_york_features_gdf.dropna(subset=["major_roads"], how="all").plot(
    ax=ax, color="#fac9a9", linewidth=3
)

# plot buildings
new_york_features_gdf.dropna(subset=["buildings"], how="all").plot(ax=ax, color="#cecebd")

xmin, ymin, xmax, ymax = manhattan_bbox_gdf.total_bounds
ax.set_xlim(xmin, xmax)
ax.set_ylim(ymin, ymax)

ax.set_axis_off()
ax = manhattan_bbox_gdf.plot(color="#e7e7df", figsize=(16, 16))

# plot greenery
new_york_features_gdf[new_york_features_gdf["leisure"] == "leisure=park"].plot(
    ax=ax, color="#bae5ce"
)

# plot water
new_york_features_gdf.dropna(subset=["water", "waterways"], how="all").plot(ax=ax, color="#c7eced")

# plot streets
new_york_features_gdf.dropna(subset=["paths_unsuitable_for_cars"], how="all").plot(
    ax=ax, color="#e7e7df", linewidth=1
)
new_york_features_gdf.dropna(
    subset=["very_small_roads", "highway_links", "minor_roads"], how="all"
).plot(ax=ax, color="#fff", linewidth=2)
new_york_features_gdf.dropna(subset=["major_roads"], how="all").plot(
    ax=ax, color="#fac9a9", linewidth=3
)

# plot buildings
new_york_features_gdf.dropna(subset=["buildings"], how="all").plot(ax=ax, color="#cecebd")

xmin, ymin, xmax, ymax = manhattan_bbox_gdf.total_bounds
ax.set_xlim(xmin, xmax)
ax.set_ylim(ymin, ymax)

ax.set_axis_off()

Using OSMPbfLoader to download data for a specific area and transforming it to GeoParquet file¶

Download all grouped features based on Geofabrik layers in Reykjavík, Iceland¶

In [14]:

Copied!





loader = OSMPbfLoader()
reykjavik_gdf = geocode_to_region_gdf("Reykjavík, IS")
reykjavik_features_gpq = loader.load_to_geoparquet(reykjavik_gdf, GEOFABRIK_LAYERS)
reykjavik_features_gpq
loader = OSMPbfLoader()
reykjavik_gdf = geocode_to_region_gdf("Reykjavík, IS")
reykjavik_features_gpq = loader.load_to_geoparquet(reykjavik_gdf, GEOFABRIK_LAYERS)
reykjavik_features_gpq

Finished operation in 0:00:22

Out[14]:

PosixPath('files/3b68f6ecc515eba6f588efa104d7eee92b450cf9c7e54f445afae421e604cf3f_098931824b94bfe02e01bc4987422a39ec47bd3f6924e5325ddae0846badafba_exploded.parquet')

Read those features using DuckDB¶

In [15]:

Copied!

import duckdb

connection = duckdb.connect()

connection.load_extension("parquet")
connection.load_extension("spatial")

features_relation = connection.read_parquet(str(reykjavik_features_gpq))
features_relation
import duckdb

connection = duckdb.connect()

connection.load_extension("parquet")
connection.load_extension("spatial")

features_relation = connection.read_parquet(str(reykjavik_features_gpq))
features_relation

Out[15]:

┌─────────────────┬───────────────┬─────────────┬───────────┬──────────┬───────────┬──────────────┬─────────┬───────────────┬─────────┬─────────┬─────────────┬─────────────┬─────────┬─────────┬─────────┬───────────────────────────┬─────────┬─────────┬──────────┬──────────────┬─────────┬───────────────────────────┬───────────┬──────────────────┬─────────┬───────────────┬───────────┬────────────────────────────────┐
│   feature_id    │ accommodation │ air_traffic │ buildings │ catering │ education │ fuel_parking │ health  │ highway_links │ landuse │ leisure │ major_roads │ minor_roads │ miscpoi │  money  │ natural │ paths_unsuitable_for_cars │  pofw   │ public  │ railways │   shopping   │ tourism │          traffic          │ transport │ very_small_roads │  water  │ water_traffic │ waterways │            geometry            │
│     varchar     │    varchar    │   varchar   │  varchar  │ varchar  │  varchar  │   varchar    │ varchar │    varchar    │ varchar │ varchar │   varchar   │   varchar   │ varchar │ varchar │ varchar │          varchar          │ varchar │ varchar │ varchar  │   varchar    │ varchar │          varchar          │  varchar  │     varchar      │ varchar │    varchar    │  varchar  │            geometry            │
├─────────────────┼───────────────┼─────────────┼───────────┼──────────┼───────────┼──────────────┼─────────┼───────────────┼─────────┼─────────┼─────────────┼─────────────┼─────────┼─────────┼─────────┼───────────────────────────┼─────────┼─────────┼──────────┼──────────────┼─────────┼───────────────────────────┼───────────┼──────────────────┼─────────┼───────────────┼───────────┼────────────────────────────────┤
│ node/738189872  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8004768 64.1272348) │
│ node/739058158  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.7970908 64.1236694) │
│ node/754499254  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ shop=alcohol │ NULL    │ NULL                      │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8480784 64.1403193) │
│ node/754499261  │ tourism=hotel │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ NULL                      │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.9385928 64.1482053) │
│ node/761664757  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=motorway_junction │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.82538 64.124046)    │
│ node/761681461  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.9404569 64.14286)   │
│ node/765786389  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.9090665 64.1365761) │
│ node/765796009  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.9085977 64.1377917) │
│ node/769892993  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.9329523 64.1492124) │
│ node/786031184  │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=crossing          │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.947746 64.1479759)  │
│       ·         │  ·            │  ·          │  ·        │  ·       │  ·        │  ·           │  ·      │  ·            │  ·      │  ·      │  ·          │  ·          │  ·      │  ·      │  ·      │  ·                        │  ·      │  ·      │  ·       │  ·           │  ·      │        ·                  │  ·        │  ·               │  ·      │  ·            │  ·        │               ·                │
│       ·         │  ·            │  ·          │  ·        │  ·       │  ·        │  ·           │  ·      │  ·            │  ·      │  ·      │  ·          │  ·          │  ·      │  ·      │  ·      │  ·                        │  ·      │  ·      │  ·       │  ·           │  ·      │        ·                  │  ·        │  ·               │  ·      │  ·            │  ·        │               ·                │
│       ·         │  ·            │  ·          │  ·        │  ·       │  ·        │  ·           │  ·      │  ·            │  ·      │  ·      │  ·          │  ·          │  ·      │  ·      │  ·      │  ·                        │  ·      │  ·      │  ·       │  ·           │  ·      │        ·                  │  ·        │  ·               │  ·      │  ·            │  ·        │               ·                │
│ node/2004266249 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8835685 64.1412258) │
│ node/2004266251 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8424233 64.1412361) │
│ node/2004266253 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8527685 64.1412489) │
│ node/2004266256 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8473853 64.1412514) │
│ node/2004266258 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8522624 64.1412539) │
│ node/2004266260 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8511452 64.141254)  │
│ node/2004266262 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8517469 64.1412553) │
│ node/2004266265 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8621219 64.1412681) │
│ node/2004266267 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8649903 64.1412749) │
│ node/2004266269 │ NULL          │ NULL        │ NULL      │ NULL     │ NULL      │ NULL         │ NULL    │ NULL          │ NULL    │ NULL    │ NULL        │ NULL        │ NULL    │ NULL    │ NULL    │ NULL                      │ NULL    │ NULL    │ NULL     │ NULL         │ NULL    │ highway=street_lamp       │ NULL      │ NULL             │ NULL    │ NULL          │ NULL      │ POINT (-21.8610448 64.1412817) │
├─────────────────┴───────────────┴─────────────┴───────────┴──────────┴───────────┴──────────────┴─────────┴───────────────┴─────────┴─────────┴─────────────┴─────────────┴─────────┴─────────┴─────────┴───────────────────────────┴─────────┴─────────┴──────────┴──────────────┴─────────┴───────────────────────────┴───────────┴──────────────────┴─────────┴───────────────┴───────────┴────────────────────────────────┤
│ ? rows (>9999 rows, 20 shown)                                                                                                                                                                                                                                                                                                                                                                                      29 columns │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Count all buildings¶

In [16]:

Copied!

features_relation.filter("buildings IS NOT NULL").count("feature_id")
features_relation.filter("buildings IS NOT NULL").count("feature_id")

Out[16]:

┌───────────────────┐
│ count(feature_id) │
│       int64       │
├───────────────────┤
│             25361 │
└───────────────────┘

OSM PBF Loader¶

Using OSMPbfLoader to download data for a specific area¶

Download all features from HEX2VEC_FILTER in Warsaw, Poland¶

Plot features¶

Download all features from popular tags based on OSMTagInfo in Vienna, Austria¶

Plot features¶

Download all grouped features based on Geofabrik layers in New York, USA¶

Plot features¶

Using OSMPbfLoader to download data for a specific area and transforming it to GeoParquet file¶

Download all grouped features based on Geofabrik layers in Reykjavík, Iceland¶

Read those features using DuckDB¶

Count all buildings¶

Download all features from `HEX2VEC_FILTER` in Warsaw, Poland¶