OpenStreetMap extracts¶
QuackOSM exposes a simple API allowing user to download OSM extracts from multiple sources.
Library keeps a local state of available extracts in cache. Those indexes are precalculated and available in the GitHub repository. Library tries to download those if they are not available locally, or can regenerate them locally from scratch. The full logic related to OSM extracts is available in the osmfinder library.
This example notebook shows how to:
- display available extracts,
- query extracts by name,
- find extracts covering a given geometry.
Display available OSM extracts¶
Extracts have short name and a full name. Short name is just a current description of the region (eg. Monaco). Full name contains information about the extract source and all a whole nesting hierarchy (eg. geofabrik_europe_monaco).
By default, function for displaying those extracts shows short name. In these examples below full names have been used.
from osmfinder import OsmExtractSource, display_available_extracts
Geofabrik¶
display_available_extracts(OsmExtractSource.geofabrik, head=10)
Geofabrik └── 🌐 geofabrik_africa (78.8M km²) ├── 🌐 geofabrik_africa_algeria (2.41M km²) ├── 🌐 geofabrik_africa_angola (1.56M km²) ├── 🌐 geofabrik_africa_benin (125K km²) ├── 🌐 geofabrik_africa_botswana (586K km²) ├── 🌐 geofabrik_africa_burkina-faso (279K km²) ├── 🌐 geofabrik_africa_burundi (28K km²) ├── 🌐 geofabrik_africa_cameroon (496K km²) ├── 🌐 geofabrik_africa_canary-islands (142K km²) ├── 🌐 geofabrik_africa_cape-verde (176K km²) └── 🌐 geofabrik_africa_central-african-republic (648K km²)
OpenStreetMap FR¶
display_available_extracts(OsmExtractSource.osm_fr, head=10)
osmfr └── 🌐 osmfr_africa (77.8M km²) ├── 🌐 osmfr_africa_algeria (2.37M km²) ├── 🌐 osmfr_africa_angola (1.32M km²) ├── 🌐 osmfr_africa_benin (123K km²) ├── 🌐 osmfr_africa_bir_tawil (2.12K km²) ├── 🌐 osmfr_africa_bouvet_island (10.7K km²) ├── 🌐 osmfr_africa_burkina_faso (282K km²) ├── 🌐 osmfr_africa_burundi (29.1K km²) ├── 🌐 osmfr_africa_cameroon (499K km²) ├── 🌐 osmfr_africa_cape_verde (66.3K km²) └── 🌐 osmfr_africa_central_african_republic (645K km²)
BBBike¶
display_available_extracts(OsmExtractSource.bbbike, head=10)
BBBike ├── 🌐 bbbike_aachen (2.13K km²) ├── 🌐 bbbike_aarhus (1.14K km²) ├── 🌐 bbbike_adelaide (795 km²) ├── 🌐 bbbike_albuquerque (734 km²) ├── 🌐 bbbike_alexandria (1.73K km²) ├── 🌐 bbbike_amsterdam (1.62K km²) ├── 🌐 bbbike_antwerpen (1.51K km²) ├── 🌐 bbbike_arnhem (1.22K km²) ├── 🌐 bbbike_auckland (2.91K km²) └── 🌐 bbbike_augsburg (2.13K km²)
Geo2Day¶
display_available_extracts(OsmExtractSource.geo2day, head=10)
GEO2Day └── 🌐 geo2day_africa (77.8M km²) ├── 🌐 geo2day_africa_algeria (2.37M km²) ├── 🌐 geo2day_africa_angola (1.32M km²) ├── 🌐 geo2day_africa_benin (123K km²) ├── 🌐 geo2day_africa_botswana (586K km²) ├── 🌐 geo2day_africa_bouvet_island (10.7K km²) ├── 🌐 geo2day_africa_burkina_faso (282K km²) ├── 🌐 geo2day_africa_burundi (29.1K km²) ├── 🌐 geo2day_africa_cameroon (498K km²) ├── 🌐 geo2day_africa_canary-islands (142K km²) └── 🌐 geo2day_africa_cape_verde (66.3K km²)
Movisda - admin¶
display_available_extracts(OsmExtractSource.movisda_admin, head=10)
Movisda-admin └── 🌐 movisda-admin_afghanistan (642K km²) ├── 🌐 movisda-admin_afghanistan_badakhshan_province (43.4K km²) ├── 🌐 movisda-admin_afghanistan_badghis_province (20.7K km²) ├── 🌐 movisda-admin_afghanistan_baghlan_province (17.9K km²) ├── 🌐 movisda-admin_afghanistan_balkh_province (16.7K km²) ├── 🌐 movisda-admin_afghanistan_bamyan_province (17.8K km²) ├── 🌐 movisda-admin_afghanistan_daykundi_province (16.7K km²) ├── 🌐 movisda-admin_afghanistan_farah_province (49.6K km²) ├── 🌐 movisda-admin_afghanistan_faryab_province (20.4K km²) ├── 🌐 movisda-admin_afghanistan_ghazni_province (21.9K km²) └── 🌐 movisda-admin_afghanistan_ghor_province (37K km²)
Movisda - grid¶
display_available_extracts(OsmExtractSource.movisda_grid, head=10)
Movisda-grid ├── 🌐 movisda-grid_n00e000-10 (1.23M km²) ├── 🌐 movisda-grid_n00e008 (12.4K km²) ├── 🌐 movisda-grid_n00e010-10 (1.23M km²) ├── 🌐 movisda-grid_n00e020-10 (1.23M km²) ├── 🌐 movisda-grid_n00e023 (12.4K km²) ├── 🌐 movisda-grid_n00e024 (12.4K km²) ├── 🌐 movisda-grid_n00e027 (12.4K km²) ├── 🌐 movisda-grid_n00e029 (12.4K km²) ├── 🌐 movisda-grid_n00e030 (12.4K km²) └── 🌐 movisda-grid_n00e030-10 (1.23M km²)
Query OSM extract by name¶
To find an OSM extract by text query and get the data from it, you can use the convert_osm_extract_to_geodataframe and convert_osm_extract_to_parquet functions.
from quackosm import convert_osm_extract_to_geodataframe
Vatican City¶
Download data for the Vatican city from any repository. Since it is available in multiple sources, the extract with the smallest area is selected and a warning is displayed.
convert_osm_extract_to_geodataframe("Vatican City")
/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages/osmfinder/finder/_query.py:157: OsmExtractMultipleMatchesWarning: Multiple extracts matched by query "Vatican City" (matching full names: "geo2day_europe_vatican_city", "movisda-admin_vatican_city", "osmfr_europe_vatican_city"). Selected "movisda-admin_vatican_city". Use the full name as a query or set `select_first_match=False` to control this behaviour. warnings.warn(
Finished operation in 0:00:05
| tags | geometry | |
|---|---|---|
| feature_id | ||
| node/13050566010 | {'natural': 'tree'} | POINT (12.45417 41.90029) |
| node/13050566009 | {'natural': 'tree'} | POINT (12.45409 41.90029) |
| node/13050566008 | {'natural': 'tree'} | POINT (12.454 41.90031) |
| node/13050566007 | {'natural': 'tree'} | POINT (12.45393 41.90031) |
| way/1408206941 | {'building': 'yes'} | POLYGON ((12.45407 41.90035, 12.45407 41.90036... |
| ... | ... | ... |
| way/1017966595 | {'access': 'private', 'leisure': 'playground'} | POLYGON ((12.45481 41.90717, 12.45485 41.90709... |
| way/1415382584 | {'building': 'roof', 'layer': '1'} | POLYGON ((12.45486 41.90717, 12.45489 41.90717... |
| way/1017966594 | {'building': 'yes', 'ref:osm': '1017966594'} | POLYGON ((12.4549 41.9072, 12.45489 41.90722, ... |
| node/522810848 | {'highway': 'turning_circle'} | POINT (12.45537 41.90725) |
| way/111733624 | {'building': 'yes'} | POLYGON ((12.45516 41.90732, 12.4552 41.90714,... |
3741 rows × 2 columns
Monaco extract from Geofabrik¶
Download data for the country of Monaco from the Geofabrik repository.
convert_osm_extract_to_geodataframe("Monaco", osm_extract_source="Geofabrik")
Finished operation in 0:00:05
| tags | geometry | |
|---|---|---|
| feature_id | ||
| way/1225956054 | {'name': '7.5 degrees east'} | LINESTRING (7.5 43.5164, 7.5 37.26898) |
| way/161191598 | {'admin_level': '2', 'border_type': 'contiguou... | LINESTRING (7.50024 43.51654, 7.50109 43.51438... |
| way/773124328 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (7.5 43.5164, 7.49789 43.51525, 7.4... |
| way/46428446 | {'border_type': 'contiguous', 'boundary': 'mar... | LINESTRING (7.53299 43.53683, 7.53384 43.53467... |
| way/1225956053 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (7.50024 43.51654, 7.5 43.5164) |
| ... | ... | ... |
| node/254667935 | {'crossing': 'marked', 'crossing:markings': 'z... | POINT (7.43955 43.75317) |
| node/1523103399 | {'bus': 'yes', 'name': 'Apollon', 'public_tran... | POINT (7.4397 43.75326) |
| node/254667939 | {'crossing': 'marked', 'crossing:markings': 'z... | POINT (7.44082 43.75375) |
| node/8722523691 | {'bus': 'yes', 'name': 'Chemin des Grottes', '... | POINT (7.44088 43.75377) |
| way/37794466 | {'admin_level': '8', 'boundary': 'administrati... | LINESTRING (7.42661 43.75948, 7.42662 43.75923... |
9697 rows × 2 columns
Query with multiple matches¶
Some extracts have the same name, or the same extract is available in multiple sources. Trying to get an extract by name with multiple matches will result in an error with list of extracts ids that can be used to get a specific one.
Getting a Ceuta region (autonomous city of Spain bordering with Marocco).
Extract for this region is available both in Geofabrik and OSM fr sources.
from rich import print as rprint
from rich.traceback import Traceback
from quackosm._exceptions import OsmExtractMultipleMatchesError
try:
convert_osm_extract_to_geodataframe("Ceuta", select_first_match=False)
except OsmExtractMultipleMatchesError as ex:
rprint(Traceback.from_exception(type(ex), ex, None))
OsmExtractMultipleMatchesError: Multiple extracts matched by query "Ceuta". Matching extracts full names: "geofabrik_europe_spain_ceuta", "movisda-admin_spain_ceuta", "osmfr_africa_spain_ceuta".
OsmExtractMultipleMatchesError has a property matching_full_names with a list of found extracts full names. It can be used programatically to access those values.
from osmfinder import find_extract_by_query
matched_extracts = []
try:
find_extract_by_query("Ceuta", select_first_match=False)
except OsmExtractMultipleMatchesError as ex:
for full_name in ex.matching_full_names:
matched_extracts.append(find_extract_by_query(full_name).extract)
matched_extracts
[OpenStreetMapExtract(id='Geofabrik_ceuta', name='ceuta', parent='Geofabrik_spain', url='https://download.geofabrik.de/europe/spain/ceuta-latest.osm.pbf', geometry=<MULTIPOLYGON (((-5.391 35.922, -5.279 35.959, -5.257 35.891, -5.342 35.867,...>, file_name='geofabrik_europe_spain_ceuta'), OpenStreetMapExtract(id='Movisda-admin_ES-CE', name='Ceuta', parent='Movisda-admin_ES', url='https://osm.download.movisda.io/admin/ES-CE-latest.osm.pbf', geometry=<MULTIPOLYGON (((-5.343 35.871, -5.319 35.887, -5.319 35.89, -5.314 35.893, ...>, file_name='movisda-admin_spain_ceuta'), OpenStreetMapExtract(id='osmfr_africa_spain_ceuta', name='ceuta', parent='osmfr_africa_spain', url='https://download.openstreetmap.fr/extracts/africa/spain/ceuta-latest.osm.pbf', geometry=<MULTIPOLYGON (((-5.386 35.914, -5.384 35.918, -5.38 35.921, -5.375 35.92, -...>, file_name='osmfr_africa_spain_ceuta')]
We can display both extracts extents on the map.
from osmfinder import extracts_to_geodataframe
extracts_to_geodataframe(matched_extracts).explore(column="id")
Let's download data for the extract from the OpenStreetMap.fr.
convert_osm_extract_to_geodataframe("osmfr_africa_spain_ceuta")
Finished operation in 0:00:05
| tags | geometry | |
|---|---|---|
| feature_id | ||
| way/182763951 | {'admin_level': '2', 'border_type': 'nation', ... | LINESTRING (-5.02132 35.82576, -5.07411 35.826... |
| way/153932243 | {'highway': 'trunk', 'lanes': '1', 'maxspeed':... | LINESTRING (-5.34539 35.86773, -5.34534 35.867... |
| way/87259503 | {'destination': 'Sebta;سبتة', 'destination:lan... | LINESTRING (-5.34506 35.86684, -5.34507 35.866... |
| node/12468816125 | {'man_made': 'survey_point', 'note:en': 'This ... | POINT (-5.34577 35.86744) |
| node/12468816138 | {'man_made': 'survey_point', 'note:en': 'This ... | POINT (-5.34577 35.86744) |
| ... | ... | ... |
| way/998251666 | {'natural': 'bare_rock'} | POLYGON ((-5.3717 35.91912, -5.3717 35.91917, ... |
| way/998251668 | {'natural': 'bare_rock'} | POLYGON ((-5.37066 35.91934, -5.37066 35.91938... |
| way/44630766 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (-5.39583 36.00214, -5.39133 35.972... |
| way/152976804 | {'bdouble': 'no', 'bicycle': 'yes', 'duration'... | LINESTRING (-5.43827 36.13131, -5.43588 36.133... |
| node/240947699 | {'amenity': 'ferry_terminal', 'ferry': 'yes', ... | POINT (-5.43827 36.13131) |
7737 rows × 2 columns
Query with zero matches¶
Sometimes query doesn't match any of the available extracts. Function for finding the extracts, automatically detects close names in case this was a typo and suggests them to the user. Suggestions are sorted based on the closeness to the query.
from quackosm._exceptions import OsmExtractZeroMatchesError
try:
convert_osm_extract_to_geodataframe("Gremany")
except OsmExtractZeroMatchesError as ex:
rprint(Traceback.from_exception(type(ex), ex, None))
OsmExtractZeroMatchesError: Zero extracts matched by query "Gremany". Found full names close to query: "movisda-admin_germany", "geo2day_europe_germany", "osmfr_europe_germany", "geofabrik_europe_germany".
OsmExtractZeroMatchesError has a property matching_full_names with a list of suggested matching names. It can be used programatically to access those values.
closest_matching_extract = None
try:
find_extract_by_query("pland")
except OsmExtractZeroMatchesError as ex:
rprint(Traceback.from_exception(type(ex), ex, None))
closest_matching_extract_name = ex.matching_full_names[0]
closest_matching_extract = find_extract_by_query(closest_matching_extract_name).extract
closest_matching_extract
OsmExtractZeroMatchesError: Zero extracts matched by query "pland". Found full names close to query: "movisda-admin_poland", "geo2day_europe_poland", "osmfr_europe_poland", "geofabrik_europe_poland", "geo2day_europe_norway_oppland", "osmfr_europe_norway_oppland", "movisda-admin_finland_lapland", "geo2day_europe_finland_lapland", "osmfr_europe_finland_lapland", "movisda-admin_bolivia_pando", "geo2day_europe_finland_aland", "osmfr_europe_finland_aland".
OpenStreetMapExtract(id='Movisda-admin_PL', name='Poland', parent='Movisda-admin', url='https://osm.download.movisda.io/admin/PL-latest.osm.pbf', geometry=<POLYGON ((14.078 54.441, 14.07 54.278, 14.168 54.239, 14.242 53.988, 14.227...>, file_name='movisda-admin_poland')
Sometimes query can yield zero closest matches.
try:
find_extract_by_query("totally_nonexistent_osm_extract")
except OsmExtractZeroMatchesError as ex:
rprint(Traceback.from_exception(type(ex), ex, None))
OsmExtractZeroMatchesError: Zero extracts matched by query "totally_nonexistent_osm_extract". Zero close matches have been found.
Find OSM extracts covering a given geometry¶
One ot the most interesting feature of QuackOSM is the ability to automatically find extracts for a selected region without any domain knowledge of the services providing these extracts.
Search algorithm tries to find the best matching extracts to cover a given geometry filter.
To find an OSM extract by text query and get the data from it, you can use the convert_geometry_to_geodataframe and convert_geometry_to_parquet functions.
Flowchart¶
Here is the flowchart diagram of the algorithm:
flowchart TD
A(Input Geometry)
B{"Geometry
type?"}
B1[Split to Polygons]
subgraph 1["For each Polygon (in a loop)"]
D["Intersect Polygon
with OSM extracts"]
E{"Number of
matching
extracts?"}
E1{"`allow
uncovered
geometry`"}
E3[Warn User]
F["Calculate IoU
between extracts
and a Polygon"]
G["Select extract
with the highest
IoU value"]
H{"IoU >= threshold"}
H1[Keep the extract]
H2[Discard all
candidates]
I["Remove the intersection
area from the Polygon"]
J{Is Polygon empty?}
K(Exit loop)
end
E2(Raise Error)
L["Sort selected
extracts by area
descending"]
subgraph 2["Filter selected extracts (for each Polygon in a loop)"]
M["Initialize empty list
of filtered extracts"]
N{"Is next extract?"}
N1["Select next extract"]
N2("Exit loop")
O{"Is current extract
disjoint with Polygon?"}
O1["Keep the extract"]
O2["Discard the extract"]
P["Remove the intersection
area from the Polygon"]
end
Q["Join lists of
filtered extracts"]
subgraph 3["Simplify filtered extracts"]
R["Start iterating filtered extracts"]
S{"Is next extract?"}
S1["Select next extract"]
S2("Exit loop")
T["Union geometries
of every other extract"]
U{"Is current
extract covered
by other extracts?"}
U1["Discard the extract"]
U2["Keep the extract"]
end
V("End simplification")
A --> B
B -- MultiPolygon --> B1
B -- Polygon --> D
B1 --> D
D --> E
E -- 0 --> E1
E1 -- true --> E3
E1 -- false ----> E2
E -- > 0 --> F
F --> G
G --> H
H -- Yes --> H1
H -- No --> H2
H1 --> I
H2 --> K
I --> J
J -- Yes --> K
J -- No --> D
E3 --> K
K --> L
L --> M
M --> N
N -- Yes --> N1
N -- No ------> N2
N1 --> O
O -- Yes --> O2
O -- No --> O1
O1 --> P
O2 --> N
P --> N
N2 --> Q
Q --> R
R --> S
S -- Yes --> S1
S -- No -----> S2
S1 --> T
T --> U
U -- Yes --> U1
U -- No --> U2
U1 --> R
U2 --> S
S2 --> V
V --> W[Mark steps as
'redundant'
if their
extract was
removed
during
filtering]
W --> X[Calculate
cumulative
coverage
for each
remaining step]
X --> Y("Return results")
Before showing real examples, here is some code for visualizing the algorithm step by step.
from shapely import from_wkt
from shapely.geometry import shape
from osmfinder import find_extracts_by_geometry
from quackosm import convert_geometry_to_geodataframe, geocode_to_geometry
import contextily as cx
import geopandas as gpd
from matplotlib import patches as mpatches
from matplotlib import pyplot as plt
from shapely import Polygon
def plot_full_geometry_coverage_breakdown(result) -> None:
"""Visualize the step-by-step geometry coverage breakdown.
Steps are grouped by sub_polygon_index. Within each group, discarded
steps are reduced to a single representative (the first one encountered).
Each group is plotted in its own figure.
"""
steps = result.steps
if len(steps) == 0:
return
# Group steps by sub_polygon_index
from collections import OrderedDict
groups: OrderedDict[int | None, list] = OrderedDict()
for step in steps:
key = step.sub_polygon_index
if key not in groups:
groups[key] = []
groups[key].append(step)
for poly_idx, raw_group_steps in groups.items():
group_steps = raw_group_steps
# Reduce discarded steps within this group: keep only one
total_group_steps = len(group_steps)
discarded_indices = [i for i, s in enumerate(group_steps) if not s.selected]
discarded_to_remove = max(0, len(discarded_indices) - 1)
if discarded_to_remove > 0:
keep_discarded_idx = discarded_indices[0]
remove_set = set(discarded_indices[1:])
group_steps = [s for i, s in enumerate(group_steps) if i not in remove_set]
total_group_steps = len(group_steps)
discarded_label_idx = keep_discarded_idx
else:
discarded_label_idx = -1
if total_group_steps == 0:
continue
fig, axes = plt.subplots(nrows=total_group_steps, ncols=4, figsize=(20, 5 * total_group_steps))
geometry_to_cover = group_steps[0].geometry_to_cover
close_up_bbox = geometry_to_cover.bounds
full_bbox = (
gpd.GeoSeries([s.extract.geometry for s in group_steps], crs=4326)
.union_all()
.bounds
)
for ax_idx, step in enumerate(group_steps):
iou_metric = step.iou
is_selected = step.selected
is_redundant = not is_selected and step.reason == "redundant"
color = "C2" if is_selected else ("C8" if is_redundant else "C3")
status = "accepted" if is_selected else ("redundant" if is_redundant else "discarded")
if total_group_steps > 1:
combined_ax = axes[ax_idx, 0]
geometry_to_cover_ax = axes[ax_idx, 1]
intersection_geometry_ax = axes[ax_idx, 2]
extract_geometry_ax = axes[ax_idx, 3]
else:
combined_ax = axes[0]
geometry_to_cover_ax = axes[1]
intersection_geometry_ax = axes[2]
extract_geometry_ax = axes[3]
combined_ax.set_xlim([full_bbox[0], full_bbox[2]])
combined_ax.set_ylim([full_bbox[1], full_bbox[3]])
combined_ax.set_yticklabels([])
combined_ax.set_xticklabels([])
combined_ax.set_xticks([])
combined_ax.set_yticks([])
cx.add_basemap(combined_ax, source=cx.providers.Esri.WorldGrayCanvas, crs=4326)
for ax in (geometry_to_cover_ax, intersection_geometry_ax, extract_geometry_ax):
ax.set_xlim([close_up_bbox[0], close_up_bbox[2]])
ax.set_ylim([close_up_bbox[1], close_up_bbox[3]])
ax.set_axis_off()
cx.add_basemap(ax, source=cx.providers.Esri.WorldGrayCanvas, crs=4326)
combined_ax.set_title(step.extract.file_name)
prefix = f"[polygon {poly_idx}] " if poly_idx is not None else ""
if ax_idx == 0 and is_selected:
combined_ax.set_ylabel(f"{prefix}Step {ax_idx + 1} (always accepted)")
else:
label = f"{prefix}Step {ax_idx + 1} ({status})"
if ax_idx == discarded_label_idx and discarded_to_remove > 0:
label += f" (+ {discarded_to_remove} discarded)"
combined_ax.set_ylabel(label)
geometry_to_cover_ax.set_title("Geometry to cover")
extract_geometry_ax.set_title("Extract geometry close-up")
intersection_geometry_ax.set_title(f"Intersection (IoU: {iou_metric:.3g})")
extract_geometry = step.extract.geometry
intersection_geometry = step.intersection_geometry
gpd.GeoSeries([step.geometry_to_cover], crs=4326).plot(
ax=geometry_to_cover_ax, color="C0", alpha=0.8
)
gpd.GeoSeries([step.geometry_to_cover], crs=4326).plot(ax=combined_ax, color="C0", alpha=0.8)
gpd.GeoSeries([extract_geometry], crs=4326).plot(
ax=combined_ax,
color=color,
alpha=0.4,
zorder=1,
)
gpd.GeoSeries([extract_geometry], crs=4326).plot(
ax=extract_geometry_ax,
color=color,
alpha=0.4,
zorder=1,
)
gpd.GeoSeries([intersection_geometry], crs=4326).plot(
ax=combined_ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="C1",
linewidth=1.5,
)
gpd.GeoSeries([intersection_geometry], crs=4326).plot(
ax=intersection_geometry_ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="C1",
linewidth=1.5,
)
fig.suptitle(f"Polygon {poly_idx}" if poly_idx is not None else "Geometry coverage", fontsize=14)
fig.tight_layout()
plt.show()
def plot_features_with_geometry_filter(
features_gdf: gpd.GeoDataFrame, geometry_filter: Polygon
) -> None:
fig = plt.figure()
ax = fig.subplots()
close_up_bbox = geometry_filter.bounds
ax.set_xlim([close_up_bbox[0], close_up_bbox[2]])
ax.set_ylim([close_up_bbox[1], close_up_bbox[3]])
ax.set_axis_off()
cx.add_basemap(ax, source=cx.providers.Esri.WorldGrayCanvas, crs=4326)
features_gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.2)
features_gdf.boundary.plot(ax=ax, markersize=0, zorder=1, alpha=0.2)
gpd.GeoSeries([geometry_filter], crs=4326).plot(
ax=ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="orange",
linewidth=1.5,
)
blue_patch = mpatches.Patch(color="C0", alpha=0.8, label="OSM features")
orange_patch = mpatches.Patch(
facecolor=(0, 0, 0, 0),
edgecolor="orange",
hatch="///",
linewidth=1.5,
label="Geometry filter",
)
ax.legend(handles=[blue_patch, orange_patch], loc="lower right")
fig.tight_layout()
plt.show()
District within the city¶
Simple example of getting the data for the district of a city - here Monaco-Ville in Monaco
geometry_filter = geocode_to_geometry("Monaco-Ville, Monaco")
matched_extracts = find_extracts_by_geometry(geometry_filter)
plot_full_geometry_coverage_breakdown(matched_extracts)
matched_extracts
/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages/contextily/tile.py:680: UserWarning: The inferred zoom level of 17 is not valid for the current tile provider (valid zooms: 0 - 16). warnings.warn(msg)
OsmfinderGeometryResult
extracts:
Movisda-admin_MC — Monaco
coverage: 100.0%
iou threshold: 0.01
steps (1):
[polygon 0]:
Movisda-admin_MC — Monaco
iou: 0.0025, selected, first_extract
sources used: BBBike, GEO2Day, Geofabrik, Movisda-admin, Movisda-grid, osmfr
features_gdf = convert_geometry_to_geodataframe(geometry_filter)
plot_features_with_geometry_filter(features_gdf, geometry_filter)
features_gdf
Finished operation in 0:00:06
/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages/contextily/tile.py:680: UserWarning: The inferred zoom level of 17 is not valid for the current tile provider (valid zooms: 0 - 16). warnings.warn(msg)
| tags | geometry | |
|---|---|---|
| feature_id | ||
| relation/9516330 | {'alt_name:de': 'Pelagos-Schutzgebiet', 'alt_n... | POLYGON ((7.41852 43.73094, 7.41855 43.73091, ... |
| relation/10691624 | {'admin_level': '2', 'border_type': 'territori... | POLYGON ((7.41852 43.72476, 7.41901 43.72512, ... |
| relation/2220206 | {'ISO3166-2': 'MC-FO', 'admin_level': '10', 'b... | POLYGON ((7.41204 43.7281, 7.4121 43.72826, 7.... |
| way/398378955 | {'natural': 'coastline'} | LINESTRING (7.42282 43.72846, 7.42283 43.7286,... |
| way/398378956 | {'natural': 'coastline'} | LINESTRING (7.42259 43.72949, 7.42257 43.72923... |
| ... | ... | ... |
| relation/17990374 | {'alt_name:af': 'Côte d'Azur', 'alt_name:ar': ... | POLYGON ((7.41267 43.73163, 7.41268 43.73161, ... |
| relation/19317441 | {'boundary': 'natural', 'name': 'Préalpes de N... | POLYGON ((7.41852 43.72476, 7.41901 43.72512, ... |
| relation/2220322 | {'admin_level': '8', 'boundary': 'administrati... | MULTIPOLYGON (((7.41194 43.72815, 7.41162 43.7... |
| relation/7470949 | {'boundary': 'natural', 'name': 'Alpes Maritim... | POLYGON ((7.41849 43.73092, 7.41852 43.73094, ... |
| relation/2698607 | {'boundary': 'region', 'name': 'Alps', 'name:b... | POLYGON ((7.41849 43.73092, 7.41852 43.73094, ... |
1002 rows × 2 columns
Mismatch between Nominatim and Geofabrik extracts geometries¶
Sometimes the geometry returned by the Nominatim is different than the available extract geometry. Here you can see the deifference between Monaco geometry returned by the Nominatim geocoder (geometry getting far into the sea) vs Geofabrik extract (only land part of the Monaco country).
The algorithm also checked French region Provence-Alpes-Côte d'Azur extract that covers the sea region of the Monaco, but the Intersection over Union between the geometry filter and extract geometry is below default threshold (0.01), so it's discarded.
To force algorithm to download this region, user can set the IoU threshold to 0.
Selected extracts are coloured in green and discarded extracts are coloured in red.
geometry_filter = geocode_to_geometry("Monaco")
matched_extracts = find_extracts_by_geometry(
geometry_filter, source=OsmExtractSource.geofabrik, geometry_coverage_iou_threshold=0.01
)
plot_full_geometry_coverage_breakdown(matched_extracts)
matched_extracts
OsmfinderGeometryResult
extracts:
Geofabrik_monaco — monaco
coverage: 100.0%
iou threshold: 0.01
steps (1):
[polygon 0]:
Geofabrik_monaco — monaco
iou: 0.4330, selected, first_extract
sources used: Geofabrik
features_gdf = convert_geometry_to_geodataframe(geometry_filter)
plot_features_with_geometry_filter(features_gdf, geometry_filter)
features_gdf
/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages/osmfinder/finder/_covering.py:324: GeometryNotCoveredWarning: Discarded 19 extracts with IoU below the threshold of 0.01. warnings.warn(
Finished operation in 0:00:06
| tags | geometry | |
|---|---|---|
| feature_id | ||
| way/46428737 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (7.53299 43.53683, 7.53047 43.53527... |
| way/30837497 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (7.41852 43.72476, 7.41937 43.72259... |
| way/1563000023 | {'border_touching': 'border'} | LINESTRING (7.41852 43.72476, 7.41937 43.72259... |
| relation/9516330 | {'alt_name:de': 'Pelagos-Schutzgebiet', 'alt_n... | POLYGON ((7.41852 43.73094, 7.41855 43.73091, ... |
| relation/10691624 | {'admin_level': '2', 'border_type': 'territori... | POLYGON ((7.41852 43.72476, 7.41901 43.72512, ... |
| ... | ... | ... |
| way/128273234 | {'highway': 'residential', 'name': 'Lacets Sai... | LINESTRING (7.43237 43.74893, 7.4326 43.74902,... |
| way/981971419 | {'barrier': 'retaining_wall'} | LINESTRING (7.43234 43.74899, 7.43292 43.74921) |
| way/159175448 | {'highway': 'service', 'service': 'driveway'} | LINESTRING (7.43836 43.74932, 7.43836 43.74928... |
| way/1563000001 | {'landuse': 'residential', 'ref:osm': '1282732... | POLYGON ((7.43228 43.74906, 7.43298 43.74936, ... |
| way/279608089 | {'addr:country': 'MC', 'building': 'yes', 'nam... | POLYGON ((7.4334 43.74902, 7.43371 43.74922, 7... |
9315 rows × 2 columns
Multiple extracts to cover given geometry - Lake Constance¶
More complex example of covering the bounding box around Lake Constance - a close meetup of 4 different countries: Austria, Germany, Liechtenstein and Switzerland.
Here you can see that there are 4 extracts needed to cover the geometry.
Even though there are 5 steps to find extracts to cover the given geometry, after consolidating, only 4 are needed to be downloaded - bbbike_konstanz covers the same area as geo2day_europe_switzerland_thurgau, so this one file is skipped.
geometry_filter = from_wkt(
"POLYGON ((9.8 47.2, 9.8 47.6, 9.4 47.6, 9.4 47.2, 9.8 47.2))"
)
matched_extracts = find_extracts_by_geometry(geometry_filter)
plot_full_geometry_coverage_breakdown(matched_extracts)
matched_extracts
OsmfinderGeometryResult
extracts:
BBBike_Konstanz — Konstanz
osmfr_europe_austria_vorarlberg — vorarlberg
osmfr_europe_switzerland_saint_gallen — saint_gallen
Movisda-admin_LI — Liechtenstein
coverage: 100.0%
iou threshold: 0.01
steps (5):
[polygon 0]:
osmfr_europe_austria_vorarlberg — vorarlberg
iou: 0.1824, selected, first_extract
osmfr_europe_switzerland_saint_gallen — saint_gallen
iou: 0.1809, selected, selected
Movisda-admin_LI — Liechtenstein
iou: 0.0631, selected, selected
GEO2Day_europe_switzerland_thurgau — thurgau
iou: 0.0464, skipped, redundant
BBBike_Konstanz — Konstanz
iou: 0.0291, selected, selected
sources used: BBBike, GEO2Day, Geofabrik, Movisda-admin, Movisda-grid, osmfr
features_gdf = convert_geometry_to_geodataframe(geometry_filter)
plot_features_with_geometry_filter(features_gdf, geometry_filter)
features_gdf
Finished operation in 0:01:03
| tags | geometry | |
|---|---|---|
| feature_id | ||
| relation/9404214 | {'boat': 'yes', 'natural': 'water', 'type': 'm... | POLYGON ((9.47633 47.0495, 9.47564 47.05082, 9... |
| way/123503275 | {'ref:osm': '123503275'} | LINESTRING (9.47639 47.05188, 9.47636 47.05193... |
| way/1563000349 | {'border_touching': 'border'} | LINESTRING (9.47605 47.0518, 9.476 47.05184, 9... |
| relation/2128682 | {'name': 'Rätikon', 'name:cs': 'Rätikon', 'nam... | POLYGON ((9.49131 47.16297, 9.49332 47.15964, ... |
| relation/18459791 | {'name': 'Westliche Rätische Alpen (SOIUSA)', ... | POLYGON ((9.49131 47.16297, 9.49332 47.15964, ... |
| ... | ... | ... |
| way/69284046 | {'cycleway:left': 'no', 'cycleway:right': 'sep... | LINESTRING (9.73997 47.59763, 9.74078 47.59772... |
| way/17111109 | {'landuse': 'forest'} | POLYGON ((9.70375 47.59922, 9.70361 47.59921, ... |
| way/297102068 | {'electrified': 'contact_line', 'frequency': '... | LINESTRING (9.77652 47.61375, 9.77557 47.61313... |
| relation/959435 | {'admin_level': '8', 'boundary': 'administrati... | POLYGON ((9.74863 47.59662, 9.74842 47.59691, ... |
| relation/2539545 | {'landuse': 'meadow', 'meadow': 'agricultural'... | MULTIPOLYGON (((9.76272 47.6108, 9.76266 47.61... |
612841 rows × 2 columns
Multiple extracts to cover given geometry - US California¶
Another complex scenario - bounding box in the California US state, where 4 different extracts are used to fully coved a given geometry.
One extract is skipped, because of the low Intersection-over-Union metric.
geometry_filter = shape(
{
"type": "Polygon",
"coordinates": [
[
[-122.04132711751888, 39.47952317060893],
[-122.04132711751888, 38.586779237435586],
[-121.15664232594925, 38.586779237435586],
[-121.15664232594925, 39.47952317060893],
[-122.04132711751888, 39.47952317060893],
]
],
}
)
matched_extracts = find_extracts_by_geometry(geometry_filter)
plot_full_geometry_coverage_breakdown(matched_extracts)
matched_extracts
/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages/osmfinder/finder/_covering.py:324: GeometryNotCoveredWarning: Discarded 18 extracts with IoU below the threshold of 0.01. warnings.warn(
OsmfinderGeometryResult
extracts:
Movisda-grid_N38E122 — N38E122
Movisda-grid_N39E122 — N39E122
osmfr_north-america_us-west_california_colusa — colusa
osmfr_north-america_us-west_california_yolo — yolo
coverage: 99.8%
iou threshold: 0.01
steps (22):
[polygon 0]:
Movisda-grid_N39E122 — N39E122
iou: 0.2922, selected, first_extract
Movisda-grid_N38E122 — N38E122
iou: 0.3348, selected, selected
osmfr_north-america_us-west_california_colusa — colusa
iou: 0.0564, selected, selected
osmfr_north-america_us-west_california_yolo — yolo
iou: 0.0388, selected, selected
osmfr_north-america_us-west_california_glenn — glenn
iou: 0.0044, skipped, low_iou
Movisda-grid_N39E123 — N39E123
iou: 0.0018, skipped, low_iou
GEO2Day_north_america_us_california_norcal — norcal
iou: 0.0000, skipped, low_iou
Geofabrik_norcal — norcal
iou: 0.0000, skipped, low_iou
Movisda-admin_US-CA — California
iou: 0.0000, skipped, low_iou
Geofabrik_us/california — us_california
iou: 0.0000, skipped, low_iou
GEO2Day_north_america_us_california — california
iou: 0.0000, skipped, low_iou
osmfr_north-america_us-west_california — california
iou: 0.0000, skipped, low_iou
Movisda-grid_N30E130-10 — N30E130-10
iou: 0.0000, skipped, low_iou
osmfr_north-america_us-west — us-west
iou: 0.0000, skipped, low_iou
Geofabrik_us-west — us-west
iou: 0.0000, skipped, low_iou
GEO2Day_north_america_us-west — us-west
iou: 0.0000, skipped, low_iou
Movisda-admin_US — United States
iou: 0.0000, skipped, low_iou
GEO2Day_north_america_us — us
iou: 0.0000, skipped, low_iou
Geofabrik_us — us
iou: 0.0000, skipped, low_iou
osmfr_north-america — north-america
iou: 0.0000, skipped, low_iou
GEO2Day_north_america — north_america
iou: 0.0000, skipped, low_iou
Geofabrik_north-america — north-america
iou: 0.0000, skipped, low_iou
sources used: BBBike, GEO2Day, Geofabrik, Movisda-admin, Movisda-grid, osmfr
features_gdf = convert_geometry_to_geodataframe(geometry_filter)
plot_features_with_geometry_filter(features_gdf, geometry_filter)
features_gdf
/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages/osmfinder/finder/_covering.py:324: GeometryNotCoveredWarning: Discarded 18 extracts with IoU below the threshold of 0.01. warnings.warn(
Finished operation in 0:00:58
| tags | geometry | |
|---|---|---|
| feature_id | ||
| way/1091086155 | {'layer': '-1', 'location': 'underground', 'ma... | LINESTRING (-122 38.60177, -121.99589 38.59164... |
| way/24049489 | {'cables': '3', 'frequency': '60', 'name': 'Ol... | LINESTRING (-122 38.77919, -121.99883 38.77758... |
| way/79293843 | {'intermittent': 'no', 'name': 'Sacramento Riv... | LINESTRING (-121.62659 38.67327, -121.62477 38... |
| relation/19861019 | {'attribution': 'CASIL CSP_Opbdys072008', 'bou... | MULTIPOLYGON (((-121.51906 38.36737, -121.5189... |
| way/92633592 | {'cables': '3', 'name': 'Vaca-Plainfield', 'op... | LINESTRING (-121.92105 38.40513, -121.92137 38... |
| ... | ... | ... |
| way/10749306 | {'highway': 'track', 'tiger:cfcc': 'A41', 'tig... | LINESTRING (-121.15042 39.47781, -121.15084 39... |
| way/10749248 | {'highway': 'service', 'tiger:cfcc': 'A41', 't... | LINESTRING (-121.26875 39.47854, -121.26996 39... |
| way/10751359 | {'highway': 'track', 'name': 'Peterson Ridge R... | LINESTRING (-121.16108 39.48705, -121.16196 39... |
| way/10748357 | {'highway': 'track', 'tiger:cfcc': 'A41', 'tig... | LINESTRING (-121.18042 39.4793, -121.18054 39.... |
| way/10751813 | {'highway': 'track', 'name': 'Bean Clipper Roa... | LINESTRING (-121.14361 39.52467, -121.14361 39... |
734607 rows × 2 columns