QuackOSM Basic Usage¶
QuackOSM exposes some basic functions in the main Python module. Full documentation for them is available here.
This notebook will show how to use the library in a few simple scenarios.
To learn more about CLI usage, see this page. The help page for the CLI is available here.
To learn more details about PbfFileReader class, see this page, or documentation page.
Run this notebook in Google Colab:
Remember to install the quackosm library before running the notebook:
%pip install quackosm[cli]
In [1]:
Copied!
import quackosm as qosm
import quackosm as qosm
Reading existing PBF file to GeoDataFrame¶
In [2]:
Copied!
qosm.convert_pbf_to_geodataframe("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
qosm.convert_pbf_to_geodataframe("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
Finished operation in 0:00:05
Out[2]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| way/1225956054 | {'name': '7.5 degrees east'} | LINESTRING (7.5 43.5164, 7.5 37.26898) |
| way/161191598 | {'admin_level': '2', 'border_type': 'contiguou... | LINESTRING (7.50024 43.51654, 7.50109 43.51438... |
| way/773124328 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (7.5 43.5164, 7.49789 43.51525, 7.4... |
| way/46428446 | {'border_type': 'contiguous', 'boundary': 'mar... | LINESTRING (7.53299 43.53683, 7.53384 43.53467... |
| way/1225956053 | {'admin_level': '2', 'border_type': 'territori... | LINESTRING (7.50024 43.51654, 7.5 43.5164) |
| ... | ... | ... |
| node/254667935 | {'crossing': 'marked', 'crossing:markings': 'z... | POINT (7.43955 43.75317) |
| node/1523103399 | {'bus': 'yes', 'name': 'Apollon', 'public_tran... | POINT (7.4397 43.75326) |
| node/254667939 | {'crossing': 'marked', 'crossing:markings': 'z... | POINT (7.44082 43.75375) |
| node/8722523691 | {'bus': 'yes', 'name': 'Chemin des Grottes', '... | POINT (7.44088 43.75377) |
| way/37794466 | {'admin_level': '8', 'boundary': 'administrati... | LINESTRING (7.42661 43.75948, 7.42662 43.75923... |
9697 rows × 2 columns
Transforming existing PBF file to GeoParquet¶
In [3]:
Copied!
qosm.convert_pbf_to_parquet("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
qosm.convert_pbf_to_parquet("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
Out[3]:
PosixPath('files/monaco-latest_nofilter_noclip_compact_sorted.parquet')
In [4]:
Copied!
qosm.convert_osm_extract_to_geodataframe("Vatican City")
qosm.convert_osm_extract_to_geodataframe("Vatican City")
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/quackosm/osm_extracts/__init__.py:344: OsmExtractMultipleMatchesWarning: Multiple extracts matched by query "Vatican City" (matching full names: "geo2day_europe_vatican_city", "movisda-admin_vatican_city", "osmfr_europe_vatican_city"). Selected "movisda-admin_vatican_city". Use the full name as a query or set `select_first_match=False` to control this behaviour. warnings.warn(
Finished operation in 0:00:05
Out[4]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| node/13050566010 | {'natural': 'tree'} | POINT (12.45417 41.90029) |
| node/13050566009 | {'natural': 'tree'} | POINT (12.45409 41.90029) |
| node/13050566008 | {'natural': 'tree'} | POINT (12.454 41.90031) |
| node/13050566007 | {'natural': 'tree'} | POINT (12.45393 41.90031) |
| way/1408206941 | {'building': 'yes'} | POLYGON ((12.45407 41.90035, 12.45407 41.90036... |
| ... | ... | ... |
| way/1017966595 | {'access': 'private', 'leisure': 'playground'} | POLYGON ((12.45481 41.90717, 12.45485 41.90709... |
| way/1415382584 | {'building': 'roof', 'layer': '1'} | POLYGON ((12.45486 41.90717, 12.45489 41.90717... |
| way/1017966594 | {'building': 'yes', 'ref:osm': '1017966594'} | POLYGON ((12.4549 41.9072, 12.45489 41.90722, ... |
| node/522810848 | {'highway': 'turning_circle'} | POINT (12.45537 41.90725) |
| way/111733624 | {'building': 'yes'} | POLYGON ((12.45516 41.90732, 12.4552 41.90714,... |
3732 rows × 2 columns
Find an OSM PBF extract file using text and transform it to a GeoParquet¶
In [5]:
Copied!
qosm.convert_osm_extract_to_parquet("Vatican City")
qosm.convert_osm_extract_to_parquet("Vatican City")
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/quackosm/osm_extracts/__init__.py:344: OsmExtractMultipleMatchesWarning: Multiple extracts matched by query "Vatican City" (matching full names: "geo2day_europe_vatican_city", "movisda-admin_vatican_city", "osmfr_europe_vatican_city"). Selected "movisda-admin_vatican_city". Use the full name as a query or set `select_first_match=False` to control this behaviour. warnings.warn(
Out[5]:
PosixPath('files/movisda-admin_vatican_city_nofilter_noclip_compact_sorted.parquet')
Get OSM data for a given geometry as a GeoDataFrame¶
In [6]:
Copied!
area = qosm.geocode_to_geometry("Songpa-gu, Seoul")
qosm.convert_geometry_to_geodataframe(area)
area = qosm.geocode_to_geometry("Songpa-gu, Seoul")
qosm.convert_geometry_to_geodataframe(area)
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/quackosm/osm_extracts/__init__.py:950: GeometryNotCoveredWarning: Skipping extract because of low IoU value (bbbike_seoul, 0.00023). warnings.warn(
Finished operation in 0:00:10
Out[6]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| way/180676693 | {'admin_level': '4', 'boundary': 'administrati... | LINESTRING (127.09645 37.45648, 127.0983 37.45... |
| relation/13773715 | {'boundary': 'postal_code', 'name': '13107', '... | MULTIPOLYGON (((127.11691 37.45929, 127.1169 3... |
| way/1532000846 | {'border_touching': 'border'} | LINESTRING (127.11994 37.46342, 127.11999 37.4... |
| way/440415895 | {'bridge': 'yes', 'bridge:name': '대왕교', 'bridg... | LINESTRING (127.12198 37.46686, 127.12281 37.4... |
| way/1532000124 | {'highway': 'secondary', 'layer': '-3', 'name'... | LINESTRING (127.12509 37.46849, 127.12497 37.4... |
| ... | ... | ... |
| relation/3769500 | {'alt_name:en': 'Hangang River', 'name': '한강',... | MULTIPOLYGON (((127.11517 37.55648, 127.11561 ... |
| way/531415891 | {'bicycle': 'designated', 'highway': 'cycleway... | LINESTRING (127.11621 37.54191, 127.11628 37.5... |
| relation/152336 | {'alt_name:en': 'Hangang River', 'name': '한강',... | POLYGON ((126.80183 37.60433, 126.80259 37.605... |
| relation/2297418 | {'ISO3166-2': 'KR-11', 'admin_level': '4', 'al... | POLYGON ((126.76622 37.55424, 126.76636 37.553... |
| relation/13333338 | {'alt_name': '한강 이북 서울', 'boundary': 'region',... | POLYGON ((126.85363 37.57179, 126.8542 37.5710... |
27123 rows × 2 columns
Save OSM data for a given geometry as a GeoParquet¶
In [7]:
Copied!
qosm.convert_geometry_to_parquet(area)
qosm.convert_geometry_to_parquet(area)
Out[7]:
PosixPath('files/95be730f_nofilter_compact_sorted.parquet')
More advanced examples¶
Filter out data based on geometry from existing PBF file¶
In [8]:
Copied!
area = qosm.geocode_to_geometry("Monaco-Ville, Monaco")
gdf = qosm.convert_pbf_to_geodataframe(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", geometry_filter=area
)
gdf
area = qosm.geocode_to_geometry("Monaco-Ville, Monaco")
gdf = qosm.convert_pbf_to_geodataframe(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", geometry_filter=area
)
gdf
Finished operation in 0:00:06
Out[8]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| relation/10691624 | {'admin_level': '2', 'border_type': 'territori... | POLYGON ((7.41852 43.72476, 7.41901 43.72512, ... |
| relation/2220206 | {'ISO3166-2': 'MC-FO', 'admin_level': '10', 'b... | POLYGON ((7.41204 43.7281, 7.4121 43.72826, 7.... |
| way/398378955 | {'natural': 'coastline'} | LINESTRING (7.42282 43.72846, 7.42283 43.7286,... |
| way/398378956 | {'natural': 'coastline'} | LINESTRING (7.42259 43.72949, 7.42257 43.72923... |
| way/1388228527 | {'man_made': 'breakwater', 'material': 'rock'} | POLYGON ((7.4228 43.72941, 7.42269 43.72945, 7... |
| ... | ... | ... |
| way/1089844226 | {'handrail': 'yes', 'highway': 'steps', 'incli... | LINESTRING (7.42795 43.73294, 7.42791 43.73291... |
| way/1089844259 | {'highway': 'steps', 'incline': 'down'} | LINESTRING (7.42767 43.73287, 7.42775 43.73288... |
| way/1089844299 | {'highway': 'steps', 'incline': 'down'} | LINESTRING (7.42782 43.73295, 7.42782 43.73292... |
| relation/2221178 | {'ISO3166-2': 'MC-CO', 'admin_level': '10', 'b... | POLYGON ((7.41588 43.7314, 7.41592 43.73136, 7... |
| relation/2220322 | {'admin_level': '8', 'boundary': 'administrati... | MULTIPOLYGON (((7.41194 43.72815, 7.41162 43.7... |
933 rows × 2 columns
Plot downloaded data
In [9]:
Copied!
import geopandas as gpd
import matplotlib.patches as mpatches
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(10, 10))
ax = fig.subplots()
gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.4)
gdf.boundary.plot(ax=ax, markersize=0, zorder=1, alpha=0.8)
gpd.GeoSeries([area], crs=4326).plot(
ax=ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="orange",
linewidth=1.5,
)
blue_patch = mpatches.Patch(color="C0", alpha=0.8, label="OSM features")
orange_patch = mpatches.Patch(
facecolor=(0, 0, 0, 0), edgecolor="orange", hatch="///", linewidth=1.5, label="Geometry filter"
)
ax.legend(handles=[blue_patch, orange_patch], loc="lower right")
plt.show()
import geopandas as gpd
import matplotlib.patches as mpatches
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(10, 10))
ax = fig.subplots()
gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.4)
gdf.boundary.plot(ax=ax, markersize=0, zorder=1, alpha=0.8)
gpd.GeoSeries([area], crs=4326).plot(
ax=ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="orange",
linewidth=1.5,
)
blue_patch = mpatches.Patch(color="C0", alpha=0.8, label="OSM features")
orange_patch = mpatches.Patch(
facecolor=(0, 0, 0, 0), edgecolor="orange", hatch="///", linewidth=1.5, label="Geometry filter"
)
ax.legend(handles=[blue_patch, orange_patch], loc="lower right")
plt.show()
In [10]:
Copied!
area = qosm.geocode_to_geometry("Barcelona")
gdf = qosm.convert_geometry_to_geodataframe(
area, tags_filter={"amenity": "bicycle_rental"}
)
gdf
area = qosm.geocode_to_geometry("Barcelona")
gdf = qosm.convert_geometry_to_geodataframe(
area, tags_filter={"amenity": "bicycle_rental"}
)
gdf
Finished operation in 0:00:09
Out[10]:
| amenity | geometry | |
|---|---|---|
| feature_id | ||
| node/10130905487 | bicycle_rental | POINT (2.14355 41.34662) |
| node/11306277559 | bicycle_rental | POINT (2.14504 41.351) |
| node/11306326521 | bicycle_rental | POINT (2.14364 41.35397) |
| node/4297576789 | bicycle_rental | POINT (2.14166 41.35715) |
| node/3859732557 | bicycle_rental | POINT (2.13716 41.35735) |
| ... | ... | ... |
| node/11079312169 | bicycle_rental | POINT (2.19859 41.44813) |
| node/11055410322 | bicycle_rental | POINT (2.19883 41.44823) |
| node/10130805220 | bicycle_rental | POINT (2.18308 41.44912) |
| node/5262548992 | bicycle_rental | POINT (2.19098 41.44987) |
| node/5262548991 | bicycle_rental | POINT (2.19267 41.45088) |
516 rows × 2 columns
Show downloaded data on a map
In [11]:
Copied!
m = gdf.explore(color="orangered", tiles="CartoDB positron")
gpd.GeoSeries([area], crs=4326).boundary.explore(m=m)
m = gdf.explore(color="orangered", tiles="CartoDB positron")
gpd.GeoSeries([area], crs=4326).boundary.explore(m=m)
Out[11]:
Make this Notebook Trusted to load map: File -> Trust Notebook
Save the result GeoParquet with WKT geometry¶
In [12]:
Copied!
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf",
save_as_wkt=True,
sort_result=False, # sorting is disabled for wkt output
)
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf",
save_as_wkt=True,
sort_result=False, # sorting is disabled for wkt output
)
Finished operation in 0:00:05
Out[12]:
PosixPath('files/monaco-latest_nofilter_noclip_compact_wkt.parquet')
Specify result file path¶
In [13]:
Copied!
qosm.convert_geometry_to_parquet(
area, result_file_path="barcelona_osm_output.parquet"
)
qosm.convert_geometry_to_parquet(
area, result_file_path="barcelona_osm_output.parquet"
)
Finished operation in 0:00:23
Out[13]:
PosixPath('barcelona_osm_output.parquet')
Force recalculation of the result¶
By default, running the same command twice will result in reusing the saved GeoParquet file. You can force QuackOSM to recalculate the data.
In [14]:
Copied!
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", ignore_cache=True
)
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", ignore_cache=True
)
Finished operation in 0:00:05
Out[14]:
PosixPath('files/monaco-latest_nofilter_noclip_compact_sorted.parquet')
Result file sorting¶
By default, QuackOSM sorts the result file by geometry using Hilbert curve to make it smaller. It adds some time to the overall execution, but can significantly reduce the file size.
Sorting can be disabled by the user.
In [15]:
Copied!
unsorted_pq = qosm.convert_geometry_to_parquet(
area, tags_filter={"building": True}, sort_result=False
)
unsorted_pq = qosm.convert_geometry_to_parquet(
area, tags_filter={"building": True}, sort_result=False
)
Finished operation in 0:00:13
In [16]:
Copied!
sorted_pq = qosm.convert_geometry_to_parquet(
area, tags_filter={"building": True}, sort_result=True
)
sorted_pq = qosm.convert_geometry_to_parquet(
area, tags_filter={"building": True}, sort_result=True
)
Finished operation in 0:00:15
In [17]:
Copied!
unsorted_pq, sorted_pq
unsorted_pq, sorted_pq
Out[17]:
(PosixPath('files/41b45844_ae99e3d9_exploded.parquet'),
PosixPath('files/41b45844_ae99e3d9_exploded_sorted.parquet'))
In [18]:
Copied!
import geopandas as gpd
from matplotlib import pyplot as plt
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(20, 10))
gpd.read_parquet(unsorted_pq).reset_index().reset_index().plot(
column="index", ax=ax1, cmap="jet", markersize=1
)
gpd.read_parquet(sorted_pq).reset_index().reset_index().plot(
column="index", ax=ax2, cmap="jet", markersize=1
)
unsorted_size = unsorted_pq.stat().st_size
sorted_size = sorted_pq.stat().st_size
ax1.set_title(f"Unsorted: {unsorted_size} bytes")
ax2.set_title(
f"Sorted: {sorted_size} bytes ({100 - (100 * sorted_size) / unsorted_size:.2f}% reduction)"
)
plt.show()
import geopandas as gpd
from matplotlib import pyplot as plt
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(20, 10))
gpd.read_parquet(unsorted_pq).reset_index().reset_index().plot(
column="index", ax=ax1, cmap="jet", markersize=1
)
gpd.read_parquet(sorted_pq).reset_index().reset_index().plot(
column="index", ax=ax2, cmap="jet", markersize=1
)
unsorted_size = unsorted_pq.stat().st_size
sorted_size = sorted_pq.stat().st_size
ax1.set_title(f"Unsorted: {unsorted_size} bytes")
ax2.set_title(
f"Sorted: {sorted_size} bytes ({100 - (100 * sorted_size) / unsorted_size:.2f}% reduction)"
)
plt.show()
