QuackOSM Basic Usage¶
QuackOSM exposes some basic functions in the main Python module. Full documentation for them is available here.
This notebook will show how to use the library in a few simple scenarios.
To learn more about CLI usage, see this page. The help page for the CLI is available here.
To learn more details about PbfFileReader class, see this page, or documentation page.
In [1]:
Copied!
import quackosm as qosm
import quackosm as qosm
Reading existing PBF file to GeoDataFrame¶
In [2]:
Copied!
qosm.convert_pbf_to_geodataframe("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
qosm.convert_pbf_to_geodataframe("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
0%| | 0.00/549k [00:00<?, ?B/s]
1%|▌ | 8.19k/549k [00:00<00:08, 64.6kB/s]
7%|██▌ | 36.9k/549k [00:00<00:03, 157kB/s]
17%|██████▋ | 94.2k/549k [00:00<00:01, 286kB/s]
38%|███████████████▏ | 209k/549k [00:00<00:00, 520kB/s]
79%|███████████████████████████████▋ | 434k/549k [00:00<00:00, 954kB/s]
0%| | 0.00/549k [00:00<?, ?B/s]
100%|███████████████████████████████████████| 549k/549k [00:00<00:00, 1.24GB/s]
Finished operation in 0:00:05
Out[2]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| node/21911886 | {'crossing': 'zebra', 'crossing:island': 'no',... | POINT (7.4235 43.73724) |
| node/21912962 | {'crossing': 'zebra', 'crossing_ref': 'zebra',... | POINT (7.42691 43.73791) |
| node/21914341 | {'crossing': 'uncontrolled', 'crossing_ref': '... | POINT (7.42337 43.73701) |
| node/21915639 | {'highway': 'traffic_signals'} | POINT (7.4256 43.74044) |
| node/21917308 | {'bus': 'yes', 'name': 'Monte-Carlo (Casino)',... | POINT (7.42599 43.74098) |
| ... | ... | ... |
| node/8721086461 | {'natural': 'tree'} | POINT (7.42192 43.73337) |
| node/8721086462 | {'natural': 'tree'} | POINT (7.42188 43.73346) |
| node/8721086463 | {'natural': 'tree'} | POINT (7.42149 43.73444) |
| node/8721086464 | {'natural': 'tree'} | POINT (7.42148 43.73453) |
| node/8721086465 | {'natural': 'tree'} | POINT (7.42146 43.73462) |
8175 rows × 2 columns
Transforming existing PBF file to GeoParquet¶
In [3]:
Copied!
qosm.convert_pbf_to_parquet("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
qosm.convert_pbf_to_parquet("https://download.geofabrik.de/europe/monaco-latest.osm.pbf")
Out[3]:
PosixPath('files/monaco-latest_nofilter_noclip_compact.parquet')
In [4]:
Copied!
qosm.convert_osm_extract_to_geodataframe("Vatican City")
qosm.convert_osm_extract_to_geodataframe("Vatican City")
Out[4]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| node/1360222286 | {'emergency': 'phone'} | POINT (12.44855 41.90663) |
| node/1362791662 | {'emergency': 'phone'} | POINT (12.44726 41.90727) |
| node/1367095388 | {'emergency': 'phone'} | POINT (12.44754 41.90704) |
| node/1367095390 | {'emergency': 'phone'} | POINT (12.44803 41.90737) |
| node/1383456269 | {'emergency': 'fire_hydrant', 'fire_hydrant:po... | POINT (12.4576 41.90917) |
| ... | ... | ... |
| node/10610749830 | {'barrier': 'bollard', 'bollard': 'fixed', 'co... | POINT (12.45718 41.90228) |
| node/10610755636 | {'highway': 'street_lamp', 'lamp_mount': 'pole... | POINT (12.45746 41.90239) |
| node/10610755647 | {'highway': 'street_lamp', 'lamp_mount': 'pole... | POINT (12.45707 41.90208) |
| node/10610755653 | {'highway': 'street_lamp', 'lamp_mount': 'pole... | POINT (12.45747 41.90209) |
| node/10610755660 | {'highway': 'street_lamp', 'lamp_mount': 'pole... | POINT (12.45705 41.90239) |
8473 rows × 2 columns
Find an OSM PBF extract file using text and transform it to a GeoParquet¶
In [5]:
Copied!
qosm.convert_osm_extract_to_parquet("Vatican City")
qosm.convert_osm_extract_to_parquet("Vatican City")
Out[5]:
PosixPath('files/osmfr_europe_vatican_city_nofilter_noclip_compact.parquet')
Get OSM data for a given geometry as a GeoDataFrame¶
In [6]:
Copied!
area = qosm.geocode_to_geometry('Shibuya, Tokyo')
qosm.convert_geometry_to_geodataframe(area)
area = qosm.geocode_to_geometry('Shibuya, Tokyo')
qosm.convert_geometry_to_geodataframe(area)
0%| | 0.00/51.8M [00:00<?, ?B/s]
0%| | 32.8k/51.8M [00:00<02:52, 301kB/s]
0%| | 81.9k/51.8M [00:00<02:14, 386kB/s]
0%|▏ | 197k/51.8M [00:00<01:15, 685kB/s]
1%|▎ | 426k/51.8M [00:00<00:41, 1.24MB/s]
2%|▋ | 885k/51.8M [00:00<00:22, 2.29MB/s]
3%|█▎ | 1.80M/51.8M [00:00<00:11, 4.34MB/s]
7%|██▌ | 3.62M/51.8M [00:00<00:05, 8.31MB/s]
12%|████▎ | 5.96M/51.8M [00:00<00:03, 12.4MB/s]
15%|█████▋ | 8.01M/51.8M [00:00<00:03, 14.4MB/s]
20%|███████▍ | 10.4M/51.8M [00:01<00:02, 16.7MB/s]
25%|█████████▎ | 13.0M/51.8M [00:01<00:02, 18.6MB/s]
30%|██████████▉ | 15.4M/51.8M [00:01<00:01, 19.7MB/s]
34%|████████████▌ | 17.6M/51.8M [00:01<00:01, 19.7MB/s]
39%|██████████████▎ | 20.0M/51.8M [00:01<00:01, 20.5MB/s]
44%|████████████████ | 22.6M/51.8M [00:01<00:01, 21.3MB/s]
48%|█████████████████▉ | 25.1M/51.8M [00:01<00:01, 21.7MB/s]
53%|███████████████████▋ | 27.6M/51.8M [00:01<00:01, 22.1MB/s]
58%|█████████████████████▌ | 30.2M/51.8M [00:01<00:00, 22.6MB/s]
63%|███████████████████████▍ | 32.8M/51.8M [00:02<00:00, 22.8MB/s]
68%|█████████████████████████▎ | 35.4M/51.8M [00:02<00:00, 23.1MB/s]
73%|███████████████████████████▏ | 38.0M/51.8M [00:02<00:00, 23.3MB/s]
79%|█████████████████████████████ | 40.7M/51.8M [00:02<00:00, 23.6MB/s]
84%|██████████████████████████████▉ | 43.4M/51.8M [00:02<00:00, 23.7MB/s]
89%|████████████████████████████████▉ | 46.1M/51.8M [00:02<00:00, 24.1MB/s]
94%|██████████████████████████████████▊ | 48.8M/51.8M [00:02<00:00, 24.2MB/s]
99%|████████████████████████████████████▊| 51.6M/51.8M [00:02<00:00, 24.4MB/s]
0%| | 0.00/51.8M [00:00<?, ?B/s]
100%|█████████████████████████████████████| 51.8M/51.8M [00:00<00:00, 94.7GB/s]
Finished operation in 0:00:16
Out[6]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| node/366856009 | {'junction': 'yes', 'name': '神泉町', 'name:en': ... | POINT (139.69342 35.65391) |
| node/366856257 | {'junction': 'yes', 'name': '神泉町', 'name:en': ... | POINT (139.69325 35.65412) |
| node/366856264 | {'bicycle': 'designated', 'button_operated': '... | POINT (139.69335 35.6542) |
| node/366856763 | {'highway': 'traffic_signals', 'traffic_signal... | POINT (139.69361 35.65403) |
| node/380462678 | {'highway': 'traffic_signals', 'name': '笹塚一丁目東... | POINT (139.67122 35.67393) |
| ... | ... | ... |
| way/1123880557 | {'building': 'roof', 'layer': '1'} | POLYGON ((139.71894 35.65413, 139.71896 35.654... |
| way/1123880558 | {'footway': 'sidewalk', 'highway': 'footway'} | LINESTRING (139.71859 35.65419, 139.71904 35.6... |
| way/1123880559 | {'highway': 'footway'} | LINESTRING (139.71904 35.65419, 139.71904 35.6... |
| way/1124142895 | {'highway': 'tertiary_link', 'lane_markings': ... | LINESTRING (139.69087 35.67042, 139.69081 35.6... |
| way/1124142896 | {'highway': 'tertiary', 'lanes': '2', 'maxspee... | LINESTRING (139.69131 35.66849, 139.69146 35.6... |
49086 rows × 2 columns
Save OSM data for a given geometry as a GeoParquet¶
In [7]:
Copied!
qosm.convert_geometry_to_parquet(area)
qosm.convert_geometry_to_parquet(area)
Out[7]:
PosixPath('files/9754fc89be2f46cd7cc5ff3d09058a18133fe9cc2a93abe3276de276c6ca0a6c_nofilter_compact.parquet')
More advanced examples¶
Filter out data based on geometry from existing PBF file¶
In [8]:
Copied!
area = qosm.geocode_to_geometry("Monaco-Ville, Monaco")
gdf = qosm.convert_pbf_to_geodataframe(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", geometry_filter=area
)
gdf
area = qosm.geocode_to_geometry("Monaco-Ville, Monaco")
gdf = qosm.convert_pbf_to_geodataframe(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", geometry_filter=area
)
gdf
Finished operation in 0:00:06
Out[8]:
| tags | geometry | |
|---|---|---|
| feature_id | ||
| node/1573123086 | {'crossing': 'marked', 'crossing:markings': 'y... | POINT (7.42238 43.73214) |
| node/1662811584 | {'description': 'Mène à la gallerie commercial... | POINT (7.41797 43.7317) |
| node/1702431977 | {'addr:country': 'MC', 'amenity': 'restaurant'... | POINT (7.42561 43.73063) |
| node/1702431980 | {'hiking': 'yes', 'information': 'guidepost', ... | POINT (7.42495 43.73071) |
| node/1702431983 | {'tourism': 'viewpoint'} | POINT (7.42606 43.73088) |
| ... | ... | ... |
| node/7932373266 | {'board_type': 'history', 'description': '1970... | POINT (7.42351 43.73095) |
| node/7932373267 | {'fee': 'yes', 'name': 'Musée de la Chapelle d... | POINT (7.42542 43.73146) |
| node/8408663240 | {'noexit': 'yes'} | POINT (7.42052 43.73216) |
| node/8639732899 | {'amenity': 'bench'} | POINT (7.41833 43.73188) |
| node/8639732906 | {'crossing': 'traffic_signals', 'highway': 'cr... | POINT (7.41834 43.73192) |
919 rows × 2 columns
Plot downloaded data
In [9]:
Copied!
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import geopandas as gpd
fig = plt.figure(figsize=(10, 10))
ax = fig.subplots()
gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.4)
gdf.boundary.plot(ax=ax, markersize=0, zorder=1, alpha=0.8)
gpd.GeoSeries([area], crs=4326).plot(
ax=ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="orange",
linewidth=1.5,
)
blue_patch = mpatches.Patch(color="C0", alpha=0.8, label="OSM features")
orange_patch = mpatches.Patch(
facecolor=(0, 0, 0, 0), edgecolor="orange", hatch="///", linewidth=1.5, label="Geometry filter"
)
ax.legend(handles=[blue_patch, orange_patch], loc="lower right")
plt.show()
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import geopandas as gpd
fig = plt.figure(figsize=(10, 10))
ax = fig.subplots()
gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.4)
gdf.boundary.plot(ax=ax, markersize=0, zorder=1, alpha=0.8)
gpd.GeoSeries([area], crs=4326).plot(
ax=ax,
color=(0, 0, 0, 0),
zorder=2,
hatch="///",
edgecolor="orange",
linewidth=1.5,
)
blue_patch = mpatches.Patch(color="C0", alpha=0.8, label="OSM features")
orange_patch = mpatches.Patch(
facecolor=(0, 0, 0, 0), edgecolor="orange", hatch="///", linewidth=1.5, label="Geometry filter"
)
ax.legend(handles=[blue_patch, orange_patch], loc="lower right")
plt.show()
In [10]:
Copied!
area = qosm.geocode_to_geometry("Barcelona")
gdf = qosm.convert_geometry_to_geodataframe(
area, tags_filter={"amenity": "bicycle_rental"}
)
gdf
area = qosm.geocode_to_geometry("Barcelona")
gdf = qosm.convert_geometry_to_geodataframe(
area, tags_filter={"amenity": "bicycle_rental"}
)
gdf
0%| | 0.00/49.9M [00:00<?, ?B/s]
0%| | 32.8k/49.9M [00:00<02:45, 301kB/s]
0%| | 81.9k/49.9M [00:00<02:08, 387kB/s]
0%|▏ | 197k/49.9M [00:00<01:12, 687kB/s]
1%|▎ | 426k/49.9M [00:00<00:39, 1.24MB/s]
2%|▋ | 885k/49.9M [00:00<00:21, 2.29MB/s]
4%|█▎ | 1.80M/49.9M [00:00<00:11, 4.35MB/s]
7%|██▋ | 3.62M/49.9M [00:00<00:05, 8.32MB/s]
12%|████▍ | 5.91M/49.9M [00:00<00:03, 12.3MB/s]
16%|█████▉ | 8.01M/49.9M [00:00<00:02, 14.4MB/s]
21%|███████▌ | 10.3M/49.9M [00:01<00:02, 16.3MB/s]
25%|█████████▏ | 12.4M/49.9M [00:01<00:02, 17.3MB/s]
29%|██████████▊ | 14.7M/49.9M [00:01<00:01, 18.3MB/s]
34%|████████████▍ | 16.9M/49.9M [00:01<00:01, 18.7MB/s]
38%|██████████████▏ | 19.1M/49.9M [00:01<00:01, 19.2MB/s]
43%|███████████████▊ | 21.3M/49.9M [00:01<00:01, 19.6MB/s]
47%|█████████████████▍ | 23.6M/49.9M [00:01<00:01, 19.8MB/s]
52%|███████████████████▏ | 25.9M/49.9M [00:01<00:01, 20.2MB/s]
56%|████████████████████▊ | 28.2M/49.9M [00:01<00:01, 20.3MB/s]
61%|██████████████████████▌ | 30.5M/49.9M [00:02<00:00, 20.5MB/s]
66%|████████████████████████▎ | 32.8M/49.9M [00:02<00:00, 20.7MB/s]
70%|██████████████████████████ | 35.2M/49.9M [00:02<00:00, 20.8MB/s]
75%|███████████████████████████▊ | 37.5M/49.9M [00:02<00:00, 20.9MB/s]
80%|█████████████████████████████▌ | 39.8M/49.9M [00:02<00:00, 21.0MB/s]
84%|███████████████████████████████▎ | 42.2M/49.9M [00:02<00:00, 21.2MB/s]
89%|█████████████████████████████████ | 44.6M/49.9M [00:02<00:00, 21.2MB/s]
94%|██████████████████████████████████▊ | 47.0M/49.9M [00:02<00:00, 21.4MB/s]
99%|████████████████████████████████████▌| 49.3M/49.9M [00:02<00:00, 21.5MB/s]
0%| | 0.00/49.9M [00:00<?, ?B/s]
100%|█████████████████████████████████████| 49.9M/49.9M [00:00<00:00, 91.6GB/s]
Finished operation in 0:00:12
Out[10]:
| amenity | geometry | |
|---|---|---|
| feature_id | ||
| node/704038070 | bicycle_rental | POINT (2.17025 41.38605) |
| node/1867549305 | bicycle_rental | POINT (2.17432 41.39076) |
| node/1867549334 | bicycle_rental | POINT (2.17064 41.39699) |
| node/1867549338 | bicycle_rental | POINT (2.16664 41.3986) |
| node/1867549345 | bicycle_rental | POINT (2.1835 41.38909) |
| ... | ... | ... |
| node/1727175387 | bicycle_rental | POINT (2.17839 41.42693) |
| node/2572157092 | bicycle_rental | POINT (2.18833 41.37862) |
| node/4823518421 | bicycle_rental | POINT (2.18145 41.42003) |
| node/5060672224 | bicycle_rental | POINT (2.17343 41.37527) |
| node/6218031779 | bicycle_rental | POINT (2.18428 41.43246) |
515 rows × 2 columns
Show downloaded data on a map
In [11]:
Copied!
m = gdf.explore(color="orangered", tiles="CartoDB positron")
gpd.GeoSeries([area], crs=4326).boundary.explore(m=m)
m = gdf.explore(color="orangered", tiles="CartoDB positron")
gpd.GeoSeries([area], crs=4326).boundary.explore(m=m)
Out[11]:
Make this Notebook Trusted to load map: File -> Trust Notebook
Save the result GeoParquet with WKT geometry¶
In [12]:
Copied!
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", save_as_wkt=True
)
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", save_as_wkt=True
)
Finished operation in 0:00:04
Out[12]:
PosixPath('files/monaco-latest_nofilter_noclip_compact_wkt.parquet')
Specify result file path¶
In [13]:
Copied!
qosm.convert_geometry_to_parquet(
area, result_file_path='barcelona_osm_output.parquet'
)
qosm.convert_geometry_to_parquet(
area, result_file_path='barcelona_osm_output.parquet'
)
Finished operation in 0:00:19
Out[13]:
PosixPath('barcelona_osm_output.parquet')
Force recalculation of the result¶
By default, running the same command twice will result in reusing the saved GeoParquet file. You can force QuackOSM to recalculate the data.
In [14]:
Copied!
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", ignore_cache=True
)
qosm.convert_pbf_to_parquet(
"https://download.geofabrik.de/europe/monaco-latest.osm.pbf", ignore_cache=True
)
Finished operation in 0:00:04
Out[14]:
PosixPath('files/monaco-latest_nofilter_noclip_compact.parquet')