OSM Online Loader¶
In [1]:
Copied!
from functional import seq
from srai.loaders.osm_loaders import OSMOnlineLoader
from srai.loaders.osm_loaders.filters import GEOFABRIK_LAYERS, HEX2VEC_FILTER
from srai.loaders.osm_loaders.filters.popular import get_popular_tags
from srai.plotting.folium_wrapper import plot_regions
from srai.regionalizers import geocode_to_region_gdf
from functional import seq
from srai.loaders.osm_loaders import OSMOnlineLoader
from srai.loaders.osm_loaders.filters import GEOFABRIK_LAYERS, HEX2VEC_FILTER
from srai.loaders.osm_loaders.filters.popular import get_popular_tags
from srai.plotting.folium_wrapper import plot_regions
from srai.regionalizers import geocode_to_region_gdf
Filters¶
Filters are dictionaries used for specifying what type of objects one would like to download from OpenStreetMap.
There is currently one predefined filter (from Hex2Vec paper) and one way to download a filter - using popular tags from taginfo API.
They can also be defined manually in code.
Additionally, few predefined grouped filters are available (eg. BASE_OSM_GROUPS_FILTER and GEOFABRIK_LAYERS).
Grouped filters categorize base filters into groups.
Get popular tags from taginfo API¶
In [2]:
Copied!
all_popular_tags = get_popular_tags()
num_keys = len(all_popular_tags)
num_values = seq(all_popular_tags.values()).map(len).sum()
f"Unique keys: {num_keys}. Key/value pairs: {num_values}"
all_popular_tags = get_popular_tags()
num_keys = len(all_popular_tags)
num_values = seq(all_popular_tags.values()).map(len).sum()
f"Unique keys: {num_keys}. Key/value pairs: {num_values}"
Out[2]:
'Unique keys: 1208. Key/value pairs: 12725'
In [3]:
Copied!
seq(all_popular_tags.items()).take(10).dict()
seq(all_popular_tags.items()).take(10).dict()
Out[3]:
{'4wd_only': ['recommended', 'yes'],
'CLC:code': ['112', '211', '221', '231', '242', '243', '311', '312', '313'],
'CLC:explanation': ['See http://wiki.openstreetmap.org/wiki/Romania_CLC_Import.'],
'CLC:year': ['2006'],
'FMMP_modified': ['no'],
'FMMP_reviewed': ['no'],
'GNS:dsg_code': ['PPL', 'WAD'],
'GNS:dsg_name': ['populated place', 'wadi'],
'GNS:dsg_string': ['populated place'],
'GeoBaseNHN:DatasetName': ['02LG000', '02LH000']}
In [4]:
Copied!
frequent_in_wiki_only_tags = get_popular_tags(in_wiki_only=True, min_fraction=0.001)
frequent_in_wiki_only_tags
frequent_in_wiki_only_tags = get_popular_tags(in_wiki_only=True, min_fraction=0.001)
frequent_in_wiki_only_tags
Out[4]:
{'access': ['private'],
'addr:country': ['DE'],
'building': ['house', 'residential', 'yes'],
'highway': ['crossing',
'footway',
'path',
'residential',
'service',
'track',
'unclassified'],
'landuse': ['farmland'],
'natural': ['tree', 'water', 'wood'],
'oneway': ['yes'],
'power': ['pole', 'tower'],
'service': ['driveway'],
'source': ['BAG', 'Bing', 'NRCan-CanVec-10.0'],
'surface': ['asphalt', 'unpaved'],
'wall': ['no'],
'waterway': ['stream']}
Import a predefined filter¶
In [5]:
Copied!
hex_2_vec_keys = len(HEX2VEC_FILTER)
hex_2_vec_key_values = seq(HEX2VEC_FILTER.values()).map(len).sum()
f"Unique keys: {hex_2_vec_keys}. Key/value pairs: {hex_2_vec_key_values}"
hex_2_vec_keys = len(HEX2VEC_FILTER)
hex_2_vec_key_values = seq(HEX2VEC_FILTER.values()).map(len).sum()
f"Unique keys: {hex_2_vec_keys}. Key/value pairs: {hex_2_vec_key_values}"
Out[5]:
'Unique keys: 15. Key/value pairs: 725'
In [6]:
Copied!
geofabrik_layers_keys = len(GEOFABRIK_LAYERS)
geofabrik_layers_key_values = (
seq(GEOFABRIK_LAYERS.values()).flat_map(lambda filter: filter.items()).map(len).sum()
)
f"Unique groups: {geofabrik_layers_keys}. Key/value pairs: {geofabrik_layers_key_values}"
geofabrik_layers_keys = len(GEOFABRIK_LAYERS)
geofabrik_layers_key_values = (
seq(GEOFABRIK_LAYERS.values()).flat_map(lambda filter: filter.items()).map(len).sum()
)
f"Unique groups: {geofabrik_layers_keys}. Key/value pairs: {geofabrik_layers_key_values}"
Out[6]:
'Unique groups: 28. Key/value pairs: 116'
Using OSMOnlineLoader to download data for a specific area¶
Download all parks in Wrocław, Poland¶
In [7]:
Copied!
loader = OSMOnlineLoader()
parks_filter = {"leisure": "park"}
wroclaw_gdf = geocode_to_region_gdf("Wrocław, Poland")
parks_gdf = loader.load(wroclaw_gdf, parks_filter)
parks_gdf
loader = OSMOnlineLoader()
parks_filter = {"leisure": "park"}
wroclaw_gdf = geocode_to_region_gdf("Wrocław, Poland")
parks_gdf = loader.load(wroclaw_gdf, parks_filter)
parks_gdf
0%| | 0/1 [00:00<?, ?it/s]
Downloading leisure: park: 0%| | 0/1 [00:00<?, ?it/s]
Downloading leisure: park: 100%|██████████| 1/1 [00:00<00:00, 4.49it/s]
Downloading leisure: park: 100%|██████████| 1/1 [00:00<00:00, 4.48it/s]
Out[7]:
| geometry | leisure | |
|---|---|---|
| feature_id | ||
| relation/1348101 | POLYGON ((17.07422 51.08476, 17.07431 51.08479... | park |
| relation/3654662 | MULTIPOLYGON (((17.05674 51.12007, 17.05682 51... | park |
| relation/4552866 | POLYGON ((16.87549 51.13633, 16.87538 51.1364,... | park |
| relation/6629819 | POLYGON ((16.97217 51.08334, 16.97213 51.08324... | park |
| relation/6727464 | POLYGON ((16.97677 51.0944, 16.97681 51.0944, ... | park |
| ... | ... | ... |
| way/1308687185 | POLYGON ((17.02632 51.11988, 17.02644 51.11991... | park |
| way/1308687186 | POLYGON ((17.02649 51.11924, 17.02656 51.1189,... | park |
| way/1316998957 | POLYGON ((17.03063 51.12886, 17.03065 51.12886... | park |
| way/1316998958 | POLYGON ((17.03157 51.1295, 17.03162 51.12933,... | park |
| way/1320420245 | POLYGON ((16.94538 51.10385, 16.94556 51.10381... | park |
303 rows × 2 columns
In [8]:
Copied!
folium_map = plot_regions(wroclaw_gdf, colormap=["lightgray"], tiles_style="CartoDB positron")
parks_gdf.explore(m=folium_map, color="forestgreen")
folium_map = plot_regions(wroclaw_gdf, colormap=["lightgray"], tiles_style="CartoDB positron")
parks_gdf.explore(m=folium_map, color="forestgreen")
Out[8]:
Make this Notebook Trusted to load map: File -> Trust Notebook
Download hotels, bars, cafes, pubs and sport related objects in Barcelona¶
Uses grouped filters as an example.
In [9]:
Copied!
barcelona_gdf = geocode_to_region_gdf("Barcelona")
barcelona_filter = {
"tourism": {"building": "hotel", "amenity": ["bar", "cafe", "pub"]},
"sport": {"sport": "soccer", "leisure": ["pitch", "sports_centre", "stadium"]},
}
barcelona_objects_gdf = loader.load(barcelona_gdf, barcelona_filter)
barcelona_objects_gdf
barcelona_gdf = geocode_to_region_gdf("Barcelona")
barcelona_filter = {
"tourism": {"building": "hotel", "amenity": ["bar", "cafe", "pub"]},
"sport": {"sport": "soccer", "leisure": ["pitch", "sports_centre", "stadium"]},
}
barcelona_objects_gdf = loader.load(barcelona_gdf, barcelona_filter)
barcelona_objects_gdf
0%| | 0/8 [00:00<?, ?it/s]
Downloading building: hotel : 0%| | 0/8 [00:00<?, ?it/s]
Downloading building: hotel : 12%|█▎ | 1/8 [00:00<00:01, 5.33it/s]
Downloading amenity: bar : 12%|█▎ | 1/8 [00:00<00:01, 5.33it/s]
Downloading amenity: bar : 25%|██▌ | 2/8 [00:00<00:01, 4.49it/s]
Downloading amenity: cafe : 25%|██▌ | 2/8 [00:00<00:01, 4.49it/s]
Downloading amenity: cafe : 38%|███▊ | 3/8 [00:00<00:01, 4.07it/s]
Downloading amenity: pub : 38%|███▊ | 3/8 [00:00<00:01, 4.07it/s]
Downloading amenity: pub : 50%|█████ | 4/8 [00:00<00:00, 4.71it/s]
Downloading sport: soccer : 50%|█████ | 4/8 [00:00<00:00, 4.71it/s]
Downloading sport: soccer : 62%|██████▎ | 5/8 [00:01<00:00, 5.20it/s]
Downloading leisure: pitch : 62%|██████▎ | 5/8 [00:01<00:00, 5.20it/s]
Downloading leisure: pitch : 75%|███████▌ | 6/8 [00:01<00:00, 4.47it/s]
Downloading leisure: sports_centre: 75%|███████▌ | 6/8 [00:01<00:00, 4.47it/s]
Downloading leisure: sports_centre: 88%|████████▊ | 7/8 [00:01<00:00, 4.85it/s]
Downloading leisure: stadium : 88%|████████▊ | 7/8 [00:01<00:00, 4.85it/s]
Downloading leisure: stadium : 100%|██████████| 8/8 [00:01<00:00, 5.41it/s]
Downloading leisure: stadium : 100%|██████████| 8/8 [00:01<00:00, 4.94it/s]
Grouping features: 0%| | 0/2 [00:00<?, ?it/s]
Grouping features: 100%|██████████| 2/2 [00:00<00:00, 31.15it/s]
Out[9]:
| geometry | tourism | sport | |
|---|---|---|---|
| feature_id | |||
| node/216330105 | POINT (2.17484 41.3858) | amenity=bar | NaN |
| node/432592965 | POINT (2.17077 41.38035) | amenity=cafe | NaN |
| node/499122827 | POINT (2.17432 41.43045) | amenity=bar | NaN |
| node/499122828 | POINT (2.17371 41.43198) | amenity=bar | NaN |
| node/499122829 | POINT (2.17252 41.43052) | amenity=cafe | NaN |
| ... | ... | ... | ... |
| way/1343146768 | POLYGON ((2.16453 41.42136, 2.16457 41.42139, ... | amenity=cafe | NaN |
| way/1344398510 | POLYGON ((2.18329 41.42699, 2.18339 41.42699, ... | NaN | leisure=pitch |
| way/1346389090 | POLYGON ((2.15443 41.43601, 2.15517 41.43559, ... | NaN | leisure=sports_centre |
| way/1346389589 | POLYGON ((2.1556 41.43705, 2.1561 41.43742, 2.... | NaN | leisure=pitch |
| way/1347797774 | POLYGON ((2.17744 41.45049, 2.17759 41.45049, ... | NaN | leisure=pitch |
4240 rows × 3 columns
Tourism group¶
In [10]:
Copied!
folium_map = plot_regions(barcelona_gdf, colormap=["lightgray"], tiles_style="CartoDB positron")
barcelona_objects_gdf.query("tourism.notna()").explore(
m=folium_map,
color="orangered",
marker_kwds=dict(radius=1),
)
folium_map = plot_regions(barcelona_gdf, colormap=["lightgray"], tiles_style="CartoDB positron")
barcelona_objects_gdf.query("tourism.notna()").explore(
m=folium_map,
color="orangered",
marker_kwds=dict(radius=1),
)
Out[10]:
Make this Notebook Trusted to load map: File -> Trust Notebook