Hex2vec embedder
In [1]:
Copied!
from pytorch_lightning import seed_everything
from srai.embedders import Hex2VecEmbedder
from srai.joiners import IntersectionJoiner
from srai.loaders import OSMOnlineLoader
from srai.neighbourhoods import H3Neighbourhood
from srai.plotting import plot_numeric_data, plot_regions
from srai.regionalizers import H3Regionalizer, geocode_to_region_gdf
from pytorch_lightning import seed_everything
from srai.embedders import Hex2VecEmbedder
from srai.joiners import IntersectionJoiner
from srai.loaders import OSMOnlineLoader
from srai.neighbourhoods import H3Neighbourhood
from srai.plotting import plot_numeric_data, plot_regions
from srai.regionalizers import H3Regionalizer, geocode_to_region_gdf
In [2]:
Copied!
SEED = 71
seed_everything(SEED)
SEED = 71
seed_everything(SEED)
Seed set to 71
Out[2]:
71
Load data from OSM¶
First use geocoding to get the area
In [3]:
Copied!
area_gdf = geocode_to_region_gdf("Wrocław, Poland")
plot_regions(area_gdf, tiles_style="CartoDB positron")
area_gdf = geocode_to_region_gdf("Wrocław, Poland")
plot_regions(area_gdf, tiles_style="CartoDB positron")
Out[3]:
Make this Notebook Trusted to load map: File -> Trust Notebook
Next, download the data for the selected region and the specified tags. We're using OSMOnlineLoader here, as it's faster for low numbers of tags. In a real life scenario with more tags, you would likely want to use the OSMPbfLoader.
In [4]:
Copied!
tags = {
"leisure": "park",
"landuse": "forest",
"amenity": ["bar", "restaurant", "cafe"],
"water": "river",
"sport": "soccer",
}
loader = OSMOnlineLoader()
features_gdf = loader.load(area_gdf, tags)
folium_map = plot_regions(area_gdf, colormap=["rgba(0,0,0,0)"], tiles_style="CartoDB positron")
features_gdf.explore(m=folium_map)
tags = {
"leisure": "park",
"landuse": "forest",
"amenity": ["bar", "restaurant", "cafe"],
"water": "river",
"sport": "soccer",
}
loader = OSMOnlineLoader()
features_gdf = loader.load(area_gdf, tags)
folium_map = plot_regions(area_gdf, colormap=["rgba(0,0,0,0)"], tiles_style="CartoDB positron")
features_gdf.explore(m=folium_map)
0%| | 0/7 [00:00<?, ?it/s]
Downloading leisure: park : 0%| | 0/7 [00:00<?, ?it/s]
Downloading leisure: park : 14%|█▍ | 1/7 [00:00<00:01, 4.41it/s]
Downloading landuse: forest : 14%|█▍ | 1/7 [00:00<00:01, 4.41it/s]
Downloading landuse: forest : 29%|██▊ | 2/7 [00:00<00:01, 3.88it/s]
Downloading amenity: bar : 29%|██▊ | 2/7 [00:00<00:01, 3.88it/s]
Downloading amenity: bar : 43%|████▎ | 3/7 [00:00<00:00, 4.78it/s]
Downloading amenity: restaurant: 43%|████▎ | 3/7 [00:00<00:00, 4.78it/s]
Downloading amenity: restaurant: 57%|█████▋ | 4/7 [00:00<00:00, 5.11it/s]
Downloading amenity: cafe : 57%|█████▋ | 4/7 [00:00<00:00, 5.11it/s]
Downloading amenity: cafe : 71%|███████▏ | 5/7 [00:00<00:00, 5.42it/s]
Downloading water: river : 71%|███████▏ | 5/7 [00:00<00:00, 5.42it/s]
Downloading water: river : 86%|████████▌ | 6/7 [00:01<00:00, 5.16it/s]
Downloading sport: soccer : 86%|████████▌ | 6/7 [00:01<00:00, 5.16it/s]
Downloading sport: soccer : 100%|██████████| 7/7 [00:01<00:00, 5.46it/s]
Downloading sport: soccer : 100%|██████████| 7/7 [00:01<00:00, 5.10it/s]
Out[4]:
Make this Notebook Trusted to load map: File -> Trust Notebook
Prepare the data for embedding¶
After downloading the data, we need to prepare it for embedding. Namely - we need to regionalize the selected area, and join the features with regions.
In [5]:
Copied!
regionalizer = H3Regionalizer(resolution=9)
regions_gdf = regionalizer.transform(area_gdf)
plot_regions(regions_gdf, tiles_style="CartoDB positron")
regionalizer = H3Regionalizer(resolution=9)
regions_gdf = regionalizer.transform(area_gdf)
plot_regions(regions_gdf, tiles_style="CartoDB positron")
Out[5]:
Make this Notebook Trusted to load map: File -> Trust Notebook
In [6]:
Copied!
joiner = IntersectionJoiner()
joint_gdf = joiner.transform(regions_gdf, features_gdf)
joint_gdf
joiner = IntersectionJoiner()
joint_gdf = joiner.transform(regions_gdf, features_gdf)
joint_gdf
Out[6]:
| region_id | feature_id |
|---|---|
| 891e20442c7ffff | way/381939983 |
| 891e2042dafffff | way/346862818 |
| 891e204624bffff | way/339335815 |
| 891e20473b7ffff | way/56444171 |
| 891e20420dbffff | way/131328271 |
| ... | ... |
| 891e2055063ffff | way/217674535 |
| way/340894667 | |
| way/334028946 | |
| way/217674534 | |
| 891e205539bffff | way/334028946 |
3978 rows × 0 columns
Embedding¶
After preparing the data we can proceed with generating embeddings for the regions.
In [7]:
Copied!
import warnings
neighbourhood = H3Neighbourhood(regions_gdf)
embedder = Hex2VecEmbedder([15, 10])
with warnings.catch_warnings():
warnings.simplefilter("ignore")
embeddings = embedder.fit_transform(
regions_gdf,
features_gdf,
joint_gdf,
neighbourhood,
trainer_kwargs={"max_epochs": 5, "accelerator": "cpu"},
batch_size=100,
)
embeddings
import warnings
neighbourhood = H3Neighbourhood(regions_gdf)
embedder = Hex2VecEmbedder([15, 10])
with warnings.catch_warnings():
warnings.simplefilter("ignore")
embeddings = embedder.fit_transform(
regions_gdf,
features_gdf,
joint_gdf,
neighbourhood,
trainer_kwargs={"max_epochs": 5, "accelerator": "cpu"},
batch_size=100,
)
embeddings
0%| | 0/3168 [00:00<?, ?it/s]
100%|██████████| 3168/3168 [00:00<00:00, 32363.68it/s]
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
Missing logger folder: /home/runner/work/srai/srai/examples/embedders/lightning_logs
| Name | Type | Params --------------------------------------- 0 | encoder | Sequential | 280 --------------------------------------- 280 Trainable params 0 Non-trainable params 280 Total params 0.001 Total estimated model params size (MB)
`Trainer.fit` stopped: `max_epochs=5` reached.
Out[7]:
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | |
|---|---|---|---|---|---|---|---|---|---|---|
| region_id | ||||||||||
| 891e20471bbffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
| 891e2042247ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
| 891e20442c7ffff | -0.311252 | 0.004352 | -0.316782 | -0.464754 | 0.035981 | 0.239774 | -0.364358 | 0.348595 | -0.082190 | -0.002130 |
| 891e20402a3ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
| 891e2042dafffff | 0.063493 | -0.191839 | 0.108596 | -0.438075 | -0.153718 | 0.050943 | 0.183005 | 0.137182 | -0.036608 | -0.567610 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 891e2055247ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
| 891e2042c7bffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
| 891e205539bffff | 0.063493 | -0.191839 | 0.108596 | -0.438075 | -0.153718 | 0.050943 | 0.183005 | 0.137182 | -0.036608 | -0.567610 |
| 891e204290bffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
| 891e2047427ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 |
3168 rows × 10 columns
Visualizing the embeddings' similarity¶
In [8]:
Copied!
from sklearn.cluster import KMeans
clusterizer = KMeans(n_clusters=5, random_state=SEED)
clusterizer.fit(embeddings)
embeddings["cluster"] = clusterizer.labels_
embeddings
from sklearn.cluster import KMeans
clusterizer = KMeans(n_clusters=5, random_state=SEED)
clusterizer.fit(embeddings)
embeddings["cluster"] = clusterizer.labels_
embeddings
Out[8]:
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | cluster | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| region_id | |||||||||||
| 891e20471bbffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
| 891e2042247ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
| 891e20442c7ffff | -0.311252 | 0.004352 | -0.316782 | -0.464754 | 0.035981 | 0.239774 | -0.364358 | 0.348595 | -0.082190 | -0.002130 | 1 |
| 891e20402a3ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
| 891e2042dafffff | 0.063493 | -0.191839 | 0.108596 | -0.438075 | -0.153718 | 0.050943 | 0.183005 | 0.137182 | -0.036608 | -0.567610 | 4 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 891e2055247ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
| 891e2042c7bffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
| 891e205539bffff | 0.063493 | -0.191839 | 0.108596 | -0.438075 | -0.153718 | 0.050943 | 0.183005 | 0.137182 | -0.036608 | -0.567610 | 4 |
| 891e204290bffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
| 891e2047427ffff | 0.352490 | -0.265234 | -0.000894 | 0.272734 | 0.215892 | -0.133884 | -0.236737 | -0.106074 | -0.217266 | 0.325965 | 0 |
3168 rows × 11 columns
In [9]:
Copied!
plot_numeric_data(regions_gdf, "cluster", embeddings)
plot_numeric_data(regions_gdf, "cluster", embeddings)
Out[9]:
Make this Notebook Trusted to load map: File -> Trust Notebook