An overview of a typical place recognition pipeline. At first, the input data is encoded into a query descriptor. Then, a K-nearest neighbors search is performed between the query and the database. Finally, the position of the closest database descriptor found is considered as the answer.
This library is suitable for:
- 🚗 Navigation of autonomous cars, robots, and drones using cameras and lidars, especially in areas with limited or unavailable GPS signals.
- 📦 Localization of delivery robots needing reliable positioning both indoors and outdoors.
- 🔬 Research and development of computer vision algorithms, related to multimodal place recognition and localization.
- 🎓 Educational purposes and research projects, involving robotics, autonomous systems, and computer vision.
Documentation: https://openplacerecognition.readthedocs.io
Our library comes packed with ready-to-use modules that you can use "as-is" or as inspiration for your custom creations. Dive into the details in our documentation.
-
x86_64:
- CPU: 6 or more physical cores
- RAM: at least 8 GB
- GPU: NVIDIA RTX 2060 or higher (to ensure adequate performance)
- Video memory: at least 4 GB
- Storage: SSD recommended for faster loading of data and models
-
NVIDIA Jetson:
- We recommend using NVIDIA Jetson Xavier AGX. The library should be compatible with all newer versions of devices, but we haven't tested them yet.
-
Operating System:
- x86_64: Any OS with support for Docker and CUDA >= 11.1. Ubuntu 20.04 or later is recommended.
- NVIDIA Jetson: Ubuntu 20.04 or later with Jetpack >= 5.0.
-
Dependencies (if not using Docker):
- Python >= 3.10
- CUDA Toolkit >= 11.1
- cuDNN >= 7.5
More details on the installation without Docker can be found in the Advanced Installation section of our documentation.
At first, ensure that you cloned the repository https://github.com/OPR-Project/OpenPlaceRecognition and changed the directory to the root of the repository:
git clone https://github.com/OPR-Project/OpenPlaceRecognition.git
cd OpenPlaceRecognition
# do not forget to load git submodules
git submodule update --initThe recommended and easiest way to use the library is through the provided Docker environment.
The Dockerfile.base contains all prerequisites needed to run the library,
including optional MinkowskiEngine and faiss libraries.
You can either pull this image from Docker Hub (docker pull alexmelekhin/open-place-recognition:base),
or build it manually (bash docker/build_base.sh).
The Dockerfile.devel installs additional requirements from
requirements.txt,
requirements-dev.txt,
and requirements-notebook.txt files,
and creates a non-root user inside the image to avoid permission issues when using the container with mounted volumes.
The devel version of the image should be built manually by the user:
bash docker/build_devel.shWhen starting the container, you must provide a data directory that will be mounted to ~/Datasets inside the container:
bash docker/start.sh [DATASETS_DIR]To enter the container's /bin/bash terminal, use the docker/into.sh script:
bash docker/into.shAfter you enter the container, install the library (we recommend installing in editable mode with the -e flag to be able to make modifications to the code):
pip install -e ~/OpenPlaceRecognitionFor more detailed instructions on installing third-party dependencies, configuring your environment manually, and additional setup options, please refer to the Installation section of our documentation.
You can load the weights from the Hugging Face organization OPR-Project. This is the recommended way to load the weights, as you can see the license and the README file for each model group. Note that models may have different licenses, depending on the dataset they were trained on.
Alternatively, you can download the weights from the public Google Drive folder. But please check the license for each model group on the OPR-Project page before using them.
Developers only
We use DVC to manage the weights storage. To download the weights, run the following command (assuming that dvc is already installed):
dvc pullYou will be be asked to authorize the Google Drive access. After that, the weights will be downloaded to the weights directory. For more details, see the DVC documentation.
Explore multimodal Place Recognition with ITLP Campus — a diverse dataset of indoor and outdoor university environments featuring synchronized RGB images, LiDAR point clouds, semantic masks, and rich scene descriptions. Built for real-world challenges, day or night, floor or field.
You can find more details in the OPR-Project/ITLP-Campus repository.
You can download the dataset:
- Kaggle:
- Hugging Face:
We provide several datasets that can be used with the library.
-
Oxford RobotCar is a comprehensive autonomous driving dataset featuring over 1,000 km of traffic data, nearly 20 million images, and sensor data collected in diverse weather conditions from an autonomous vehicle in Oxford between 2014 and 2015. We provide a specifically pre-processed subset designed for the place recognition task in the OpenPlaceRecognition library. This subset is primarily based on the PointNetVLAD paper.
-
NCLT is the University of Michigan North Campus Long-Term Vision and LIDAR Dataset. We provide a modified version of the NCLT dataset, which is primarily based on the AdaFusion paper.
You can learn more about the package structure in the Introduction - Package Structure section of the documentation.
You can find the models for place recognition in the Hugging Face model hub:
- https://huggingface.co/OPR-Project/PlaceRecognition-NCLT - NCLT-trained models (including models with ITLP Campus fine-tuning)
You can find the models for point cloud registration in the Hugging Face model hub:
- https://huggingface.co/OPR-Project/Registration-nuScenes - nuScenes-trained models
- https://huggingface.co/OPR-Project/Registration-KITTI - KITTI-trained models
| Model | Modality | Train Dataset | Config | Weights |
|---|---|---|---|---|
| GeoTransformer (paper) | LiDAR | KITTI | geotransformer_kitti.yaml | geotransformer_kitti.pth |
| HRegNet (paper) | LiDAR | KITTI | hregnet.yaml | hregnet_kitti.pth |
| HRegNet Coarse (1-step) (paper) | LiDAR | KITTI | hregnet_coarse.yaml | hregnet_kitti.pth |
| HRegNet (paper) | LiDAR | nuScenes | hregnet.yaml | hregnet_nuscenes.pth |
| HRegNet Coarse (1-step) (paper) | LiDAR | nuScenes | hregnet_coarse.yaml | hregnet_nuscenes.pth |
| HRegNet w/o Sim-feats (paper) | LiDAR | nuScenes | hregnet_nosim.yaml | hregnet_nosim_nuscenes.pth |
| HRegNet Light (custom modification) | LiDAR | nuScenes | hregnet_light_feats.yaml | hregnet_light_feats_nuscenes.pth |
You can find the weights for PaddleOCR in the Hugging Face model hub: https://huggingface.co/OPR-Project/PaddleOCR
Make sure huggingface_hub is installed: pip install huggingface_hub.
Load the weights using the following code:
from pathlib import Path
from huggingface_hub import snapshot_download
ocr_weights_path = Path("/home/docker_opr/OpenPlaceRecognition/weights/paddleocr") # change to your path
if not ocr_weights_path.exists():
ocr_weights_path.mkdir(parents=True)
snapshot_download(repo_id="OPR-Project/PaddleOCR", repo_type="model", local_dir=ocr_weights_path)- OPR-Project/OpenPlaceRecognition-ROS2 - ROS-2 implementation of OpenPlaceRecognition modules
- OPR-Project/ITLP-Campus - ITLP-Campus dataset tools.
- KirillMouraviev/simple_toposlam_model - An implementation of the Topological SLAM method that uses the OPR library.