Author: Diego Cusicanqui
Contributors: Jean Baptiste Barré
contact: diego.cusicanqui@univ-grenoble-alpes.fr
Python scripts to quickly download and resample LiDAR HD classified point clouds from IGN-France.
This Python script can be executed on the command-line interface (CLI) as well as within a Jupyter Notebook. Refer to the How to use section for more details.
Visit the IGN-France website for updates on data processing.
- Required packages
- Clone repository
- Installation
- Pdal_wrench installation (optional but very useful)
- How to use
- Contact and citation
- python (conda) environment
- python>=3.10
- pdal==2.5.6
- draco=1.5.6 (we should specify draco version because there is an issue on the latest conda python-pdal version)
- python-pdal
- gdal
- untwine
- geopandas
- pathlib2
- tqdm
- ipykernel
- pyogrio
- pdal_wrench (optional if point density map is desired)
In order to install properly, you have to clone lidar_ign_downloader repository.
git clone https://github.com/cusicand/lidarhd_ign_downloader.git
cd ./lidarhd_ign_downloaderMost of the libraries required for this script are standard and are often pre-installed in conda Python environments. Please follow the instructions below based on your requirements.
If you already have a conda Python environment pre-installed, please follow the instructions in the section Install on an existing conda Python environment. Otherwise, you will need to install a conda Python environment to use this script. Instructions are provided in the section Install packages on a new conda Python environment.
Run the next command lines in your command-line prompt:
conda activate <your-env-name>or
conda install -c conda-forge pdal==2.5.6 draco=1.5.6 python-pdal gdal untwine geopandas python-wget pathlib2 tqdm ipykernel pyogrioWe encourage the use of mamba since this library as is faster than conda.
If you want to use mamba, run the following lines:
conda activate <your-env-name>mamba install -c conda-forge pdal==2.5.6 draco=1.5.6 python-pdal gdal untwine geopandas python-wget pathlib2 tqdmIf you want to create a specific python environment, please follow the instructions below.
Go to the Miniconda website and download the lastest version of Miniconda. Detailed instructions on how to install conda python environments for your operating system are available on the Anaconda website.
Once conda installed, you can
First, install mamba
conda install -n base -c conda-forge mamba then, install all packages using the pdal_env.yml file provided.
mamba env create -f pdal_env.ymlTo active the environment, type conda activate pdal_env
conda env create -f pdal_env.ymlTo active the environment, type:
conda activate pdal_envIf you want to run the script anywhere in your computer from CLI, you need to add the following lines to your .bashrc file to have full access to scripts.
Open your .bashrc file using vi ~/.bashrc or nano ~/.bashrc and copy the following lines at the end.
export LIDAR_PATH=$HOME/lidarhd_ign_downloader
export PATH=$LIDAR_PATH:$PATH
export PYTHONPATH=$LIDAR_PATH:$PYTHONPATHNote
If your installation directory is different than $HOME, Replace $HOME by the full directory path.
Once changes saved, run chmod +x lidar_downloader.py into github repository to ensure the execution of the script.
Use source ~/.bashrc to reload changes.
After version 2.0 of lidar_downloader, we introduced the possibility to generate a point density map to quantitatively estimate the number of points at a given resolution. This task is based on pdal_wrench and requires a separate installation. A detailed description can be found on the wrench GitHub. Further investigations are ongoing to better integrate pdal_wrench within lidar_downloader.
Note
Pdal_wrench density tool has been tested in high mountain environments where vegetation is not an issue. Future test should be conducted in environments with denser vegetation.
Once pdal_wrench installed, it is better you can run it from everywhere in the computer. To do so, add the following lines to your .bashrc.
export PDWRENCH=$HOME/wrench/build/
export PATH=$PDWRENCH:$PATHNote
If your installation directory is different than $HOME, Replace $HOME by the full directory path.
Inside the command line prompt, type python lidar_downloader.py -h to access the help for the tool.
This tool requires an Area of Interest (AOI) in *.shp or *.gpkg format as a mandatory argument. After version 3.0, several AOIs can be specified. Extracted DEM s=will take the name of the AOI file by default. Additionally, several parameters can be specified, such as:
-out_dataor--out_data_path: Specifies the output data directory. If not provided, data will be stored in the base path oflidar_downloader.pyby default.-tror--dem_resolution: Sets the desired resolution of the output DEM.-compute_elevor--compute_elevation: Determines how elevation is computed. Options are{mean, min, max, median}. The default value ismean. This parameter is still experimental.-dtypeor--file_data_type: Switches betweengtifandvrtfile formats.-force_databaseor--force_redownload_database: After version 3.0, we manage to download database from WFS service. Use this parameter re-download LiDAR HD database. Previous file will be deleted. Default value:False.-rm_tilesor--remove_tiles: Removes individual downloaded tiles after processing. Default value:True.-pdensityor--point_density_map: Generates a point density map for the given resolution. Requirespdal_wrenchto be installed. Default value:True. Refer to the PDAL_WRENCH installation (optional) section for more details.-cpu_wor--cpu_workload: Specifies the CPU usage ratio for processing data using a multi-threaded strategy. This value represents the percentage of CPU resources allocated for processing. Warning: It is recommended to reserve some CPUs for the OS and other processes (at least 4 CPUs). Default value:0.6. Maximum value:0.8.
Below is an example using the supplied shapefile:
lidar_downloader.py /path/to/aoi_example.gpkg -out_data /home/user/some/path/directory/ -tr 1 -compute_elev mean -dtype gtif -rm_tiles -pdensity -cpu_w 0.6or with several AOI files:
lidar_downloader.py /path/to/aoi_example1.gpkg /path/to/aoi_example2.gpkg /path/to/aoi_example2.gpkg --out_data_path /home/user/some/path/directory/ --dem_resolution 1 --compute_elevation mean --file_data_type gtif --remove_tiles --point_density_map --cpu_workload 0.6Inside a Jupyter-Notebook, you can also run the lidar_downloader. However, the set up of parameters are is slightly different. You can see and use the example provided within the project.
import lidar_downloaderargs_list = ['/path/to/aoi/aoi_example.gpkg',
'--out_data_path', '/some/path/directory/lidar_ign_test/',
'--dem_resolution', '1.0',
'--compute_elevation', 'mean',
'--file_data_type', 'gtif',
'--remove_tiles',
'--point_density_map',
'--cpu_workload', '0.6']parser = lidar_downloader.getparser()
args = parser.parse_args(args_list)lidar_downloader.main(args)Note
After version 3.0, the script will iterate through all the *.gpkg files provided in the command line see recommended use). It will create a folder for each specific AOI based on their filename by default. AOI file can contain any projection (EPSG:4326).
For any question/bug/issue regarding this tool, please report it on issues section or contact diego.cusicanqui@univ-grenoble-alpes.fr.
Regarding the LiDAR HD data, please visit their webpage to know more about how to cite their data.
Important
If you use this tool, please cite using the following DOI. This will allow some recognition of the time invested and open access to this tool.