Description
Hi,
I'm currently trying to update Siconos, still for PlatRock usage. I know @fperignon that you worked hard on Siconos install / python package paths a few months ago, thanks for that. The documentation was improved as well, that's nice !
I'm using Poetry as python package manager but I get a python path-related error on Siconos import, that's why I have a suggestion. Note that the whole idea here is to improve the virtual envs managers compatibility, I'm aware that I could have used SICONOS_CUSTOM_INSTALL and ISOLATED_INSTALL cmake options, but that's not the point.
First, I compiled/installed Siconos successfully, with cmake telling me the two main paths:
- The binaries, libraries and headers will be installed in
/home/francois/PlatRock/.venv- Siconos python packages will be installed in
/home/francois/PlatRock
The first (.venv) dir is the poetry virtual env I'm using with PlatRock / Siconos. Below that, I have classic bin,lib,include,... tree.
After that, I can import siconos within the virtual environment:
In [1]: import siconos
In [2]: siconos.__file__
Out[2]: '/home/francois/PlatRock/.venv/lib/python3.12/site-packages/siconos/__init__.py'But the following fails, probably because of the way install paths are handled:
In [3]: import siconos.kernel as sk
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Cell In[3], line 1
----> 1 import siconos.kernel as sk
File ~/PlatRock/.venv/lib/python3.12/site-packages/siconos/kernel.py:20
18 # Import the low-level C/C++ module
19 if __package__ or "." in __name__:
---> 20 from . import _sicpykernel
21 else:
22 import _sicpykernel
ImportError: cannot import name '_sicpykernel' from 'siconos' (/home/francois/PlatRock/.venv/lib/python3.12/site-packages/siconos/__init__.py)It turns out that _sicpykernel is installed in /home/francois/PlatRock/siconos, just like cmake tells. But as a consequence, siconos install is split into two folders, resulting in internal import issue.
The same procedure using venv instead of poetry works well, so I dug a little. With venv, the Siconos python package dir is different (note the full site-package path):
- The binaries, libraries and headers will be installed in /home/francois/siconos/siconosenv
- Siconos python packages will be installed in /home/francois/siconos/siconosenv/lib/python3.12/site-packages
Looking further in Siconos cmake files, I found here that PY_INSTALL_DIR relies on sys.path, more precisely on its last entry. With venv, the last entry is the site-package path, whereas in poetry the last entry is the virtual env root path, and this make _sicpykernel (and others) to reside in a place it is not found when locally importing them.
My suggestion is to use standard sysconfig.get_paths() instead of sys.path, which gives a dict with nice paths that seems consistent across different virtual env managers.
- venv:
{'data': '/home/francois/siconos/siconosenv',
'include': '/usr/include/python3.12',
'platinclude': '/usr/include/python3.12',
'platlib': '/home/francois/siconos/siconosenv/lib/python3.12/site-packages',
'platstdlib': '/home/francois/siconos/siconosenv/lib/python3.12',
'purelib': '/home/francois/siconos/siconosenv/lib/python3.12/site-packages',
'scripts': '/home/francois/siconos/siconosenv/bin',
'stdlib': '/usr/lib/python3.12'}
- poetry:
{'data': '/home/francois/PlatRock/.venv',
'include': '/usr/include/python3.12',
'platinclude': '/usr/include/python3.12',
'platlib': '/home/francois/PlatRock/.venv/lib/python3.12/site-packages',
'platstdlib': '/home/francois/PlatRock/.venv/lib/python3.12',
'purelib': '/home/francois/PlatRock/.venv/lib/python3.12/site-packages',
'scripts': '/home/francois/PlatRock/.venv/bin',
'stdlib': '/usr/lib/python3.12'}
purelib key inside that looks like a good choice, but I didn't checked conda compatiblity.
As a conclusion, my suggestion in SiconosInstallSetup.cmake would be to replace L101:
execute_process(COMMAND ${Python_EXECUTABLE} -c "import sys;print(sys.path[-1])" OUTPUT_VARIABLE PY_INSTALL_DIR)
with:
execute_process(COMMAND ${Python_EXECUTABLE} -c "import sysconfig;print(sysconfig.get_paths()['purelib'])" OUTPUT_VARIABLE PY_INSTALL_DIR)
I'm not sure, but it may also be valid for conda, which would make IN_CONDA detection no longer necessary.