8000 Problems with Visualizing Model Outputs · Issue #908 · talmolab/sleap · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Problems with Visualizing Model Outputs #908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 of 4 tasks
roomrys opened this issue Aug 11, 2022 Discussed in #907 · 2 comments
Closed
1 of 4 tasks

Problems with Visualizing Model Outputs #908

roomrys opened this issue Aug 11, 2022 Discussed in #907 · 2 comments
Labels
bug Something isn't working open pr A fix has been written, but is still being reviewed. stale but not fixed Issues that have been backlogged for a long time, but may be addressed in the future

Comments

@roomrys
Copy link
Collaborator
roomrys commented Aug 11, 2022

Bug description

When attempting to Predict > Visualize Model Outputs... an error occurs.

Expected behaviour

The confidence maps used in training will be plotted/visualized.

Actual behaviour

An error occurs.

Your personal set up

  • OS:

Windows 10

  • Version(s):
Environment packages 
(sleap-retracking) λ conda list
# packages in environment at C:\Users\TalmoLab\miniconda3\envs\sleap-retracking:
#
# Name                    Version                   Build  Channel
absl-py                   1.2.0                    pypi_0    pypi
alabaster                 0.7.12                   pypi_0    pypi
anyio                     3.6.1                    pypi_0    pypi
appdirs                   1.4.4                    pypi_0    pypi
argon2-cffi               21.3.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
astunparse                1.6.3                    pypi_0    pypi
atomicwrites              1.4.1                    pypi_0    pypi
attrs                     21.2.0                   pypi_0    pypi
babel                     2.10.3                   pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
backports-zoneinfo        0.2.1                    pypi_0    pypi
beautifulsoup4            4.11.1                   pypi_0    pypi
black                     21.6b0                   pypi_0    pypi
bleach                    5.0.1                    pypi_0    pypi
ca-certificates           2022.07.19           haa95532_0
cached-property           1.5.2                      py_0
cachetools                5.2.0                    pypi_0    pypi
cattrs                    1.1.1                    pypi_0    pypi
certifi                   2021.10.8                pypi_0    pypi
cffi                      1.15.1                   pypi_0    pypi
cfgv                      3.3.1                    pypi_0    pypi
charset-normalizer        2.1.0                    pypi_0    pypi
click                     8.0.4                    pypi_0    pypi
colorama                  0.4.5                    pypi_0    pypi
commonmark                0.9.1                    pypi_0    pypi
coverage                  6.4.2                    pypi_0    pypi
cuda-nvcc                 11.3.58              hb8d16a4_0    nvidia
cudatoolkit               11.3.1               h59b6b97_2
cudnn                     8.2.1                cuda11.3_0
cycler                    0.11.0                   pypi_0    pypi
debugpy                   1.6.2                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
deprecated                1.2.13                   pypi_0    pypi
distlib                   0.3.5                    pypi_0    pypi
docutils                  0.17.1                   pypi_0    pypi
efficientnet              1.0.0                    pypi_0    pypi
entrypoints               0.4                      pypi_0    pypi
ffmpeg                    4.2.2                he774522_0
filelock                  3.7.1                    pypi_0    pypi
flatbuffers               2.0                      pypi_0    pypi
fonttools                 4.34.4                   pypi_0    pypi
freetype                  2.10.4               hd328e21_0
gast                      0.5.3                    pypi_0    pypi
geos                      3.8.0                h33f27b4_0
gitdb                     4.0.9                    pypi_0    pypi
gitpython                 3.1.27                   pypi_0    pypi
google-auth               2.9.1                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
google-pasta              0.2.0                    pypi_0    pypi
greenlet                  1.1.2                    pypi_0    pypi
grpcio                    1.47.0                   pypi_0    pypi
h5py                      3.1.0           nompi_py37h19fda09_100    conda-forge
hdf5                      1.10.6               h1756f20_1
hdmf                      3.3.2                    pypi_0    pypi
icc_rt                    2019.0.0             h0cc432a_1
icu                       69.1                 h0e60522_0    conda-forge
identify                  2.5.3                    pypi_0    pypi
idna                      3.3                      pypi_0    pypi
image-classifiers         1.0.0                    pypi_0    pypi
imageio                   2.15.0                   pypi_0    pypi
imagesize                 1.4.1                    pypi_0    pypi
imgaug                    0.4.0                    pypi_0    pypi
imgstore                  0.2.9                    pypi_0    pypi
importlib-metadata        4.12.0                   pypi_0    pypi
importlib-resources       5.9.0                    pypi_0    pypi
iniconfig                 1.1.1                    pypi_0    pypi
intel-openmp              2022.0.0          haa95532_3663
ipykernel                 6.15.1                   pypi_0    pypi
ipython                   7.34.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.7.1                    pypi_0    pypi
jedi                      0.17.2                   pypi_0    pypi
jinja2                    3.1.2                    pypi_0    pypi
joblib                    1.1.0                    pypi_0    pypi
jpeg                      9e                   h2bbff1b_0
jsmin                     3.0.1                    pypi_0    pypi
json5                     0.9.9                    pypi_0    pypi
jsonpickle                1.2                      pypi_0    pypi
jsonschema                4.9.1                    pypi_0    pypi
jupyter-cache             0.4.3                    pypi_0    pypi
jupyter-client            7.3.4                    pypi_0    pypi
jupyter-core              4.11.1                   pypi_0    pypi
jupyter-server            1.13.5                   pypi_0    pypi
jupyter-server-mathjax    0.2.6                    pypi_0    pypi
jupyter-sphinx            0.3.2                    pypi_0    pypi
jupyterlab                3.3.4                    pypi_0    pypi
jupyterlab-pygments       0.2.2                    pypi_0    pypi
jupyterlab-server         2.15.0                   pypi_0    pypi
jupyterlab-widgets        1.1.1                    pypi_0    pypi
keras                     2.8.0                    pypi_0    pypi
keras-applications        1.0.8                    pypi_0    pypi
keras-preprocessing       1.1.2                    pypi_0    pypi
keyring                   23.7.0                   pypi_0    pypi
kiwisolver                1.4.4                    pypi_0    pypi
libblas                   3.8.0                    20_mkl    conda-forge
libcblas                  3.8.0                    20_mkl    conda-forge
libclang                  14.0.6                   pypi_0    pypi
libiconv                  1.16                 h2bbff1b_2
liblapack                 3.8.0                    20_mkl    conda-forge
libpng                    1.6.37               h2a8f88b_0
libtiff                   4.2.0                he0120a3_1
libwebp                   1.2.2                h2bbff1b_0
libxml2                   2.9.14               h0ad7f3c_0
libxslt                   1.1.35               h2bbff1b_0
libzlib                   1.2.12               h8ffe710_2    conda-forge
linkify-it-py             2.0.0                    pypi_0    pypi
livereload                2.6.3                    pypi_0    pypi
lz4-c                     1.9.3                h2bbff1b_1
m2w64-gcc-libgfortran     5.3.0                         6
m2w64-gcc-libs            5.3.0                         7
m2w64-gcc-libs-core       5.3.0                         7
m2w64-gmp                 6.1.0                         2
m2w64-libwinpthread-git   5.0.0.4634.697f757               2
markdown                  3.4.1                    pypi_0    pypi
markdown-it-py            1.1.0                    pypi_0    pypi
markupsafe                2.1.1                    pypi_0    pypi
matplotlib                3.5.2                    pypi_0    pypi
matplotlib-inline         0.1.3                    pypi_0    pypi
mdit-py-plugins           0.2.8                    pypi_0    pypi
mistune                   0.8.4                    pypi_0    pypi
mkl                       2020.2                      256
msys2-conda-epoch         20160418                      1
mypy-extensions           0.4.3                    pypi_0    pypi
myst-nb                   0.13.2                   pypi_0    pypi
myst-parser               0.15.2                   pypi_0    pypi
nbclassic                 0.4.3                    pypi_0    pypi
nbclient                  0.5.13                   pypi_0    pypi
nbconvert                 6.5.0                    pypi_0    pypi
nbdime                    3.1.1                    pypi_0    pypi
nbformat                  5.1.3                    pypi_0    pypi
ndx-pose                  0.1.1                    pypi_0    pypi
nest-asyncio              1.5.5                    pypi_0    pypi
networkx                  2.6.3                    pypi_0    pypi
nodeenv                   1.7.0                    pypi_0    pypi
notebook                  6.4.12                   pypi_0    pypi
notebook-shim             0.1.0                    pypi_0    pypi
numpy                     1.21.5           py37h2830a78_1    conda-forge
oauthlib                  3.2.0                    pypi_0    pypi
olefile                   0.46                     py37_0
opencv-python             4.6.0.66                 pypi_0    pypi
opencv-python-headless    4.5.5.62                 pypi_0    pypi
openssl                   1.1.1q               h2bbff1b_0
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 21.3                     pypi_0    pypi
pandas                    1.3.5            py37h9386db6_0    conda-forge
pandocfilters             1.5.0                    pypi_0    pypi
parso                     0.7.1                    pypi_0    pypi
pathspec                  0.9.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    8.4.0            py37hd45dc43_0
pip                       22.0.3             pyhd8ed1ab_0    conda-forge
pkginfo                   1.8.3                    pypi_0    pypi
pkgutil-resolve-name      1.3.10                   pypi_0    pypi
platformdirs              2.5.2                    pypi_0    pypi
pluggy                    1.0.0                    pypi_0    pypi
pre-commit                2.20.0                   pypi_0    pypi
prometheus-client         0.14.1                   pypi_0    pypi
prompt-toolkit            3.0.30                   pypi_0    pypi
protobuf                  3.19.4                   pypi_0    pypi
psutil                    5.9.1                    pypi_0    pypi
py                        1.11.0                   pypi_0    pypi
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycparser                 2.21                     pypi_0    pypi
pydata-sphinx-theme       0.8.1                    pypi_0    pypi
pygithub                  1.55                     pypi_0    pypi
pygments                  2.12.0                   pypi_0    pypi
pyjwt                     2.4.0                    pypi_0    pypi
pykalman                  0.9.5                    pypi_0    pypi
pynacl                    1.5.0                    pypi_0    pypi
pynwb                     2.1.0                    pypi_0    pypi
pyparsing                 3.0.9                    pypi_0    pypi
pyreadline                2.1                      py37_1
pyrsistent                0.18.1                   pypi_0    pypi
pyside2                   5.14.1                   pypi_0    pypi
pytest                    7.1.2                    pypi_0    pypi
pytest-cov                3.0.0                    pypi_0    pypi
pytest-qt                 4.1.0                    pypi_0    pypi
pytest-xvfb               2.0.0                    pypi_0    pypi
python                    3.7.13               h6244533_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-rapidjson          1.8                      pypi_0    pypi
python_abi                3.7                     2_cp37m    conda-forge
pytz                      2022.1           py37haa95532_0
pytz-deprecation-shim     0.1.0.post0              pypi_0    pypi
pyvirtualdisplay          3.0                      pypi_0    pypi
pywavelets                1.3.0                    pypi_0    pypi
pywin32                   304                      pypi_0    pypi
pywin32-ctypes            0.2.0                    pypi_0    pypi
pywinpty                  1.1.6                    pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
pyzmq                     23.2.0                   pypi_0    pypi
qimage2ndarray            1.8.3                    pypi_0    pypi
qt                        5.12.9               h556501e_6    conda-forge
readme-renderer           35.0                     pypi_0    pypi
regex                     2022.7.25                pypi_0    pypi
requests                  2.28.1                   pypi_0    pypi
requests-oauthlib         1.3.1                    pypi_0    pypi
requests-toolbelt         0.9.1                    pypi_0    pypi
rfc3986                   2.0.0                    pypi_0    pypi
rich                      10.16.1                  pypi_0    pypi
rsa                       4.9                      pypi_0    pypi
ruamel-yaml               0.17.21                  pypi_0    pypi
ruamel-yaml-clib          0.2.6                    pypi_0    pypi
scikit-image              0.19.3                   pypi_0    pypi
scikit-learn              1.0.2                    pypi_0    pypi
scikit-video              1.1.11                   pypi_0    pypi
scipy                     1.7.3            py37hb6553fb_0    conda-forge
seaborn                   0.11.2                   pypi_0    pypi
segmentation-models       1.0.1                    pypi_0    pypi
send2trash                1.8.0                    pypi_0    pypi
setuptools                58.0.4           py37haa95532_0
shapely                   1.7.1            py37h06580b3_0
shiboken2                 5.14.1                   pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1
sleap                     1.2.6                     dev_0    <develop>
smmap                     5.0.0                    pypi_0    pypi
sniffio                   1.2.0                    pypi_0    pypi
snowballstemmer           2.2.0                    pypi_0    pypi
soupsieve                 2.3.2.post1              pypi_0    pypi
sphinx                    4.5.0                    pypi_0    pypi
sphinx-autobuild          2021.3.14                pypi_0    pypi
sphinx-book-theme         0.3.3                    pypi_0    pypi
sphinx-copybutton         0.5.0                    pypi_0    pypi
sphinx-togglebutton       0.3.2                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    2.0.0                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.5                    pypi_0    pypi
sqlalchemy                1.4.39                   pypi_0    pypi
sqlite                    3.39.0               h2bbff1b_0
tensorboard               2.8.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tensorflow                2.8.2                    pypi_0    pypi
tensorflow-estimator      2.8.0                    pypi_0    pypi
tensorflow-io-gcs-filesystem 0.26.0                   pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.15.0                   pypi_0    pypi
threadpoolctl             3.1.0                    pypi_0    pypi
tifffile                  2021.11.2                pypi_0    pypi
tinycss2                  1.1.1                    pypi_0    pypi
tk                        8.6.12               h2bbff1b_0
toml                      0.10.2                   pypi_0    pypi
tomli                     2.0.1                    pypi_0    pypi
tornado                   6.2                      pypi_0    pypi
tqdm                      4.64.0                   pypi_0    pypi
traitlets                 5.3.0                    pypi_0    pypi
twine                     3.3.0                    pypi_0    pypi
typed-ast                 1.5.4                    pypi_0    pypi
typing-extensions         4.3.0                    pypi_0    pypi
tzdata                    2022.1                   pypi_0    pypi
tzlocal                   4.2                      pypi_0    pypi
uc-micro-py               1.0.1                    pypi_0    pypi
urllib3                   1.26.11                  pypi_0    pypi
vc                        14.2                 h21ff451_1
virtualenv                20.16.3                  pypi_0    pypi
vs2015_runtime            14.27.29016          h5e58377_2
wcwidth                   0.2.5                    pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.3.3                    pypi_0    pypi
werkzeug                  2.2.1                    pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0
widgetsnbextension        3.6.1                    pypi_0    pypi
wincertstore              0.2              py37haa95532_2
wrapt                     1.14.1                   pypi_0    pypi
xz                        5.2.5                h8cc25b3_1
zipp                      3.8.1                    pypi_0    pypi
zlib                      1.2.12               h8cc25b3_2
zstd                      1.5.2                h19a0ad4_0
Logs 
Traceback (most recent call last):
  File "d:\social-leap-estimates-animal-poses\pull-requests\sleap_retracking\sleap\sleap\gui\app.py", line 1709, in _handle_model_overlay_command
    predictor = DataOverlay.make_predictor(filename)
  File "d:\social-leap-estimates-animal-poses\pull-requests\sleap_retracking\sleap\sleap\gui\overlays\base.py", line 118, in make_predictor
    return VisualPredictor.from_trained_models(filename)
  File "d:\social-leap-estimates-animal-poses\pull-requests\sleap_retracking\sleap\sleap\nn\inference.py", line 464, in from_trained_models
    return cls(config=cfg, model=model)
TypeError: Can't instantiate abstract class VisualPredictor with abstract methods _initialize_inference_model, is_grayscale

Screenshots

How to reproduce

  1. Open GUI
  2. Click on Predict > Visualize Model Outputs...
  3. Select model
  4. See error

Discussed in #907

Originally posted by DEQ21 August 11, 2022
I recently have started the process of labeling and using SLEAP to cover most of the labels in the video. However, I am trying to generate the confidence graphs and velocity graphs that appear in the validation step. When I press visualize model outputs I get this : "TypeError: Can't instantiate abstract class VisualPredictor with abstract methods _initialize_inference_model, is_grayscale"
or I get a heading error. Could someone let me know how I should proceed?
@roomrys roomrys added the bug Something isn't working label Aug 11, 2022
@roomrys
Copy link
Collaborator Author
roomrys commented Aug 11, 2022 •
edited
Loading

Problem Analysis

It seems we updated the Predictor base class with abstract methods while leaving VisualPredictor behind, namely is_grayscale and _inititalize_inference_model are the abstract methods that are not present in VisualPredictor (but should be). If we create dummy methods for VisualPredictor s.a.

def is_grayscale(self):
     pass


def _inititalize_inference_model(self):
     pass

then, VisualPredictor starts working again! It is very slow to switch between frames, and I don't see a way to exit the predictor..... In fact, there is a comment
# TODO: Rewrite this class.
which will likely be the focus of this PR.

Proposal

Rewrite the VisualPredictor class to include the neccessary abstract methods. Also add an exit key/method and speed-up functionality if possible (data cache so that we don't redo predictions).

Observed inference + postprocessing times

INFO:sleap.nn.inference:Finished 1 examples in 1.62 seconds (inference + 
INFO:sleap.nn.inference:examples/s = 0.6168956864408537
INFO:sleap.nn.inference:Finished 1 examples in 1.59 seconds (inference + 
INFO:sleap.nn.inference:examples/s = 0.6301173038943354
INFO:sleap.nn.inference:Finished 1 examples in 1.58 seconds (inference + 
INFO:sleap.nn.inference:examples/s = 0.6349023548871147
ostprocessing)
INFO:sleap.nn.inference:examples/s = 0.6398051374733892
INFO:sleap.nn.inference:Finished 1 examples in 1.57 seconds (inference + postprocessing)
INFO:sleap.nn.inference:examples/s = 0.6361333659920815

@roomrys roomrys added the good first issue This issue is relatively self-contained. label Nov 8, 2022
@roomrys roomrys mentioned this issue Jan 4, 2023
Closed
11 tasks
@roomrys roomrys added open pr A fix has been written, but is still being reviewed. and removed good first issue This issue is relatively self-contained. labels Jan 13, 2023
@roomrys roomrys added the stale but not fixed Issues that have been backlogged for a long time, but may be addressed in the future label Feb 24, 2023
@roomrys
Copy link
Collaborator Author
roomrys commented Apr 22, 2025

We are closing this issue as we have decided to remove this portion of code for the time being:

@roomrys roomrys closed this as completed Apr 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working open pr A fix has been written, but is still being reviewed. stale but not fixed Issues that have been backlogged for a long time, but may be addressed in the future
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@roomrys
0