8000 Import multiple DLC datasets from a folder · Pull Request #437 · talmolab/sleap · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Import multiple DLC datasets from a folder #437

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Dec 24, 2020
Merged

Conversation

ghost
Copy link
@ghost ghost commented Dec 16, 2020

Import multiple DLC datasets in a folder.

Assuming a folder structure like this:

labeled-data/
    video1/
        my_labels.csv
        img00000.png
        img00001.png
        ...
    video2/
    ...

user can select the 'labeled-data' folder in the dialog, and then all '*.csv' files 2 levels below will be imported and merged into a single Sleap dataset.

Fixes #412

@codecov-io
Copy link
codecov-io commented Dec 16, 2020

Codecov Report

Merging #437 (94f321e) into develop (f4f0070) will increase coverage by 0.00%.
The diff coverage is 58.82%.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #437   +/-   ##
========================================
  Coverage    60.40%   60.41%           
========================================
  Files          119      119           
  Lines        17945    17979   +34     
========================================
+ Hits         10840    10862   +22     
- Misses        7105     7117   +12     
Impacted Files Coverage Δ
sleap/gui/commands.py 60.14% <57.57%> (-0.09%) ⬇️
sleap/gui/app.py 77.84% <100.00%> (+0.03%) ⬆️
sleap/io/dataset.py 82.84% <0.00%> (+0.13%) ⬆️
sleap/io/video.py 90.11% <0.00%> (+0.17%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef3189f...94f321e. Read the comment docs.

Comment on lines 783 to 788
print("Merged labels:")
merged_labels.describe()

merged_labels.save("{}.slp".format(params["filename"]))
merged_labels.save("{}.pkg.slp".format(params["filename"]),
with_images=True) # comment this out to skip saving with embedded images
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this -- the labels will be loaded from memory

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@staticmethod
def do_action(context: "CommandContext", params: dict):

csv_files = glob("{}/*/*.csv".format(params["dirname"]))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Prefer f-strings to .format(), i.e., glob(f"{params['dirname']}/*/*.csv")

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if len(dirname) == 0:
return False
params["dirname"] = dirname
params["filename"] = "converted_dlc_dataset"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above -- filename not needed. This op should not do any I/O. User can save the labels from File -> Save after importing

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@ghost ghost changed the title Adds utility to convert multiple DLC files into SLP file Import multiple DLC datasets from a folder Dec 17, 2020
@ghost ghost marked this pull request as ready for review December 17, 2020 17:10
Copy link
Collaborator
@talmo talmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just missing the tests and we're good to go :)

coords,x,y,x,y,x,y,x,y,x,y,x,y
labeled-data/video/img000.png,0,1,2,3,4,5,6,7,8,9,10,11
labeled-data/video/img001.png,12,13,,,15,16,17,18,,,20,21
labeled-data/video/img002.png,22,23,24,25,26,27,,,,,,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe change a couple of these numbers so we can test that we're not just importing the same thing twice?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split the 2 files to have a bit more realistic test. One has labels from the first 2 frames, the other one from the last frame. Also added the images for manual verification via UI

@ghost
Copy link
Author
B5C8 ghost commented Dec 23, 2020

PTAL

@talmo talmo merged commit 0a00384 into develop Dec 24, 2020
@talmo talmo deleted the arie/batch_dlc_import branch February 4, 2021 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0