8000 espnet-spk data preparation part by Jungjee · Pull Request #5184 · espnet/espnet · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

espnet-spk data preparation part #5184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 26 commits into from
Jun 3, 2023
Merged

espnet-spk data preparation part #5184

merged 26 commits into from
Jun 3, 2023

Conversation

Jungjee
Copy link
Contributor
@Jungjee Jungjee commented May 25, 2023

This pr includes the data preparation part for ESPnet-spk project.
It cannot be used at current status. Following PR will be dealing with the training part.

  • Includes VoxCeleb download script (crawl from YouTube)

@mergify mergify bot added the ESPnet2 label May 25, 2023
Emrys365
Emrys365 reviewed May 25, 2023
View reviewed changes
Copy link
Collaborator
@Emrys365 Emrys365 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to include other files/directories in the TEMPLATE/sv1 folder, just like the other tasks. For example:

conf/
local/path.sh
README.md
db.sh -> ../../TEMPLATE/asr1/db.sh
sv.sh
pyscripts -> ../../TEMPLATE/asr1/pyscripts
setup.sh
steps -> ../../TEMPLATE/asr1/steps
utils -> ../../TEMPLATE/asr1/utils

egs2/voxceleb/sv1/sv.sh should be a symbolic link pointing to egs2/TEMPLATE/sv1/sv.sh.

Emrys365
Emrys365 reviewed May 25, 2023
View reviewed changes
@ftshijt ftshijt added SID Speaker identification/embedding New Features labels May 25, 2023
@Jungjee
Copy link
Contributor Author
Jungjee commented May 25, 2023

Thanks @Emrys365 for your review :)

ftshijt
ftshijt reviewed May 25, 2023
View reviewed changes
@sw005320
Copy link
Contributor

Can you fix the CI error?
https://github.com/espnet/espnet/actions/runs/5075591243/jobs/9128549995?pr=5184#step:7:281

@sw005320 sw005320 added this to the v.202307 milestone May 31, 2023
Jee-weon Jung and others added 2 commits May 31, 2023 09:54
@codecov
Copy link
codecov bot commented May 31, 2023 •
edited
Loading

Codecov Report

Merging #5184 (2d89067) into master (e6a628e) will increase coverage by 10.18%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #5184       +/-   ##
===========================================
+ Coverage   64.35%   74.54%   +10.18%     
===========================================
  Files         640      640               
  Lines       57057    57267      +210     
===========================================
+ Hits        36719    42688     +5969     
+ Misses      20338    14579     -5759
Flag Coverage Δ
test_integration_espnet1 66.28% <ø> (?)
test_integration_espnet2 47.58% <ø> (?)
test_python 65.28% <ø> (+0.92%) ⬆️
test_utils 23.28% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 199 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@sw005320 sw005320 added the auto-merge Enable auto-merge label May 31, 2023
@sw005320
Copy link
Contributor

This is good enough as a first commit.
I'll merge it after the CI is finished.

@sw005320
Copy link
Contributor
sw005320 commented Jun 1, 2023

Can you fix https://github.com/espnet/espnet/actions/runs/5136094955/jobs/9242613162?pr=5184#step:12:104181 by following the instruction?

@sw005320
Copy link
Contributor
sw005320 commented Jun 1, 2023

@Jungjee, sorry, I tried to fix the issues, but it turns out that I need to change egs2/accented_french_openslr57/asr1/cmd.sh, egs2/voxceleb/sv1/cmd.sh, (and maybe egs2/TEMPLATE/sv1/cmd.sh) from symlink to the normal file.
Can you do that for me?

Basically, cmd.sh and slurm.conf etc. could be used for the user by directly editing them.
So, it would be better to use the actual file instead of the symlink.

@Jungjee
Copy link
Contributor Author
Jungjee commented Jun 1, 2023

@sw005320 , sure, but the error you ask me to fix before is
Error: egs2/accented_french_openslr57/asr1/cmd.sh and egs2/TEMPLATE/sv1/cmd.sh differ: To solve: for f in egs2/*/*/cmd.sh; do cp egs2/TEMPLATE/asr1/cmd.sh ${f}; done

which means to changes local cmd.sh with the symlink of TEMPLATE one.

So my understanding is to undo this and make CI error come again.

Pls let me know if I'm thinking something wrong.

Meanwhile, I cannot pull from Jungjee/master somehow, trying to solve this issue.

@Jungjee Jungjee closed this Jun 1, 2023
@Jungjee
Merge pull request #7 from Jungjee/speaker
636755f 
make cmd.sh into real files
@Jungjee Jungjee reopened this Jun 1, 2023
@sw005320
Copy link
Contributor
sw005320 commented Jun 1, 2023

which means to changes local cmd.sh with the symlink of TEMPLATE one.

First, egs2/TEMPLATE/asr1/cmd.sh is an actual file.
Not a symlink.
Then, do cp egs2/TEMPLATE/asr1/cmd.sh ${f} means that it just copies an actual file.
So, cmd.sh under the recipe directory should be an actual file.

@Jungjee
Copy link
Contributor Author
Jungjee commented Jun 1, 2023

Sure, I think it's okay now? after commit 46c17d2

Uh oh!

There was an error while loading. Please reload this page.
@sw005320
Copy link
Contributor
sw005320 commented Jun 1, 2023

Thanks, it should be fine, and let's ask the CI for the rest.

@sw005320
Copy link
Contributor
sw005320 commented Jun 2, 2023

Another issue
https://github.com/espnet/espnet/actions/runs/5150271646/jobs/9274249165?pr=5184#step:12:103790
I recommend you run the integration test locally by yourself to fix this in advance.

Emrys365
Emrys365 reviewed Jun 2, 2023
View reviewed changes
@sw005320
Copy link
Contributor
sw005320 commented Jun 2, 2023

Can you remove some files under TEMPLATE/sv1/local?
We should not put recipe-related files under the template.
Please check https://github.com/espnet/espnet/tree/master/egs2/TEMPLATE/asr1/local

@sw005320
Copy link
Contributor
sw005320 commented Jun 2, 2023

I just did it by myself.
942E

@mergify mergify bot merged commit d9ae3f7 into espnet:master Jun 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Emrys365 Emrys365 Emrys365 left review comments

@ftshijt ftshijt ftshijt left review comments

Assignees
No one assigned
Labels
auto-merge Enable auto-merge ESPnet2 New Features SID Speaker identification/embedding
Projects
None yet
Milestone
v.202307
Development

Successfully merging this pull request may close these issues.
4 participants
@Jungjee @sw005320 @Emrys365 @ftshijt
0