8000 High memory consumption of `modcall` · Issue #94 · twolinin/longphase · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

High memory consumption of modcall #94

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occas 8000 ionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
leonschuetz opened this issue Dec 12, 2024 · 2 comments
Open

High memory consumption of modcall #94

leonschuetz opened this issue Dec 12, 2024 · 2 comments

Comments

@leonschuetz
Copy link

Hello,

we noticed that modcall requires a huge amount of memory.
For a 70x longread genome (220GB) it uses between 34GB (1 thread) to 273GB (20 threads). For me it looks like a bug since I do not see a reason for the tool to keep the whole BAM file/chr in memory.

Do you have an explanation for that?

Best,
Leon
@ythuang0522
Copy link
Collaborator

Hi Leon, it's due to the brute-force parallelization during methylation-calling graph construction. The memory was released after the graph stage, But because the memory is too large, it's too slow to release and you will feel it's kept through the entire process. We are checking the code again to see if there is any better parallelization implementation.

@BeneKenobi
Copy link

Actually it leads to a OOM kill on our system (10 Threads, 64 GB RAM).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@BeneKenobi @ythuang0522 @leonschuetz
0