8000 chunkserver to handle out-of-inodes condition gracefully · Issue #358 · moosefs/moosefs · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
8000

chunkserver to handle out-of-inodes condition gracefully #358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
onlyjob opened this issue Apr 18, 2020 · 2 comments
Open

chunkserver to handle out-of-inodes condition gracefully #358

onlyjob opened this issue Apr 18, 2020 · 2 comments
Labels
confirmed bug Confirmed bug fix committed / please test

Comments

@onlyjob
Copy link
Contributor
onlyjob commented Apr 18, 2020

When HDD runs out of inodes, chunkserver is getting stuck with <500> load.
Perhaps such situation could be handled gracefully, with reported 100% HDD usage.

Reproducing should be easy on f2fs file system that have rather low inodes limit.

$ df -hi /mnt/NVMe
Filesystem         Inodes IUsed IFree IUse% Mounted on
/dev/nvme0n1         7.2M  7.2M     4  100% /mnt/NVMe
@acid-maker
Copy link
Member

We do not observe this high load problem when there are no inodes left, but still this is something that needs to be addressed.

@acid-maker acid-maker added the confirmed bug Confirmed bug label Apr 20, 2020
@onlyjob
Copy link
Contributor Author
onlyjob commented Apr 20, 2020

Maybe high load develops over time? Basically chunkserver became unresponsive causing logs flood on other chunkservers:

replicator,get # of blocks: got status: Operation not completed from (192.168.0.250:72EF)

Could it be a contributing factor to the high load?

Clients may have been affected also, without falling back to other chunkservers...

acid-maker added a commit that referenced this issue Apr 21, 2020
@acid-maker
(cs) added handling inode limits in local filesystems on chunkservers…
f1aaccd 
… (issue #358)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
confirmed bug Confirmed bug fix committed / please test
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@onlyjob @acid-maker
0