[RFC] src/aiori-CEPHFS: New libcephfs backend #217
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Mark Nelson <mnelson@redhat.com>
JulianKunkel
approved these changes
Mar 10, 2020
| .prefix = NULL, | ||
| }; | ||
|
|
||
| static option_help options [] = { |
There was a problem hiding this comment.
great that you use the new options ^-^
| "cannot total data moved"); | ||
| if (tmpMin != tmpMax) { | ||
| if (rank == 0) { | ||
| WARN("inconsistent file size by different tasks"); |
There was a problem hiding this comment.
Nice check, albeit it costs a little performance.
As it assumes to be a collective operation that may lead to unexpected behavior (in terms of AIORI semantics) if not all processes invoke the same function. I'm not 100% sure if we should generally allow that but also not so worried as IO500 runs...
| @@ -197,6 +197,20 @@ AM_COND_IF([USE_RADOS_AIORI],[ | |||
| AC_DEFINE([USE_RADOS_AIORI], [], [Build RADOS backend AIORI]) | |||
| ]) | |||
|
|
|||
| # CEPHFS support | |||
| AC_ARG_WITH([cephfs], | |||
There was a problem hiding this comment.
In the future, the code might benefit from automatic detection of include/library files. It is reasonable to keep it function for now though.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a new aiori backend using libcephfs that is loosely based on the existing POSIX and RADOS backends. It also borrows the "prefix" concept from the DFS backend for an existing POSIX mount point (necessary for ior/mdtest to function properly even when using a library for direct filesystem access). A slight change to libcephfs.h is needed for IOR to properly compile (this does not appear to be necessary for C++ clients using libcpehfs however):
io500 tests on a 10 node in-house test cluster with 2X replication and co-located clients appeared to function properly with similar (though much better in the case of sequential reads) scores vs using the POSIX backend with kernel based CephFS mount points. In the following results, the mdtest easy directories are being round-robin pinned prior to the test, though in the near future ceph will do ephemeral pinning across MDSes automatically with a single top-level xattr.
2020-03-06-RedHatLibCephFS-10-30.zip
Generally, lower scores in unaligned reads/writes and build-up time for dynamic subtree partitioning in the ior and mdtest hard test cases held us back (we actually see higher scores with longer run times!). Given how scores are calculated these will be prime targets for future optimization.
Signed-off-by: Mark Nelson mnelson@redhat.com