Remove test_and_set operations #263
Conversation
This is not possible to ensure it without locking everything which is slow and uncessary. At an higher level, Irmin could do it by enforcing a single-writer policy, so let Irmin handle this and simplify ocaml-git instead.
| XXX(dinosaure): About the PACK file, we use directly mmap by the | ||
| [MAPPER] abstraction to compute efficiently all. I don't know if | ||
| it's reliable to keep this behaviour when we ensure than we use | ||
| [read] only for the loose object and mmap for the PACK file. |
There was a problem hiding this comment.
@dinosaure do you know if this comment is still up-to-date? What happen if one tries to open the same file a lot? Do we run out of inodes? Is seems that we don't have that weak cache of fd anymore. See #143 for the initial PR.
There was a problem hiding this comment.
To the current implementation the PACK was handle in 2 two:
- when we meet a new PACK file (without the associated IDX file), we read entirely the PACK file (with
readsyscall) continuously to produce the IDX file and calculate some stuff. - for all others, we use only the [MAPPER] abstraction (and, by this way, the
mmapsyscall) and keep file-descriptors of these PACK files along than the program run (so, one file-descriptor per PACK file - exactly one for the IDX file and one for the PACK file)
This is only about PACK file. And it's why, then, I explained it could be relevant to keep this for loose (big) object like blob. So we can delete the pool of open if you want when, only for PACK files, we use directly mmap - see Unpack.Decoder and specifically optimized_get.
There was a problem hiding this comment.
So, I believed this comments stills up-to-date :) .
There was a problem hiding this comment.
But that mmap_threashold variable is only used when doing a test-and-set, e.g. for modifying references. I don't understand how this is used for large blobs.
There was a problem hiding this comment.
Ah, yeah. You are true, so if you delete test_and_set, you should delete mmap_ threshold because ... And because the reference size should not upper than mmap_threshold, it's useless to keep it.
This is not possible to ensure it without locking everything which is
slow and uncessary. At an higher level, Irmin could do it by enforcing
a single-writer policy, so let Irmin handle this and simplify ocaml-git
instead.