8000 Gc stats fix by mdelvecchio-jsc · Pull Request #2825 · oxcaml/oxcaml · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Gc stats fix #2825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

mdelvecchio-jsc
Copy link
Contributor

Fixes an issue where Gc.quick_stat () would not update minor words until a gc promotion happened. Also fixes a mis-reporting of minor words on caml_do_exit.

Testing: In current HEAD, repeatedly calling Gc.quick_stat () would not update the minor words until you allocate enough to cause a gc, but now it updates on every call to Gc.quick_stat, which matches the behavior of the Ocaml 4 runtime.

@ncik-roberts ncik-roberts self-requested a review July 19, 2024 19:39
@ncik-roberts
Copy link
Contributor
ncik-roberts commented Jul 19, 2024

This seems right to me! @stedolan should also glance. We should upstream this if it's correct.

@@ -57,11 +57,18 @@ void caml_accum_alloc_stats(
acc->forced_major_collections += s->forced_major_collections;
}

double caml_minor_words(caml_domain_state *local)
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function shouldn't take a caml_domain_state* parameter, because its mechanism only works to compute the minor words of the current domain. Other domains (a) do not synchronise accesses to young_end / young_ptr, so this code causes a data race and (b) do not keep these values up to date, since young_ptr is kept in a register while running OCaml code.

void caml_collect_alloc_stats_sample(
caml_domain_state *local,
struct alloc_stats *sample)
{
sample->minor_words = local->stat_minor_words;
sample->minor_words = caml_minor_words(local);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is in the wrong place, since this only makes sense to do for the current domain (see other comment)

@stedolan
Copy link
Contributor

It would be good to get a clear spec (and ideally, tests) for what the intended behaviour actually is in multi-domain programs. There are several functions that compute stats about the GC, including stat, quick_stat, counters and minor_words. In the switch to OCaml 5, we needed to pick whether these refer to single-domain or whole-system stats.

The original design was that stat and quick_stat would be whole-system stats, sampled periodically but not immediately up-to-date (computing exact up-to-date stats is expensive), while counters and minor_words would be single-domain and up-to-date.

This changed at some point, so now stat and quick_stat are sampled stats for other domains plus immediate stats for the current domain. That patch isn't quite right though, because it gets the wrong value for minor allocations, and I think the intent here is to change that. So, is the idea that after this patch, the minor allocations / promotions fields of quick_stat and counters should change by the same amount if only one domain is allocating?

What sort of code motivated this change? If it was using quick_stat to measure allocations of some code, then even with this change it'll still break in a multi-domain program, and maybe it should be changed to use counters instead.

@ncik-roberts
8000
Copy link
Contributor
ncik-roberts commented Jul 22, 2024

It was a single-domain program that motivated this change: Users noticed that Gc.quick_stat () / Gc.stat () weren't picking up new minor allocations. (We were confused for a little while too.)

That patch isn't quite right though, because it gets the wrong value for minor allocations, and I think the intent here is to change that. So, is the idea that after this patch, the minor allocations / promotions fields of quick_stat and counters should change by the same amount if only one domain is allocating?

Yes.

I'd be in favor of making the necessary changes in this patch to just dereference young_ptr/young_end for the local domain. I'd hazard that many programs continue to be single domain, and this will ensure that quick_stat behaves more as it did in 4.14. What do you think about that? (In my view, the standard is whether we'd be happy to upstream this change.)

@stedolan
Copy link
Contributor

I'd be in favor of making the necessary changes in this patch to just dereference young_ptr/young_end for the local domain. I'd hazard that many programs continue to be single domain, and this will ensure that quick_stat behaves more as it did in 4.11. What do you think about that? (In my view, the standard is whether we'd be happy to upstream this change.)

Sounds good to me. I'm not completely happy with this meaning for quick_stat, but it's clearly an improvement over the current behaviour, which attempts to have this special case for the current domain but doesn't get it quite right.

Signed-off-by: Thomas Del Vecchio <tdelvecchio@janestreet.com>
@stedolan
Copy link
Contributor

Apologies, I missed that more code had been pushed here addressing my comments above! At this stage, let's wait until 5.2 is merged (since it affects a bunch of the same code), but then I'll re-read this and we should merge it.

@mshinwell
Copy link
Collaborator

@stedolan ping

@mshinwell
Copy link
Collaborator

@stedolan ping again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0