8000 GLFW callbacks not released · Issue #821 · dotnet/Silk.NET · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

GLFW callbacks not released #821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #1251
krmr opened this issue Feb 25, 2022 · 14 comments
Closed
Tracked by #1251

GLFW callbacks not released #821

krmr opened this issue Feb 25, 2022 · 14 comments
Assignees
Labels
area-SilkTouch bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@krmr
Copy link
Contributor
krmr commented Feb 25, 2022

Summary

After closing a GLFW rendering window GLFW ande some referenced objects remain in memory.

Steps to reproduce

  • Platform: Desktop
  • Framework Version: .NET 5
  • API: OpenGL
  1. Run the code from 'Tutorial 1.1 - Hello Window' in the debugger with a breakpoint on the closing brace of Main.
  2. Once the breakpoint is hit create a memory snapshot and open it.
  3. Inspecting the snapshot shows that the callbacks created in GlfwWindow (e.g. WindowCloseCallback or FramebufferSizeCallback) are still in memory.

Comments

From some debugging I think this might be due to those callbacks being lambdas/closures referenced by GlfwWindow.

Also I am not sure if GlfwWindow.UnregisterCallbacks acutally unregisters all of them. At least GCUtility.Unpin does not find the corresponding handles. To me it looks like only the most recent one is actually pinned due using PinMode.UntilNextCall.

I could get rid of the objects locally by making two changes:

  1. In Glfw change all Callbacks from [PinObject(PinMode.UntilNextCall)] to [PinObject]
  2. In GlfwWindow.UnregisterCallbacks after calling Unpin also set the fields to null

Since I do not understand this code well enough I doubt this is the right thing to do. The first step was because I noticed that GCUtilities.Pins contained only a single handle at all which causes Unpin not to find the callbacks in there.

Background

I have a service for rendering 3D scenes to images. This creates a new window for each request and I found that the memory usage keeps growing over time. Doing some memory profiling brought up that those Glfw* objects remain in memory.

@krmr krmr added the bug Something isn't working label Feb 25, 2022
@Perksey
Copy link
Member
Perksey commented Mar 5, 2022

Also I am not sure if GlfwWindow.UnregisterCallbacks acutally unregisters all of them. At least GCUtility.Unpin does not find the corresponding handles. To me it looks like only the most recent one is actually pinned due using PinMode.UntilNextCall.

UntilNextCall should pin whatever is the latest value for each individual function, and unpin any old ones. If it's pinning the latest of all calls (i.e. only pinning one delegate at any given time) then this is a bug that needs to be fixed, most likely in the SilkTouch generator.

In Glfw change all Callbacks from [PinObject(PinMode.UntilNextCall)] to [PinObject]

GC utility should unpin all handles, regardless of the pin mode. If it's not, then this is the bug that needs to be fixed in a PR.

In GlfwWindow.UnregisterCallbacks after calling Unpin also set the fields to null

This sounds reasonable, would accept this as a PR.

@Perksey
Copy link
Member
Perksey commented Mar 5, 2022

Will mark this as a community bugfix, as this the smallest memory leak ever, not variable, that in practice is benign.

We'll be happy to accept PRs for fixing this! But I'd like to keep UntilNextCall, so that might mean fixing the generator to do the right thing.

cc'ing @HurricanKai for visibility.

@krmr
Copy link
Contributor Author
krmr commented Mar 6, 2022

Thanks for the feedback.

UntilNextCall should pin whatever is the latest value for each individual function, and unpin any old ones. If it's pinning the latest of all calls (i.e. only pinning one delegate at any given time) then this is a bug that needs to be fixed, most likely in the SilkTouch generator.

Ah, I see. I think in this case the problem is that all corresponding calls to PinUntilNextCall use 0 as the slot. At least I guess it should be a different slot for each function. If that is how thing are supposed to work I agree that UntilNextCall should remain there.

Will mark this as a community bugfix, as this the smallest memory leak ever, not variable, that in practice is benign.

I agree and actually for that reason I avoided to call it memory leak as that usually sounds worse. Also my use is rather special with a service creating a new window for each call.

We'll be happy to accept PRs for fixing this! But I'd like to keep UntilNextCall, so that might mean fixing the generator to do the right thing.

After looking into the code I am not sure yet if I will be able to provide a PR for that... But I can for sure create one to set the callbacks to null after unregistering.

@HurricanKai
Copy link
Member

I'll investigate the slot functionality, I was pretty confident it worked well 🤔

@krmr
Copy link
Contributor Author
krmr commented Mar 7, 2022

Well, when debugging I see that the code generated for GLFW uses 0 as the slot value in each call to PinUntilNextCall.

After looking a bit deeper into the issue I think that the problem is that NativeApiGenerator creates a new instance of MarshalContext in each call to ProcessMethod. This causes GCCount to be 0 which ends up as the slot value.

I gave it a quick try by using a static field to back GCCount and with this the generated code actually uses non-zero values for the slots. But I think I still do not understand that code well enough to tell whether that is a good idea or not…

@Perksey
Copy link
Member
Perksey commented Mar 7, 2022

We could probably use the native API entry point’s hash code as the slot, not sure how you get that from an IMarshalContext but Kai should know

@krmr
Copy link
Contributor Author
krmr commented Mar 7, 2022

The constructor of MarshalContext already has IMethodSymbol as an argument. With that there could be something along the lines of GCCount = methodSymbol.GetHashCode(); in the constructor to initialize the value to something different than zero.

@Perksey
Copy link
Member
Perksey commented Mar 7, 2022

The constructor of MarshalContext already has IMethodSymbol as an argument.

I'd say we'd probably want entryPoint as an argument & property as well (this ProcessMethod parameter) so that the PinObject can access it and use its hash code. After that, I'd say remove AllocateGcSlot and related properties entirely.

That's my take anyway.

@Perksey
Copy link
Member
Perksey commented Mar 7, 2022

To clarify: I'd prefer the slot to be per native entrypoint rather than per entire method, because what if we had two overloads of the same native function but with different delegate types? We still need to free the other one.

By the way, thanks for the engaging conversation! You seem to have caught onto all this really quickly, and your input is really helpful :)

@HurricanKai
Copy link
Member
HurricanKai commented Mar 7, 2022

I'm not a fan of using hash codes for this sort of thing, they can collide and it's hard to debug, understand, track.
I think this code is just generally very neglected and we'll have to do a bit more work to make it function correctly. The GC Utility code isn't looking very rosy either...

@Perksey
Copy link
Member
Perksey commented Mar 7, 2022

The GC utility code has been with us for 3 years now!

@Perksey
Copy link
Member
Perksey commented Jan 11, 2023

@HurricanKai do you have any objections to just using a hash code generated from the entry point here? We could make it a dictionary but dictionaries themselves are based on hash codes so it'd be just as reliable to just use the native entry point's hash code as the GC slot. The main reason I don't want to use a slot allocation mechanism is each overload of the same native function would have its own GC slot (which is not ideal)

@HurricanKai
Copy link
Member

I'd highly prefer just fixing the allocation mechanism to work once per native function, which I think is rather easy. Not sure how using the hash code would work given that GC slots are assigned incrementally into an array.

@Perksey
Copy link
Member
Perksey commented Jan 11, 2023

The GC slots aren't actually used anywhere other than the GC Utility to my knowledge, which accepts the integer as an arbitrary index to map to a list of GC handles. GCUtility.PinUntilNextCall will resolve all GC handles for the given slot.

Right now the slot is always 0. This is the bug, as it's causing only one delegate to be pinned at any given time (well, it's one of the bugs, the other is that GCUtility doesn't do a null check and as a result doesn't properly deallocate).

We need to define that slot value, and I don't think it makes sense to give each individual overload of the same native function a slot. So, this presents us with two options:

  • Allocate a GC slot by mapping the native entry point to a slot using a dictionary (aka a hash map)
  • Let the native entry point be the slot

I could be missing something though!

@Perksey Perksey self-assigned this Jan 22, 2023
@Perksey Perksey mentioned this issue Jan 22, 2023
14 tasks
Perksey added a commit that referenced this issue Feb 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-SilkTouch bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
Status: Done
Development

No branches or pull requests

3 participants
0