Garbage collection not working

RainerMagerRainerMager JPMember ✭✭

I'm trying to figure out (and hopefully reduce) memory usage in my app. I've tracked down one large memory usage to when I allocate a byte[] of an image. That byte[] is transient, but doesn't seem to be garbage collected. I've managed to reproduce the issue in a simple repro:

Console.WriteLine("A " + GC.GetTotalMemory(true)); byte[] bytes = new byte[2048 * 2048 * 4]; Console.WriteLine("B " + GC.GetTotalMemory(true)); GC.Collect(); Console.WriteLine("C " + GC.GetTotalMemory(true)); bytes = null; System.Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce; GC.Collect(); Console.WriteLine("D " + GC.GetTotalMemory(true));

When I run this early in my app it works as expected:
A 4196960 B 21001016 C 21001024 D 4223816

But when I run it later in the middle of my app the memory is not recovered:
A 5081416 B 21858520 C 21858520 D 21858520

It's running exactly the same code. Does anyone have any ideas?

Posts

  • RainerMagerRainerMager JPMember ✭✭

    I should mention that I forgot to add GC.WaitForPendingFinalizers(); after GC.Collect();, but adding that didn't help.

    To explain a bit more, my app uses OpenGL and is binding a texture with the byte[]. Before doing that memory usage is low. Then I allocate the byte[] by reading the data from a file and bind the texture. After that memory usage is high. My understanding is the OpenGL should use graphics memory for that texture and not need to keep it in "normal" memory. But I'm seeing that memory as never freeing up.

    Because of the above I tried to repro the issue with a simpler scenario, which resulted in the original post. Obviously I'm not running GC.Collect(); in my production code, I'm just using that to try to debug this issue.

  • BrendanZagaeskiBrendanZagaeski USForum Administrator, Xamarin Team Xamurai
    edited August 2016

    I believe you might be running into a technical limitation of the Mono garbage collector (and garbage collection in general in the absence of "precise stack scanning") when allocating moderately large byte arrays.

    Here are a few comments from a bug report where I learned about this.

    Engineer:

    This is false pinned, most likely from the stack. It's reproducible on 32-bit desktop Mono, too, provided the arrays are big enough. If I allocate 128M arrays I reliably get 5 or 6 of them pinned from the stack, so the heap never shrinks below 640+M.

    The only way to properly fix this would be to implement precise stack scanning.

    Me:

    A few quick follow-up questions:

    1. If I'm understanding correctly, the memory that is false pinned by the stack behaves like any other pinned memory. So it could in principle contribute to an out-of-memory error. Correct?

    2. If you managed the memory "by hand" completely outside of the garbage collector, that could be one way to work around the issue, yes? For example, if I replace the contents of the Allocate() method in the test case with the following, that seems to let me allocate and free memory without interacting with the GC at all:

      IntPtr mem = Marshal.AllocHGlobal(size);
      Marshal.FreeHGlobal(mem);
      

    Engineer:

    1. Correct.

    2. Yes. Of course you'll also have to manage its lifetime by hand.

  • RainerMagerRainerMager JPMember ✭✭

    Hmm, interesting. I'll look into managing this memory by hand. Fortunately I only allocate this in one class, so I think it shouldn't be too hard to manage.

  • RainerMagerRainerMager JPMember ✭✭

    So I've mostly got this working using explicit memory allocation as you suggested. Doing that keeps memory usage to around 5MB, which is a lot better than then ~40MB it was before. Unfortunately mono now sometimes crashes (not my app, mono itself) which I need to continue to debug. I'm hypothesizing I'm writing outside of my allocated memory someplace.

    If only there was a safe way to read/write from/to a byte[]... oh wait, that's what I started with. :neutral:

  • TireDevTireDev AUMember

    Yeah, this one is going to be workarounds and testing galore. You're pretty much at the mercy of the uncontrollable garbage collector with very little indication of what's happening under the hood. It's just one of those limitations. I had a similar issue that I ended up having to resolve in a messy way. Good luck! :)

  • RainerMagerRainerMager JPMember ✭✭

    I got it working using manual memory management. Fortunately the OpenGL texture bing calls take an IntPtr, so it was relatively painless. My memory usage dropped from ~40 to 50 MB down to around 6.

Sign In or Register to comment.