Memory Leak: C# Objects passed to Lua do not get Garbage Collected until Lua is Closed

[notexplosive]

Hi! I've been using NLua for a little while and I'm a big fan generally speaking! However I think I've found a pretty gnarly memory leak that makes NLua basically unusable for me.

If I acquire a LuaFunction in C# and then .Call() it with a parameter. The corresponding Lua seems to take a reference to that parameter and does not let go of it until the Lua is Closed.

A simple example:

lua = new Lua();

// Assume that `myFunction` is defined in lua

var thing = new Thing();
(lua["myFunction"] as LuaFunction).Call(thing); // thing will not get garbage collected until lua is Closed.

The way my project is setup, I spin up one Lua at the start of the game and it persists for the entire game session. I use the above way of calling code as my primary way for C# code to talk to Lua. This means every single time I use Call() I leak a little bit of memory.

Repro

I was able to reproduce this bug with 2 files. Canary.cs and Program.cs with TargetFramework as net6.0

Canary.cs

namespace LuaMemoryTest;

public class Canary
{
    private readonly string _name;
    private int _tweetCount;

    public Canary(string name)
    {
        _name = name;
        if (Canary.InstanceCounts.ContainsKey(name))
        {
            Canary.InstanceCounts[name]++;
        }
        else
        {
            Canary.InstanceCounts[name] = 1;
        }
    }

    private static Dictionary<string, int> InstanceCounts { get; } = new();

    ~Canary()
    {
        Canary.InstanceCounts[_name]--;
    }

    public static void PrintStatus()
    {
        foreach (var pair in Canary.InstanceCounts)
        {
            Console.WriteLine($"Number of \"{pair.Key}\" Canaries: {pair.Value}");
        }
    }
    
    public void Tweet()
    {
        // Does nothing meaningful, I just wanted there to be some "work" happening in this method
        _tweetCount++;
    }
}

Program.cs

using LuaMemoryTest;
using NLua;

// Sanity check: Create a Canary on the stack and do nothing with it. (GC will clean it up later)
for (int i = 0; i < 5000; i++)
{
    new Canary("C# Stack from For Loop");
}

var lua = new Lua();
lua.DoString("myFunction = function(canary) canary:Tweet() end");
lua.DoString("myTable = { memberFunction = function(canary) canary:Tweet() end }");

// Call myTable.memberFunction() and pass in a new Canary each time
for (int i = 0; i < 5000; i++)
{
    ((lua["myTable"] as LuaTable)!["memberFunction"] as LuaFunction)!.Call(new Canary("Passed to Lua Table Function"));
}

// Call myFunction() and pass in a new Canary each time
for (int i = 0; i < 5000; i++)
{
    (lua["myFunction"] as LuaFunction)!.Call(new Canary("Passed to Lua Global Function"));
}

// Full GC
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();

Console.WriteLine("BEFORE DISPOSE");
Canary.PrintStatus();

lua.Close();
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();

Console.WriteLine("AFTER DISPOSE");
Canary.PrintStatus();

Output:

BEFORE DISPOSE
Number of "C# Stack from For Loop" Canaries: 1
Number of "Passed to Lua Table Function" Canaries: 904
Number of "Passed to Lua Global Function" Canaries: 5000
AFTER DISPOSE
Number of "C# Stack from For Loop" Canaries: 1
Number of "Passed to Lua Table Function" Canaries: 1
Number of "Passed to Lua Global Function" Canaries: 1

Notes

• Even in the best case I end up with 1 of each Canary. I don't know if this is a bug in my reference counter code or some quirk of how the GC works. Regardless, I think the 5000 Canaries proves there's something weird happening here.
• The exact number of Canaries varies from run to run, but not by much (on the order of + or - 5 Canaries on my machine)
• Swapping the order of the "Table Function" and "Global Function" loops yields different results (still high numbers but not necessarily 5000)
• Even if we don't call canary:Tweet() we get similar (but not identical?) results.
• I'm on dotnet version 8.0.101, with TargetFramework net6.0

[notexplosive]

Small update! I found that if I used lua.State.GarbageCollector(LuaGC.Collect, 0); the Lua runtime would let go of all the parameter objects. This is an acceptable stopgap for me.