DotNext vs Community toolkit

[netcorefan1]

Hello to all,
I 'm using Community Toolkit packages in my projects and I'm currently working with the HighPerformance package (in particular MemoryOwner).

I've discovered DotNext and realized that it basically provides the same, but with much more features that I find extremely useful (the need to resize buffer with no reallocation provided by TryResize in DotNext.Buffers.MemoryOwner is what made me open the issue).

I'm confused and don't know if I should switch or not because both the projects are part of the .Net Foundation.
Are all features provided from DotNext expected to be merged in the Community Toolkit or they should be considered distinct projects?
I also saw that DotNext still targets .Net 6, so I'm wondering if CT is more focused on new frameworks features and DotNext more on retrocompatibility.
Thanks

[sakno]

Hi @netcorefan1 , there is good discussion about differences in memory pooling: #159.

Are all features provided from DotNext expected to be merged in the Community Toolkit or they should be considered distinct projects?

Probably not. DotNext is a collection of libraries. It's not possible to merge just one. Buffer management API in DotNext is very rich and not limited by MemoryOwner<T> type. There are PooledBufferWriter<T>, SparseBufferWriter<T>, SpanReader<T>, etc.

I also saw that DotNext still targets .Net 6

This is true but I'm heavily working on .NET 8 support. Typically, major release of .NEXT is aligned with LTS release of .NET. I don't want to just upgrade dependencies and declare it as .NET 8 compatible. DotNext 4.x is actually compatible with .NET 8. I would like to utilize new features in C# (such as static abstract interface members) and .NET 8 to provide better API. That's why I need to release a new major version (because of breaking backward compatibility).

[sakno]

Mostly, API shape remains the same but some things required incompatible changes. For instance, I don't want to keep multiple overloads such as WriteInt32, WriteInt64 for SpanWriter<T>. Instead, I can utilize Generic Math from .NET 7/8 to provide single WriteLittleEndian method suitable for all numeric types.

[netcorefan1]

Many thanks for your explanation. Yes, DotNext is very rich and I'm already using some helpers and this makes your libraries a must have.
Thanks for the link to the discussion, I'm glad to see that I was not the only one to have such questions.
May be the article needs to be updated. Now even CT version supports IMemoryOwner<T> interface, custom ArrayPool<T> and indexer. I'll need to study the MemoryAllocator delegate type, this seems to be the hardest part and where most of the game is played. Same is for the other types you mentioned.

I have read here that CT version is designed to be fully inlinable and given the fact that in the beginning of .NEXT project you didn't even know that the CT was going the develop the same, I suppose both the implementations could have substantial internal differences. It would be very interesting to see how they compare.
Someone performed and published a benchmark?

The linked repo also talks about the strategy rental (stack vs pool) based on the buffer size and this was something I was prepared to implement for the community toolkit version. Is this something I can easily adapt for DotNext version or you are using a different approach that needs a different implementation?
Many thanks

P.S. My memory could fool me, but if I remember well some benchmarks I saw years ago, System.Math was very slow compared to directly write int32, int64 etc.

[sakno]

The linked repo also talks about the strategy rental (stack vs pool) based on the buffer size

Which one? DotNext already has MemoryRental<T> for that purpose as well as dynamic buffer type with ability to pre-allocate initial buffer on the stack.

Someone performed and published a benchmark?

I don't see a reason for that. Memory rental is much more expensive than calling any instance method of MemoryOwner<T>. Anyway, its methods optimized for inlining as well.

System.Math was very slow compared to directly write int32, int64 etc.

System.Math is a different thing. I meant Generic Math which was introduced since .NET 7 and not presented in earlier versions.

[netcorefan1]

The linked repo also talks about the strategy rental (stack vs pool) based on the buffer size

Which one? DotNext already has MemoryRental for that purpose as well as dynamic buffer type with ability to pre-allocate initial buffer on the stack.

SpanOwner<T> owner = SpanOwner<T>.ShouldRent(length) ? SpanOwner<T>.Rent(length) : stackalloc T[length];
Span<T> span = owner.Span;
DoSomeWorkWithSpan(span);
owner.Dispose();

See SpanOwner.cs (MayStackalloc, ShouldRent) and StringHelper.cs.
The logic behind allocation type is that based on a maximum buffer size allowed for stack, the code automatically switch to memory pool when requested size is above such limit. It seems to me a good trick that user could wrap through some convenient extension methods suitable for their needs.
I had not yet chance to get into MemoryRental and may be there is already something similar, but some quick (and uncompilable) prototype could look to something like:

public static MemoryRental<T> RentFromPoolIf<T>(this MemoryRental<T> mr, int desiredLen, int customStackSizeLimit = 0)
	where T : unmanaged {
	return MemoryRental<T>.ShouldRent(desiredLen, StringHelper.Calculate(customStackSizeLimit)) ?
			   MemoryRental<T>.Rent(desiredLen) :
			   stackalloc T[desiredLen];
}

Someone performed and published a benchmark?

I don't see a reason for that. Memory rental is much more expensive than calling any instance method of MemoryOwner<T>. Anyway, its methods optimized for inlining as well.

Oh, my sincere apologies! Sometimes my mind fools me and I end up with typos thinking that MemoryRental is the equivalent of MemoryOwner due to both having the same prefix.
I was meaning DotNext.Buffers.MemoryRental<T> vs CommunityToolkit.HighPerformance.Buffers.SpanOwner<T> and DotNext.Buffers.MemoryOwner<T> vs CommunityToolkit.HighPerformance.Buffers.MemoryOwner<T>
I understand MemoryRental vs SpanOwner, but I'm still confused about DotNext.MemoryOwner vs CommunityToolkit.MemoryOwner. Despite the same name, same methods and behaviours, from your words seems that they target completely different uses cases and trying to compare both through a benchmark will result in completely different results.

System.Math was very slow compared to directly write int32, int64 etc.

System.Math is a different thing. I meant Generic Math which was introduced since .NET 7 and not presented in earlier versions.

Oh, in this case I'm afraid I will also have to replace my old extensions methods! :-)

[sakno]

public static MemoryRental<T> RentFromPoolIf<T>(this MemoryRental<T> mr, int desiredLen, int customStackSizeLimit = 0)
	where T : unmanaged {
	return MemoryRental<T>.ShouldRent(desiredLen, StringHelper.Calculate(customStackSizeLimit)) ?
			   MemoryRental<T>.Rent(desiredLen) :
			   stackalloc T[desiredLen];
}

This code is invalid because stack-allocated memory cannot escape from the method.

Quick look at MemoryOwner from CTK shows many differences with MemoryOwner from .NEXT:

• It's a class (so it allocates)
• It supports ArrayPool<T> only. So custom memory allocation can be implemented only by deriving from ArrayPool<T>. In the same time, MemoryOwner<T> from .NEXT doesn't have this restriction. It provides ownership over any memory than can be represented by array or IMemoryOwner<T>. For instance, MemoryOwner<T> can represent unmanaged memory as well.