Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Error Checking Code #947

Closed
wants to merge 7 commits into from
Closed

Improve Error Checking Code #947

wants to merge 7 commits into from

Conversation

viceroypenguin
Copy link
Contributor

@viceroypenguin viceroypenguin commented Jan 26, 2023

This PR migrates code contracts for methods to CommunityToolkit.Diagnostics support methods. This has several advantages:

  • Improves chance that error-checking host method will be inlined, possibly improving performance
  • Reduces possibility human error by specifying parameter name only once
  • Reduces boilerplate code.

Fixes #903

Tasks:

  • ArgumentNullException
  • ArgumentOutOfRangeException
  • ArgumentException
  • InvalidOperationException (??)

@viceroypenguin viceroypenguin marked this pull request as draft January 26, 2023 18:04
@viceroypenguin viceroypenguin marked this pull request as ready for review February 1, 2023 16:02
@codecov
Copy link

codecov bot commented Feb 1, 2023

Codecov Report

Merging #947 (1220efb) into master (66f7cab) will decrease coverage by 0.14%.
The diff coverage is 98.42%.

❗ Current head 1220efb differs from pull request most recent head 9a3f721. Consider uploading reports for the commit 9a3f721 to get more accurate results

@@            Coverage Diff             @@
##           master     #947      +/-   ##
==========================================
- Coverage   92.63%   92.49%   -0.14%     
==========================================
  Files         113      113              
  Lines        3421     3464      +43     
  Branches     1056      571     -485     
==========================================
+ Hits         3169     3204      +35     
- Misses        189      195       +6     
- Partials       63       65       +2     
Files Coverage Δ
MoreLinq/Acquire.cs 100.00% <100.00%> (ø)
MoreLinq/Aggregate.g.cs 100.00% <100.00%> (ø)
MoreLinq/AggregateRight.cs 100.00% <100.00%> (ø)
MoreLinq/Append.cs 100.00% <100.00%> (ø)
MoreLinq/Assert.cs 94.11% <100.00%> (ø)
MoreLinq/AssertCount.cs 96.29% <100.00%> (+0.46%) ⬆️
MoreLinq/Batch.cs 94.33% <100.00%> (-3.74%) ⬇️
MoreLinq/Cartesian.g.cs 62.11% <100.00%> (ø)
MoreLinq/Choose.cs 92.30% <100.00%> (ø)
MoreLinq/Consume.cs 100.00% <100.00%> (ø)
... and 92 more

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@atifaziz atifaziz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking up #903 and your effort here!

I noticed one sweeping issue so could I start by asking you to fix misalignments due to tabs? You can find these with:

git grep -P '\t' -- *.cs *.tt *.csproj

I'll get round to a more detailed review after. Thanks!

Copy link
Member

@atifaziz atifaziz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. There are naturally tons of changed files so it's going to take me a few sittings, but I managed get through a quarter of the 120 files and so here are some interim/initial thoughts…

I noticed that we're only using a tiny fraction of the guards from the CommunityToolkit

  • Guard.IsBetween
  • Guard.IsBetweenOrEqualTo
  • Guard.IsGreaterThan
  • Guard.IsGreaterThanOrEqualTo
  • Guard.IsLessThan
  • Guard.IsLessThanOrEqualTo
  • Guard.IsNotNull

I'm therefore leaning towards it being simpler to just embed these into the project instead of taking an additional dependency. A downside might be eventual divergence with time, but I don't expect these to change much and it would also enable some flexibility for tailoring. For example, I wonder if the Guard methods shouldn't be changed to return the valid argument, for some future-readiness.

Do we really need a throw helper for InvalidOperationException? I think you had doubts about the value of this too? I don't think it helps the JIT case. Consistency might be one argument, but a bare throw still reads simpler. I can understand doing this consistently for arguments, but invalid use seems like an outlier.

@@ -31,14 +31,14 @@ static T[] Fold<T>(this IEnumerable<T> source, int count)
{
elements[i] = i < count ? item : throw LengthError(1);
i++;
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert:

Suggested change
}
}

}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something seems to have gone off with the indentation here.

@@ -32,6 +32,7 @@

namespace MoreLinq
{
using CommunityToolkit.Diagnostics;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove unused import:

Suggested change
using CommunityToolkit.Diagnostics;

@@ -27,6 +27,7 @@

namespace MoreLinq
{
using CommunityToolkit.Diagnostics;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove unused import:

Suggested change
using CommunityToolkit.Diagnostics;

@@ -17,7 +17,7 @@

namespace MoreLinq
{
using System;
using CommunityToolkit.Diagnostics;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Convert tab to 4 spaces:

Suggested change
using CommunityToolkit.Diagnostics;
using CommunityToolkit.Diagnostics;

@viceroypenguin
Copy link
Contributor Author

I'm therefore leaning towards it being simpler to just embed these into the project instead of taking an additional dependency. A downside might be eventual divergence with time, but I don't expect these to change much and it would also enable some flexibility for tailoring.

If you want to take the time to copy them from CommunityToolkit and implement them yourself, I won't disagree. That said, a) I'm far too lazy to redo what someone else has already done, and b) I think this goes against principles of working with libraries in the first place.

Namely if every library copies these methods, then now these methods are implemented in each library, increasing dll size and JIT size since they all have to be JIT'd separately, especially for generic methods which need to be genericized separately for each implementation.

Using a single library (CommunityToolkit for cross-TFM, BCL for net8+ which has many of these now implemented) means that this code is shared across all consumers.

For example, I wonder if the Guard methods shouldn't be changed to return the valid argument, for some future-readiness.

I used to think that too, when I started with the Throw library (from amantinband) and switched to CommunityToolkit. However, I've found as I've used Guard.Xxx more, I now prefer the non-return version because it encourages better code readability, specifically, that contractual obligations (not-null, not-null/empty/whitespace, numeric range) are specified and checked at the start of the method.

Do we really need a throw helper for InvalidOperationException? I think you had doubts about the value of this too?

The advantage of this specifically is two-fold:

  1. the calling code reduces JIT code as the throw creates a fairly large footprint

  2. calling code is no-longer allowed to be inlined (inlining is turned off for any method that includes a throw anywhere in the method). This may not be as big of a deal for a throw in the middle of an iterator method, but definitely has an impact on methods that simply return a value. Obviously all of the outer methods that MoreLinq uses are more likely to be inlined, switching to Guard.Xxx, which means many of them may turn into direct calls to the Core method if the JIT can prove that the values are not-null, etc.

    Whether this matters in the cases of the various ThrowHelper calls is less certain, but definitely TryAsCollectionLike() and AggregateRight() will benefit from it. For me, I've grown used to it, so I now actually prefer ThrowHelper. instead of throw new, but I'll concede that to be a usage/personal preference.

Copy link
Member

@atifaziz atifaziz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did another review session and posted some comments. As I did the review, I had some more overall thoughts about this.

We're only using a handful of Guard and ThrowHelper methods from the CommunityToolkit and most of them are plain one-liners. What's more, practically all of the ones we need have found their way into .NET 8. Some of the complexity that comes from the friendly type string formatting that CommunityToolkit does in the error messages doesn't seem to be warranted. This is not about redoing or not leveraging a library, but assessing whether taking a whole dependency is worth 7 dead simple methods that are landing in .NET anyway. The considerations and criteria are often different for an application than an F/OSS library.

I think we should leverage the methods added in .NET instead of relying on the CommunityToolkit. For targets where the helpers are unavailable, we can copy over the 1-2 line implementations from .NET and light them up conditionally. These will only need to be maintained until those targets have been dropped, although I don't expect there'll any maintenance to do since they can't be expected to change much at all with time.

In the related issue, you proposed two options:

Personal recommendation: add a reference to CommunityToolkit.Diagnostics, and update all guard statements to these. This would allow us to rely on a shared implementation of this boilerplate code and if others do as well, then that is mutual code shared.

Alternative recommendation: Add our own Guard and ThrowHelper methods and update all guard statements to these. This would duplicate code that could be shared, but not add a dependency on another library.

I'm still leaning on the second one, but saying that let's go with what's there & coming in .NET.


PS The whole business of helping JIT inline the argument validation is interesting for methods that are expected to be used on hot paths, but since that's where one generally avoids LINQ anyway, the overall benefits are marginal. Moreover, tiered compilation and PGO may never inline in the first place so one may not even be getting the benefits one might expect!


return source.ToListLike() switch
{
{ Count: 0 } => throw new InvalidOperationException("Sequence contains no elements."),
{ Count: 0 } => ThrowHelper.ThrowInvalidOperationException<TSource>("Sequence contains no elements."),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to:

Suggested change
{ Count: 0 } => ThrowHelper.ThrowInvalidOperationException<TSource>("Sequence contains no elements."),
{ Count: 0 } => throw new InvalidOperationException("Sequence contains no elements."),

Comment on lines +80 to +83
{
ThrowHelper.ThrowArgumentOutOfRangeException(
nameof(index), "Insertion index is greater than the length of the first sequence.");
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to:

Suggested change
{
ThrowHelper.ThrowArgumentOutOfRangeException(
nameof(index), "Insertion index is greater than the length of the first sequence.");
}
throw new ArgumentOutOfRangeException(nameof(index), "Insertion index is greater than the length of the first sequence.");

if (startIndex < 0) throw new ArgumentOutOfRangeException(nameof(startIndex));
Guard.IsNotNull(sequence);
Guard.IsGreaterThanOrEqualTo(startIndex, 0);
Guard.IsGreaterThanOrEqualTo(count, 0);

return count switch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there are only two arms left to this switch expression, might as well turn into a conditional (?:).

@@ -1496,7 +1496,8 @@ public static partial class EvaluateExtension
/// <returns>A sequence with results from invoking <paramref name="functions"/>.</returns>
/// <exception cref="ArgumentNullException">When <paramref name="functions"/> is <c>null</c>.</exception>

public static IEnumerable<T> Evaluate<T>(this IEnumerable<Func<T>> functions) => MoreEnumerable.Evaluate(functions);
public static IEnumerable<T> Evaluate<T>(this IEnumerable<Func<T>> functions)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious how did this change leaked into the generated code?

@@ -272,6 +275,6 @@ public IEnumerator<TElement> GetEnumerator()
void IList<TElement>.RemoveAt(int index) => ThrowModificationNotSupportedException();

[DoesNotReturn]
static void ThrowModificationNotSupportedException() => throw new NotSupportedException("Grouping is immutable.");
static void ThrowModificationNotSupportedException() => ThrowHelper.ThrowNotSupportedException("Grouping is immutable.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to:

Suggested change
static void ThrowModificationNotSupportedException() => ThrowHelper.ThrowNotSupportedException("Grouping is immutable.");
static void ThrowModificationNotSupportedException() => throw new NotSupportedException("Grouping is immutable.");

Aside from consistency, there's no JIT-ing benefit here. These are forbidden operations that should be discovered during development.

Comment on lines +84 to +85
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert formatting change:

Suggested change
}
}
}
}

return ToDelimitedStringImpl(source, delimiter, static (sb, e) => sb.Append(e));
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert this formatting change in the corresponding T4 template.

var column = columns[member.Name] ?? throw new ArgumentException($"Column named '{member.Name}' is missing.", nameof(table));

if (GetElementaryTypeOfPropertyOrField(member) is var type && type != column.DataType)
throw new ArgumentException($"Column named '{member.Name}' has wrong data type. It should be {type} when it is {column.DataType}.", nameof(table));

columnMembers[column.Ordinal] = member;
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert formatting change:

Suggested change
}
}

@@ -203,14 +206,14 @@ static MemberInfo GetAccessedMember(LambdaExpression lambda)
var columnMembers = new MemberInfo[columns.Count];

foreach (var member in members)
{
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert formatting change:

Suggested change
{
{

@@ -17,6 +17,7 @@

namespace MoreLinq.Test
{
using CommunityToolkit.Diagnostics;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put system namespaces first (in all files) since the project follows the default for formatting rule dotnet_sort_system_directives_first of true.

Unfortunately, IDE0055 is not enforceable since there are too many exceptions to the rules; it's therefore only configured to be a suggestion:

MoreLINQ/.editorconfig

Lines 84 to 85 in c01e646

# IDE0055: Fix formatting
dotnet_diagnostic.IDE0055.severity = suggestion

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

auto-add code-fix. usually it respects that, but alas...

@viceroypenguin
Copy link
Contributor Author

For targets where the helpers are unavailable, we can copy over the 1-2 line implementations from .NET and light them up conditionally. These will only need to be maintained until those targets have been dropped, although I don't expect there'll any maintenance to do since they can't be expected to change much at all with time.

This is not a bad suggestion. I'll have to think about how to implement in a way that a) isn't too complicated and b) doesn't require tfm based evaluations at the call site.

What are your thoughts about using global usings to hide the complexity. Could do something like:

<ItemGroup Condition="$(TargetFramework) != 'net8.0'">
  <Using Include="MoreLinq.Exceptions.ArgumentNullException" Static="true"/>
</ItemGroup>
<ItemGroup Condition="$(TargetFramework) == 'net8.0'">
  <Using Include="System.ArgumentNullException" Static="true"/>
</ItemGroup>
public static IEnumerable<TSource> Operator<TSource>(this IEnumerable<TSource> source)
{
  ThrowIfNull(source);
  // ...
}

@atifaziz
Copy link
Member

What are your thoughts about using global usings to hide the complexity.

@viceroypenguin I was thinking something along the very same lines! However, instead of statically importing methods, I'd like to try and maintain the type name qualification. Taking the example of one of the Pad overloads, ideally the code looks as you'd write it in .NET 8:

public static IEnumerable<TSource> Pad<TSource>(this IEnumerable<TSource> source, int width,
                                                Func<int, TSource> paddingSelector)
{
    ArgumentNullException.ThrowIfNull(source);
    ArgumentNullException.ThrowIfNull(paddingSelector);
    ArgumentOutOfRangeException.ThrowIfLessThan(width, 0);

    return PadImpl(source, width, default, paddingSelector);
}

The targets that won't have those methods on types could have masking definitions in the MoreLinq namespace (which could be made to take precedence), like so:

#if EXCEPTION_HELPERS

namespace MoreLinq
{
    using System.Diagnostics.CodeAnalysis;
    using System.Runtime.CompilerServices;

    static class ArgumentNullException
    {
        public static void ThrowIfNull([NotNull] object? argument,
                                       [CallerArgumentExpression(nameof(argument))]
                                       string? paramName = null)
        {
            if (argument is null)
                Throw(paramName);
        }

        [DoesNotReturn]
        static void Throw(string? paramName) =>
            throw new System.ArgumentNullException(paramName);
    }
}

#endif

A conditional symbol like EXCEPTION_HELPERS could then be defined in the project file based on the target framework. This is just a rough idea from the head and a little bit of experimentation may be needed to find the sweet spot.

This does mean however that you can't just write throw new ArgumentNullException() anymore, but that may not be a bad thing!

I would also limit this just to the main library project. I don't think we need to extend this approach, for example, to the test project.

What would be the cherry on top is to add an integration test that exercises the inlining (although as I said previously, tiered compilation means you don't always get the expected optimisations). I have some ideas for how this could be pulled off.

@atifaziz
Copy link
Member

atifaziz commented Nov 2, 2023

@viceroypenguin As a pulse check, do you plan to follow-up here based on latest discussion? I'm just asking to triage open issues and PRs.

@viceroypenguin
Copy link
Contributor Author

Yes, but will probably be a while. That's a bit of more work than I have time for right now.

@viceroypenguin viceroypenguin changed the title Migrate Error Checking to CommunityToolkit.Diagnostics Improve Error Checking Code Nov 15, 2023
@viceroypenguin
Copy link
Contributor Author

viceroypenguin commented Nov 15, 2023

@atifaziz Question on which path you want to go here.

Currently, the following code does not work:

#if !NET6_0_OR_GREATER
global using ArgumentNullException = MoreLinq.Exceptions.ArgumentNullException;
#endif

The reason is that each namespace MoreLinq { using System; } creates a tighter binding for ArgumentNullException to System than to the global alias.

I don't have this problem in SuperLinq, because I migrated to implicit usings and file-scoped namespaces. Since these are the recommended patterns, I'd recommend we do a separate PR to do these migrations.

Alternatively, each file can include

#if !NET6_0_OR_GREATER
global using ArgumentNullException = MoreLinq.Exceptions.ArgumentNullException;
#endif

at the top of the file, below the using System;. I think this path is a huge detriment to the usability of each file.

Your call, though.

@viceroypenguin viceroypenguin closed this by deleting the head repository Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update argument validation to favour JIT in-lining
2 participants