Can't get filtering working with dynamic schemas #6975
Replies: 8 comments 21 replies
-
|
Here's a minimal reproduction that @pblasucci put together. |
Beta Was this translation helpful? Give feedback.
-
|
@jimitndiaye I saw on #6272 that you appear to have gotten this working. Could you share any insights here? |
Beta Was this translation helpful? Give feedback.
-
|
@jimitndiaye did you have existing .net types for the types you were adding to the schema using the dynamic schema API? |
Beta Was this translation helpful? Give feedback.
-
|
I tried adding a called to What did you do to get filtering working with a dynamically created schema? |
Beta Was this translation helpful? Give feedback.
-
I did not. I had a type that was effectively a dictionary of key-value pairs. |
Beta Was this translation helpful? Give feedback.
-
AddFiltering registers the built-in queryable filter convention as default. My implementation did not use that. Instead I had a custom filter provider. The implementation took alot of inspiration from Hotchocolate's mongodb query provider. Thus I provided custom FilterInputType implementations as well as similar implementations for projections and sorting. You may not need all that since your datasource seems to be based on Queryable (EF?). You will definitely need custom FilterInputType implementations though since you are basing your filters on dynamic types - the default implementation uses reflection to discover the filter fields. The error message is suggesting it can't find the default filter convention though which should have been registered by AddFiltering. You may try using the overload that takes the name of a specific provider. |
Beta Was this translation helpful? Give feedback.
-
|
Our data source is not based on Queryable. So the approach you took might be what we require. We were attempting to do it this way because we have a few types that backed by .net types and we are using an annotation-first approach. So, with your approach, when you were creating types, did you have something that was equivalent to:
To set up the filtering? I've been wondering if the One reading of your answer suggests that could be the case. |
Beta Was this translation helpful? Give feedback.
-
|
Our startup is as follows: The AddDataverse*** functions are custom extension methods that do same as the equivalent In the type module we have sections of code like: That For the actual query fields we register them as such: You'll notice the use of custom FilterInputType for each entity type. Basically we inherit from Ultimately, I recommend you look at the MongoDb code within Hotchocolate and see about replacing the mongodb-specific implementations with custom code that talks to your own datasource. |
Beta Was this translation helpful? Give feedback.
-
|
The filter middleware has no dependency on projections. You can use filtering without projections. When using MongoDB I recommend not using projections at all as performance is worse if not all fields are indexed. |
Beta Was this translation helpful? Give feedback.
-
|
@jimitndiaye what does |
Beta Was this translation helpful? Give feedback.
-
|
@jimitndiaye also, I can't find |
Beta Was this translation helpful? Give feedback.
-
|
It is basically the same as namespace HotChocolate.Types;
public static class DataverseFilterObjectFieldDescriptorExtensions
{
private static readonly MethodInfo _factoryTemplate =
typeof(DataverseFilterObjectFieldDescriptorExtensions)
.GetMethod(nameof(CreateMiddleware), BindingFlags.Static | BindingFlags.NonPublic)!;
/// <summary>
/// Registers the middleware and adds the arguments for Dataverse Filtering
/// </summary>
/// <param name="descriptor">
/// The field descriptor where the arguments and middleware are
/// applied to
/// </param>
/// <param name="filterInputType">The <see cref="EntityFilterInputType" /> instance</param>
public static IObjectFieldDescriptor UseDataverseFiltering(
this IObjectFieldDescriptor descriptor,
EntityFilterInputType filterInputType)
{
if (descriptor is null) throw new ArgumentNullException(nameof(descriptor));
if (filterInputType is null) throw new ArgumentNullException(nameof(filterInputType));
if (filterInputType is {FilterableAttributes.Length: > 0})
return UseFilteringInternal(descriptor, filterInputType,
Constants.Connectors.Dataverse.ConnectorName);
return descriptor;
}
private static IObjectFieldDescriptor UseFilteringInternal(
IObjectFieldDescriptor descriptor,
IFilterInputType filterTypeInstance,
string? scope)
{
FieldMiddlewareDefinition placeholder =
new(_ => _ => default, key: WellKnownMiddleware.Filtering);
var argumentPlaceholder =
"_" + Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture);
descriptor.Extend().Definition.MiddlewareDefinitions.Add(placeholder);
descriptor
.Extend()
.OnBeforeCreate(
(c, definition) =>
{
var argumentTypeReference = TypeReference.Create(filterTypeInstance, scope);
var argumentDefinition = new ArgumentDefinition
{
Name = argumentPlaceholder,
Type = argumentTypeReference
};
definition.Arguments.Add(argumentDefinition);
definition.Configurations.Add(
new CompleteConfiguration<ObjectFieldDefinition>(
(context, def) =>
CompileMiddleware(
context,
def,
argumentTypeReference,
placeholder,
scope),
definition,
ApplyConfigurationOn.BeforeCompletion,
argumentTypeReference,
TypeDependencyFulfilled.Completed));
argumentDefinition.Configurations.Add(
new CompleteConfiguration<ArgumentDefinition>(
(context, argDef) =>
argDef.Name =
context.GetFilterConvention(scope).GetArgumentName(),
argumentDefinition,
ApplyConfigurationOn.BeforeNaming));
});
return descriptor;
}
private static void CompileMiddleware(
ITypeCompletionContext context,
ObjectFieldDefinition definition,
TypeReference argumentTypeReference,
FieldMiddlewareDefinition placeholder,
string? scope)
{
var type = context.GetType<IFilterInputType>(argumentTypeReference);
var convention = context.DescriptorContext.GetFilterConvention(scope);
var fieldDescriptor = ObjectFieldDescriptor.From(context.DescriptorContext, definition);
convention.ConfigureField(fieldDescriptor);
var factory = _factoryTemplate.MakeGenericMethod(type.EntityType.Source);
var middleware = (FieldMiddleware) factory.Invoke(null,
new object[]
{
convention
})!;
var index = definition.MiddlewareDefinitions.IndexOf(placeholder);
definition.MiddlewareDefinitions[index] =
new(middleware, key: WellKnownMiddleware.Filtering);
}
private static FieldMiddleware CreateMiddleware<TEntity>(
IFilterConvention convention) =>
convention.CreateExecutor<TEntity>();
}Generally simply calling the builtin |
Beta Was this translation helpful? Give feedback.
-
|
Note for others: instead of re-implementing the internals of the private static method public static IObjectFieldDescriptor UseCustomFiltering(this IObjectFieldDescriptor descriptor, IFilterInputType filterInputType, string? scope = null)
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(filterInputType);
// private static IObjectFieldDescriptor UseFiltering(IObjectFieldDescriptor descriptor, Type? filterType, ITypeSystemMember? filterTypeInstance, string? scope)
var _UseFilteringInternal = typeof(FilterObjectFieldDescriptorExtensions)
.GetMethod("UseFiltering", BindingFlags.Static | BindingFlags.NonPublic);
ArgumentNullException.ThrowIfNull(_UseFilteringInternal);
_UseFilteringInternal.Invoke(null, [descriptor, filterInputType.GetType(), filterInputType, scope]);
return descriptor;
}Also easy to add similar extensions for public static IObjectFieldDescriptor UseCustomSorting(this IObjectFieldDescriptor descriptor, ISortInputType sortInputType, string? scope = null)
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(sortInputType);
// private static IObjectFieldDescriptor UseSortingInternal(IObjectFieldDescriptor descriptor, Type? sortType, ITypeSystemMember? sortTypeInstance, string? scope)
var _UseSortingInternal = typeof(SortObjectFieldDescriptorExtensions)
.GetMethod("UseSortingInternal", BindingFlags.Static | BindingFlags.NonPublic);
ArgumentNullException.ThrowIfNull(_UseSortingInternal);
_UseSortingInternal.Invoke(null, [ descriptor, sortInputType.GetType(), sortInputType, scope ]);
return descriptor;
}Both tested to work on HC 13. Of course, by going the reflection route, we're probably opening ourselves up to issues when updating to latest HC later down the road if any internals change. Would be nice if we could get these as built-in overloads. Doesn't seem like it would cause any heartaches to add. Also quick note (mostly for my future self in case I need to come back to this later), depending on your situation, but assuming you're dealing with dynamic schemas/filtering/sorting (like myself and the others in this discussion), it might be suitable to use the .UseFiltering<object>(filterDescriptor =>
{
filterDescriptor.BindFieldsExplicitly(); // Ignores automatic building of fields based on the generic param
filterDescriptor.Name("{unique_filter_name}");
foreach (var field in fields)
{
filterDescriptor.Field(field.Name)
.Type<StringOperationFilterInputType>()
.Extend()
.OnBeforeCreate(e =>
{
e.Handler = new CustomFilterFieldHandler();
});
}
})
.UseSorting<object>(sortDescriptor =>
{
sortDescriptor.BindFieldsExplicitly(); // Ignores automatic building of fields based on the generic param
sortDescriptor.Name("{unique_sort_name}");
foreach (var field in fields)
{
sortDescriptor.Field(field.Name)
.Type<DefaultSortEnumType>()
.Extend()
.OnBeforeCreate(e =>
{
e.Handler = new CustomSortFieldHandler();
});
}
})Downsides being that if you need to tie into any of the other lifecycle methods available when using a custom type derived from Hope this helps someone else cause I spent WAY too much time on this. Thanks @jimitndiaye for all the above details and @SeanTAllen for creating this thread which helped point me into the right direction. P.S. Last note: if you're adding multiple different types of dynamic Filtering/Sorting under the same dynamic schema, it's crucial that the |
Beta Was this translation helpful? Give feedback.
-
|
If I understand what @jimitndiaye did. And what the MongoDB does, this appears to set up an alternate filtering system separate from the one that comes from I have some indications I could make that work. I also have some indications that there's a bug with making When trying to make
Where I already have:
At minimum, there's a misleading error message. Or perhaps there's a HC bug. Or I am missing something that is "obvious" in its non-obviousness. I would be happy to explore further with an HC maintainer should they take notice of this. Otherwise, it seems like creating a full set of custom filters etc from scratch is probably going to be more fruitful. |
Beta Was this translation helpful? Give feedback.
-
|
After additional debugging, we believe this to be a bug and have opened #6983 |
Beta Was this translation helpful? Give feedback.
-
It basically calls |
Beta Was this translation helpful? Give feedback.
-
|
@jimitndiaye Thanks for all your assistance. I was able to figure out from your work what was causing the error we were encountering and make a small fix in Hot Chocolate Data to address the problem. I'll be opening a PR for that in the not so distant future. Understanding what you were doing with your custom middleware was key to figuring out what we needed to do for our use case of "just needing filtering to work with dynamic schema types". |
Beta Was this translation helpful? Give feedback.
-
|
You're welcome. Glad you found a solution. |
Beta Was this translation helpful? Give feedback.
-
|
For folks who see this in the future, we have a PR open to facilitate the approach we took: I will be creating an example "how to" repository and adding that to this discussion thread before closing this. |
Beta Was this translation helpful? Give feedback.
-
We have been trying to get this to work for several days. No luck so far. We are starting to think perhaps there is a bug.
We are creating types using an
ITypeModulethat adds types and add fields to query those types to Query root. Everything with this works fine and we can query fine. When trying to add filtering, we get an error.It very much appears to be the same issue as detailed in this StackOverflow question. However, I tried following the "I seemed to make it work" answer and was unable to do so.
It also appears that it might be the same issue as found in this discussion. I didn't want to add on to that discussion in case it merely looks the same from outward symptoms.
We are using HotChocolate from F#. Here is extracted code. I am putting together a self contained minimal reproduction that hopefully doesn't end up being "too minimal".
Initially I only had
UseFiltering()but added a filter convention based on the stackoverflow answer.That code adds:
The setup for our app is:
We've tried a number of approaches to fix and in almost all cases we end up with a stacktrace like:
Any assistance would be appreciated. I should be able to have a self-contained reproduction sometime tomorrow.
Beta Was this translation helpful? Give feedback.
All reactions