Add secure-fraud-detection demo

[sberyozkin]

Draft PR for a secure-fraud-detection demo.

The startup issue described here earlier was resolved as advised by Georgios

[geoand]

Nice!

I am guessing that the error you are seeing is due to maven not being configured with to make javac retain method parameter names.

[sberyozkin]

Thanks @geoand, yes, I was wondering how exactly does the translate demo work which has 2 parameters, and now I've found https://github.com/quarkiverse/quarkus-langchain4j/blob/main/samples/cli-translator/pom.xml#L12 :-)

[sberyozkin]

@geoand @jmartisk
I'm getting closer to finalizing the demo flow, a user accesses the login page where a Google authentication option is offered, user logs in and is redirected to the fraud detection entry page where the user can choose to check it by amount or distance.
This in itself is useful enough to show how to use the authentication token, but I think the most interesting part is actually how to utilize the fact the OIDC session is available for correlating multiple fraud queries to get more interesting responses each time the same authenticated user tries to check the fraud status, so I'll play around next with the custom memory id idea.
Once we are all happy with the demo flow, I'll fix README

[sberyozkin]

@geoand I prototyped a custom DefaultMemoryIdProvider and in this demo I'd like to do something simple, since no WebSockets are involved, like adding a boolean property indicating if the customer has already issued a given query before (not 100% sure how to do it yet :-), right now I'm just hoping the existing chat history can be correctly detected), and then later in the secure variation of the csv chat box demo, use the security identity info in the memory id to impact the SQL query.

But the following error is becoming quite persistent:

Resulted in: java.lang.RuntimeException: dev.ai4j.openai4j.OpenAiHttpException: {
  "error": {
    "message": "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.",
    "type": "invalid_request_error",
    "param": "messages.[1].role",
    "code": null
  }
}
	at dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry(RetryUtils.java:195)
	at dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:229)
	at dev.langchain4j.model.openai.OpenAiChatModel.generate(OpenAiChatModel.java:153)
	at dev.langchain4j.model.openai.OpenAiChatModel.generate(OpenAiChatModel.java:118)
	at dev.langchain4j.model.chat.ChatLanguageModel_XNMsOaekknG7BdNZ5YSUkjh1SqE_Synthetic_ClientProxy.generate(Unknown Source)
	at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.doImplement(AiServiceMethodImplementationSupport.java:228)
	at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.implement(AiServiceMethodImplementationSupport.java:97)
	at io.quarkiverse.langchain4j.sample.FraudDetectionAi$$QuarkusImpl.detectAmountFraudForCustomer(Unknown Source)
	at io.quarkiverse.langchain4j.sample.FraudDetectionAi$$QuarkusImpl_Subclass.detectAmountFraudForCustomer$$superforward(Unknown Source)
	at io.quarkiverse.langchain4j.sample.FraudDetectionAi$$QuarkusImpl_Subclass$$function$$1.apply(Unknown Source)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
	at io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$3(FaultToleranceInterceptor.java:253)
	at io.smallrye.faulttolerance.core.InvocationContext.call(InvocationContext.java:20)
	at io.smallrye.faulttolerance.core.Invocation.apply(Invocation.java:29)
	at io.smallrye.faulttolerance.core.timeout.Timeout.doApply(Timeout.java:55)
	at io.smallrye.faulttolerance.core.timeout.Timeout.apply(Timeout.java:30)
	at io.smallrye.faulttolerance.FaultToleranceInterceptor.syncFlow(FaultToleranceInterceptor.java:255)
	at io.smallrye.faulttolerance.FaultToleranceInterceptor.intercept(FaultToleranceInterceptor.java:182)
	at io.smallrye.faulttolerance.FaultToleranceInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
	at io.quarkiverse.langchain4j.sample.FraudDetectionAi$$QuarkusImpl_Subclass.detectAmountFraudForCustomer(Unknown Source)
	at io.quarkiverse.langchain4j.sample.FraudDetectionAi$$QuarkusImpl_ClientProxy.detectAmountFraudForCustomer(Unknown Source)
	at io.quarkiverse.langchain4j.sample.FraudDetectionResource.detectBaseOnAmount(FraudDetectionResource.java:39)
	at io.quarkiverse.langchain4j.sample.FraudDetectionResource_Subclass.detectBaseOnAmount$$superforward(Unknown Source)
	at io.quarkiverse.langchain4j.sample.FraudDetectionResource_Subclass$$function$$2.apply(Unknown Source)

Can you please, when you have a few mins, have a quick look at FraudDetectionAi and both TransactionRepository and CustomerRepository, does something look wrong ? I've seen this error on the web, but I'm not sure yet where to start digging, thanks

[sberyozkin]

It is transient though, after some delay, I've restarted and I'm getting an interesting response, There are no recent transactions for the customer John Doe with the email john.doe@example.com., and then later I got a normal response.
Probably related to the OpenAi session left hanging after some restarts, so may be some extra close is needed.

[sberyozkin]

In any case, I'm moving away from using tools in favor of ContentRetriever, yet to be implemented similarly to how it is done in the csv chatbot demo, since it gives a nice option to use Query.getMetadata().getMemoryId, but I'm keeping it as a separate commit in case I get stuck :-)

Update: I've followed with a few minor updates and squashed everything, it will be easy to get back to tools if necessary

[jmartisk]

Gonna look into this one in a bit!

[sberyozkin]

@geoand @jmartisk I think this is ready for review now, have a look please, if possible, test with Google.

Later, I'll follow up with hopefully a more advanced variation involving web sockets, once Quarkus 3.11 or later is used.

[sberyozkin]

The custom ContentRetriever has a request context activated to be able to read from Panache repositories.

I was hoping that may be I can add @Authenticated there as well and get SecurityIdentity injected, instead of using a custom memory id to capture a minimum session state in the form of the authenticated user's name and email, like

@ActivateRequestContext
@Authenticated
List<Content> retrieve(Query query) {}

But it fails with

jakarta.enterprise.context.ContextNotActiveException: RequestScoped context was not active when trying to obtain a bean instance for a client proxy of CLASS bean [class=io.quarkus.vertx.http.runtime.CurrentVertxRequest, id=0_6n6EmChCiiDdd8HelptG_A0AE]
	- you can activate the request context for a specific method using the @ActivateRequestContext interceptor binding
	at io.quarkus.arc.impl.ClientProxies.notActive(ClientProxies.java:70)
	at io.quarkus.arc.impl.ClientProxies.getSingleContextDelegate(ClientProxies.java:30)
	at io.quarkus.vertx.http.runtime.CurrentVertxRequest_ClientProxy.arc$delegate(Unknown Source)
	at io.quarkus.vertx.http.runtime.CurrentVertxRequest_ClientProxy.getOtherHttpContextObject(Unknown Source)
	at io.quarkus.resteasy.reactive.server.runtime.QuarkusCurrentRequest.get(QuarkusCurrentRequest.java:21)
	at org.jboss.resteasy.reactive.server.core.CurrentRequestManager.get(CurrentRequestManager.java:8)
	at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:39)
	at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_AuthenticatedInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
	at io.quarkiverse.langchain4j.sample.FraudDetectionContentRetriever_Subclass.retrieve(Unknown Source)
	at io.quarkiverse.langchain4j.sample.FraudDetectionContentRetriever_ClientProxy.retrieve(Unknown Source)

@michalvavrik, does it ring any bell to you ?

It is strange because @ActivateRequestContext is there, it looks like it is lost during the context propagation...

So I guess for now, using a custom memory id is OK, in fact, it can make sense to combine eventually both @Authenticated and the custom memory id to compare the injected identity with the info from this memory id to assert within the retriever that the AI conversation is essentially bound to the current identity to avoid any chances of the retriever being called outside of the current security context. So I'd say the above exception is not a big problem for now as @Authenticated is already protecting the entry Login and FraudDetectionResource resources.

[michalvavrik]

It is strange because @ActivateRequestContext is there, it looks like it is lost during the context propagation...

StandardSecurityCheckInterceptor has priority PLATFORM_BEFORE while ActivateRequestContextInterceptor has PLATFORM_BEFORE + 100. Quarkus Security extension standard security interceptors have LIBRARY_BEFORE. As long as you fix this bug with anything lesser than LIBRARY_BEFORE, it will be fine. Sorry, I caused this (quite a long time ago now). You want to skip Quarkus Security interceptors so you need to run the interceptors in Quarkus REST before them (users complained security checks are repeated).

[michalvavrik]

So I guess for now, using a custom memory id is OK, in fact, it can make sense to combine eventually both @authenticated and the custom memory id to compare the injected identity with the info from this memory id to assert within the retriever that the AI conversation is essentially bound to the current identity to avoid any chances of the retriever being called outside of the current security context. So I'd say the above exception is not a big problem for now as @authenticated is already protecting the entry Login and FraudDetectionResource resources.

I know practically nothing about this project and code, but let me note that you will not be able to use standard security annotations on the FraudDetectionContentRetriever or down the stream because the exception you are getting means that CDI request context has not been activated by Quarkus REST (e.g. at that point, it's already deactivated for some reason and ActivateRequestContext will activate brand new context). Which means no SecurityIdentity unless I am missing something custom you do there.

[sberyozkin]

Thanks @michalvavrik for the feedback, getting to checking comments only now... OK, I'll open a Quarkus issue to consider what can be done with the priorities.
I've only tried it to consider optimizing the way the custom identity representation is passed through the various Quarkus LangChain4j beans, for now using a custom memory id only is fine IMHO.