-
Notifications
You must be signed in to change notification settings - Fork 333
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
help request: Not able to use custom plugin with composite architecture approach #2171
Comments
@shantanu10 Thanks for reporting. I am checking this one out. |
@shantanu10 So your plugin is successfully loaded by APISIX and also it's logic is being executed properly right? So functionally you're all good but you're just getting this error in the ingress controller? Am I right? |
@shantanu10 Can you also provide some preceeding and trailing info/debug.. logs from this sync error in ingress controller? |
@shantanu10 Hi! I was having a similar issue when working with the 'standalone' (composite deployment) of the ingress-controller. We found out that this is due to the way the schemas (configuration validation) are loaded by the ingress-controller. Here is an approximate snapshot of the trail to follow (if interested):
Long story short, as a temporary fix what I did was create another mounted file for the
You can obtain all the schemas by calling an endpoint of the
Then call the endpoint and save the result in a file:
(for a bit of clarity on the result I would recommend Finally, apply the ConfigMap:
For the sake of brevity I added apiVersion: apps/v1
kind: Deployment
metadata:
name: ingress-apisix-composite-deployment
namespace: ingress-apisix
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-apisix-composite-deployment
template:
metadata:
labels:
app.kubernetes.io/name: ingress-apisix-composite-deployment
spec:
volumes:
- name: apisix-config-yaml-configmap
configMap:
name: apisix-gw-config.yaml
- name: plugin-schema-json
configMap:
name: plugin-schema-json
containers:
- livenessProbe:
tcpSocket:
port: 8080
# [...]
# [...]
name: ingress-apisix
image: apache/apisix-ingress-controller:1.7.0
volumeMounts:
- mountPath: /ingress-apisix/conf/apisix-schema.json
name: plugin-schema-json
subPath: apisix-schema.json
# [...] Hope it helps! Looking forward to having a better solution coming from contributors 😸. I would propose using the |
Thanks @lucmichea . This approach worked out! |
Glad I could help! @Revolyssup if you could provide some feedback on the solution I provided long term, I would love to discuss further! I can open another thread / a draft MR to cover this. |
Thank you for sharing these workarounds. We are adding custom-plugins as well that require us to extend the schema. Sadly this extension is making the configmap too large and so we have had to trim the list of plugins in the schema. It would be very helpful if plugin support were improved for standalone/composite architecture given that it is the recommend architecture for k8s deployments. |
Yes, the I usually use Helm for versioning and creating a chart containing the If you are experiencing other issues concerning the size of the file I would be happy to take a look as I would be learning another limitation. Although, it would be quite out of subject for this thread. On another note, I plan on trying to do a PR containing the full integration of custom-plugins in the standalone mode once I have a bit of time. As I didn't receive any feedback here. |
Description
Hi team,
I am facing one issue where I am trying to use a custom plugin with apisix when deployed in composite architecture mode. I followed the approach mentioned here:
https://apisix.apache.org/blog/2023/10/18/ingress-apisix/#install-apisix-ingress-controller
https://apisix.apache.org/docs/ingress-controller/composite/
and to create a custom plugin I used the approach mentioned here:
https://apisix.apache.org/docs/ingress-controller/tutorials/using-custom-plugins/
After this when I tried to deployed the following apisix route resource, its not able to get synced. I checked the ingress controller logs and found these errors:
To verify whether custom plugin is actually loaded by apisix, I exec into the ingress-controller pod and ran the admin api for plugins (https://apisix.apache.org/docs/apisix/admin-api/#plugin-api). I can see that the plugin was there in the response. I also got the response for the
plugins/{{plugin_id}}
api for my custom plugin (custom-oauth).So I can see that the error is only coming when ingress-controller is trying to sync the apisixRoute with apisix. Somehow it's not able to find the custom plugin. The issue is not happening when I am using any built-in plugin. I really need some help out here. I will be very grateful if you can help here. Thanks in advance.
Changes I made in the composite.yaml
Environment
apisix version
):3.2.2
uname -a
):openresty -V
ornginx -V
):curl http://127.0.0.1:9090/v1/server_info
):curl http://127.0.0.1:12379/version
as its a mock etcd service)luarocks --version
): not usingThe text was updated successfully, but these errors were encountered: