Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support OnDemandTLS feature #55

Open
Sexual opened this issue Nov 10, 2020 · 14 comments
Open

Support OnDemandTLS feature #55

Sexual opened this issue Nov 10, 2020 · 14 comments
Assignees
@Embraser01
Labels
enhancement New feature or request

Comments

@Sexual
Copy link

Sexual commented Nov 10, 2020

Is it possible to set up an ingress to support all domains?

E.g: host: * rather than host: foo.com

For my use case, I want to support automatic cert issuing for all domains, but the amount of domains is constantly changing and dynamic and can't be manually set in the standard K8s ingress host values.
@mholt
Copy link
Member

mholt commented Nov 10, 2020

(I can't answer for the maintainers, but I'll just note, in case it is helpful, that * is not a valid wildcard representation of foo.com -- you'd need either *.com or *.*, which, as certificate subjects, most browsers/clients reject.)

@Embraser01
Copy link
Member

Hi

I think what you search for is On Demand TLS together with a default backend. It is not implemented yet but it's the next thing I want to add 🙂

@Sexual
Copy link
Author

Sexual commented Nov 10, 2020

@Embraser01 On-Demand TLS with a default backend sounds like just what I need!

Is there any ETA for this or any advice on where to get started to help implement this feature?

@Embraser01
Copy link
Member

I'm looking to implement it before the end of the month, will update here when I've made some progress

@Embraser01 Embraser01 changed the title Wildcard domains Support OnDemandTLS feature Nov 10, 2020
@Embraser01 Embraser01 added the enhancement New feature or request label Nov 10, 2020
@Embraser01 Embraser01 self-assigned this Nov 10, 2020
@Sexual
Copy link
Author

Sexual commented Dec 11, 2020

I'm looking to implement it before the end of the month, will update here when I've made some progress

Any update regarding this? Thanks

@Embraser01
Copy link
Member

Any update regarding this? Thanks

Yes! I decided to work on an improved version of the controller and refactored a bunch of things. It also add support for OnDemand TLS.

It's not merged yet as there is still some things to finish but the controller should be working.
The code is here and a docker image of the latest commit (this morning) is available here or here.

No documentation yet but it's as easy as adding a few fields in the configmap (JSON schema).

@Sexual
Copy link
Author

Sexual commented Dec 17, 2020

Any update regarding this? Thanks

Yes! I decided to work on an improved version of the controller and refactored a bunch of things. It also add support for OnDemand TLS.

It's not merged yet as there is still some things to finish but the controller should be working.
The code is here and a docker image of the latest commit (this morning) is available here or here.

No documentation yet but it's as easy as adding a few fields in the configmap (JSON schema).

Awesome work! Is this ready to be used in a live environment and are there installation instructions for the improved and refactored ingress?

@Embraser01
Copy link
Member

Awesome work! Is this ready to be used in a live environment and are there installation instructions for the improved and refactored ingress?

The project being still very young, I can't make promises. I can tell you that I've been using it in a live environment for some weeks now and it runs very nicely!
I didn't have time to update the documentation yet, but you can check out the chart folder and use helm to generate your manifests https://github.com/caddyserver/ingress/tree/66c52c682f497022f43ffb529def89c3a8ff3472/charts/caddy-ingress-controller

@Sexual
Copy link
Author

Sexual commented Dec 20, 2020 •
edited

Awesome work! Is this ready to be used in a live environment and are there installation instructions for the improved and refactored ingress?

The project being still very young, I can't make promises. I can tell you that I've been using it in a live environment for some weeks now and it runs very nicely!
I didn't have time to update the documentation yet, but you can check out the chart folder and use helm to generate your manifests https://github.com/caddyserver/ingress/tree/66c52c682f497022f43ffb529def89c3a8ff3472/charts/caddy-ingress-controller

Do you have any insight into how to configure the default backend? I've tried deploying it and checking the logs, it's constantly checking an existing ingress' hosts and trying to issue certificates for them.

Update: I've deployed the updated chart (pr-60 image tag) and I can point a domain to the IP and it loads using HTTP, but does not connect via HTTPS, just gets a standard SSL_ERR.

I've also configured an ingress with the caddy ingress class and it crashes instantly:

[caddy-caddy-ingress-controller-678dcbb84f-v8rln] {"level":"info","ts":1608503299.0471146,"caller":"controller/action_ingress.go:46","msg":"Ingress created (default/caddy-ingress)"} 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] E1220 22:28:22.173607       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] E1220 22:28:19.047582       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] goroutine 66 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] goroutine 81 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:74 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:74 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:48 +0x82 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:48 +0x82 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc000144320, 0xc0004fe390, 0xc0004fe390, 0xc000144320) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc0005981e0, 0xc0004f42d0, 0xc0004f42d0, 0xc0005981e0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004fe390, 0x0, 0x1, 0xc00020cbd0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004f42d0, 0x0, 0x1, 0xc00000e090, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc0001faa80, 0xc0001faa80, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc000268690, 0xc000268690, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc0001faa80, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc000268690, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc0001faa80) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc000268690) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003b6040) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00011b490) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003b6040, 0x2279ee0, 0xc000488240, 0x1, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00011b490, 0x2279ee0, 0xc00037f2f0, 0x1, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003b6040, 0x3b9aca00, 0x0, 0x2002c01, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00011b490, 0x3b9aca00, 0x0, 0xc000418101, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.Until(0xc0003b6040, 0x3b9aca00, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.Until(0xc00011b490, 0x3b9aca00, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:201 +0x244 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] panic: runtime error: invalid memory address or nil pointer dereference [recovered] 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:201 +0x244 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       panic: runtime error: invalid memory address or nil pointer dereference 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] panic: runtime error: invalid memory address or nil pointer dereference [recovered] 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1a7030f] 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       panic: runtime error: invalid memory address or nil pointer dereference 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]  
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1a7030f] 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] goroutine 66 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]  
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] goroutine 81 [running]: 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:55 +0x105 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/runtime/runtime.go:55 +0x105 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] panic(0x1c5b760, 0x32e81d0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc000144320, 0xc0004fe390, 0xc0004fe390, 0xc000144320) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /usr/local/go/src/runtime/panic.go:969 +0x166 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.LoadIngressConfig(0xc0005981e0, 0xc0004f42d0, 0xc0004f42d0, 0xc0005981e0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004fe390, 0x0, 0x1, 0xc00020cbd0, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/ingress.go:59 +0x56f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/caddy.Converter.ConvertToCaddyConfig(0xc00005400e, 0x7, 0xc0004f42d0, 0x0, 0x1, 0xc00000e090, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc0001faa80, 0xc0001faa80, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/caddy/convert.go:91 +0x60 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).reloadCaddy(0xc000268690, 0xc000268690, 0x0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc0001faa80, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:262 +0x67 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).processNextItem(0xc000268690, 0x203000) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc0001faa80) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:246 +0x122 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] github.com/caddyserver/ingress/internal/controller.(*CaddyController).runWorker(0xc000268690) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003b6040) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:222 +0x2b 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00011b490) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003b6040, 0x2279ee0, 0xc000488240, 0x1, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:155 +0x5f 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00011b490, 0x2279ee0, 0xc00037f2f0, 0x1, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003b6040, 0x3b9aca00, 0x0, 0x2002c01, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:156 +0xa3 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00011b490, 0x3b9aca00, 0x0, 0xc000418101, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] k8s.io/apimachinery/pkg/util/wait.Until(0xc0003b6040, 0x3b9aca00, 0xc00053e360) 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:133 +0x98 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] k8s.io/apimachinery/pkg/util/wait.Until(0xc00011b490, 0x3b9aca00, 0xc0000966c0) 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /go/pkg/mod/k8s.io/apimachinery@v0.19.4/pkg/util/wait/wait.go:90 +0x4d 
[caddy-caddy-ingress-controller-678dcbb84f-8lmbj]       /app/internal/controller/controller.go:201 +0x244 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln] created by github.com/caddyserver/ingress/internal/controller.(*CaddyController).Run 
[caddy-caddy-ingress-controller-678dcbb84f-v8rln]       /app/internal/controller/controller.go:201 +0x244

Ingress example:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: caddy-ingress
  annotations:
    kubernetes.io/ingress.class: "caddy"
spec:
  rules:
    - host: "testing.replaced-domain.com"
      http:
        paths:
          - path: /
            backend:
              serviceName: backend
              servicePort: 3000

This error seems to be due to K8s 1.16 not supporting spec.rules.http.paths.pathType? Any ideas?

UPDATE 2: Removing the ingress path fixes the crash. Now my only issue is simply getting the default backend implemented. It's supported in K8s 1.19, but is there a way of implementing it in the same manner as nginx-ingress? https://kubernetes.github.io/ingress-nginx/user-guide/default-backend/

Trying to remove the spec.rules.http.host value (which I thought would lead to all hosts being matched) results in the request just being served by a blank page by caddy and a SSL_ERR again if via HTTPS

@Sexual
Copy link
Author

Sexual commented Dec 26, 2020 •
edited

@Embraser01 So after a lot of experimenting, I've gotten a lot further on this.

  1. Having a wildcard for ALL hosts supporting automatic HTTPS isn't possible as the hostnames must be explicitly set: https://caddyserver.com/docs/automatic-https#activation

Any of the following will prevent automatic HTTPS from being activated, either in whole or in part: Not providing any hostnames or IP addresses in the config

  1. The proxyProtocol is not working preventing actual production use. When trying to access https://example.com (a domain configured in the ingress and pointed to the external load balancer IP). It logs the following errors when trying to access via HTTPS:
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994766.4413562,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:8846: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994766.6447232,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:22483: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-vt5qw] {"level":"debug","ts":1608994766.6778147,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.5.1:16605: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994766.8814533,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.4.1:59504: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994767.8734257,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.4.1:31946: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994768.0778928,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:11642: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-z7r8b] {"level":"debug","ts":1608994768.1217213,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.128.0.61:5770: invalid signature"} 
[caddy-caddy-ingress-controller-7b5c7c9577-vt5qw] {"level":"debug","ts":1608994768.3253047,"logger":"http.stdlib","msg":"http: TLS handshake error from 10.36.5.1:57416: invalid signature"}

Checking kubectl get pod -o wide, only 10.128.0.61 is visible in the list of IPs and strangely is GKE's monitoring-prometheus-node-exporter

When trying to access via HTTP, it provides a white error page with the text: 400 Bad Request

@gja gja mentioned this issue Jan 5, 2021
Merged
@Embraser01
Copy link
Member

  1. Having a wildcard for ALL hosts supporting automatic HTTPS isn't possible as the hostnames must be explicitly set: caddyserver.com/docs/automatic-https#activation

Could you provide an ingress .yml and the configmap .yml files that would help me reproduce your issue?
And if you also can provide us the applied config.json (it's logged by the controller).

  1. The proxyProtocol is not working preventing actual production use. When trying to access https://example.com (a domain configured in the ingress and pointed to the external load balancer IP). It logs the following errors when trying to access via HTTPS:

For now, when PROXY protocol is enabled, it prevent any connection that do not use PROXY protocol. I don't know exactly your setup but I know that in order to enable PROXY Protocol in AWS, I had to make sure the load balancer in front of Caddy is in ip-mode.

With ip mode enabled: LoadBalancer -> Caddy Ingress Controller Pod
Without it: LoadBalancer -> Node -> Caddy Ingress Controller Pod

@kyranb
Copy link

kyranb commented Feb 19, 2021

Just confirming, thanks to #65 will on demand TLS for any host name be possible? As is currently possible when using Caddy standalone (not as an ingress).

@Sexual
Copy link
Author

Sexual commented Mar 3, 2021

@Embraser01 It appears that this is due to the PROXY protocol not being something that is explicitly able to be set on GKE. https://projectcontour.io/guides/proxy-proto/

I'm not too sure how this works with ingress-nginx though as I have no issues with that.

Is there any workarounds to this issue?

@RickFoland
Copy link
Contributor

I'd also like to confirm that OnDemand is supposed to be working. I spun up the ingress controller with --set ingressController.config.onDemandTLS=true and although it issues certs as expected, it serves a blank page instead of following the backend specified in my ingress resource.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Embraser01 Embraser01

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@mholt @kyranb @Sexual @Embraser01 @RickFoland