External connectors

[nabokihms]

Is your feature request related to a problem?

The main problem that the Dex repo contains a lot of different connectors with various features, purposes, original authors, tests coverage, etc.

It makes Dex maintainers support a lot of custom logic that they are unfamiliar with.

Describe the solution you'd like to see

The thing to start with is writing solid design documentation (an enhancement proposal, you may say).

Nowadays, the most popular solution to this kind of problem is to develop a GRPC API interface. Connectors will be distributed as programs that implement this interface (with the separate codebase, tests, maintainers, etc.)

Examples: terraform providers, Kubernetes CSI.

Describe alternatives you've considered

If we want to move Dex forward, there are not so many good alternatives. It is possible to limit the number of connectors in the core repo and stop merging PR with new connectors until stabilizing already accepted ones. However, this looks like something other than a prospective way.

Additional context

The initial idea and its benefits were previously described in a slack thread.

Proposed solutions

• Go plugins allow loading of connectors from .so files #1302
• GRPC connector grpc connector #1020

List of previously proposed connectors

• [Facebook] Facebook Connector #956 (Callback / Refresh)
• [Onelogin] connector: add Onelogin connector #1174 (Callback / Refresh)
• [SQL] SQL Connecor #1626 (Password / Refresh)
• [Vault] For review: Vault integration #2039 (Callback / Password / Refresh)
• [Apple] Add Apple connector #1905 (Callback / Refresh)
• [CloudFoundry] CloundFoundry connector #1624 (Callback)
• [SMTP] A connector for SMTP authentication #2601 (Password)
• [LDAP cluster] feat: ldap cluster connector #1904 (Password / Refresh)
• [Alibaba Dingtalk] feat(connector/dingtalk):Add Alibaba Dingtalk connector #2914 (Callback / Refresh)

[sagikazarmark]

I think we can stop accepting new connectors until we figure this out.

[candlerb]

I think the connectors fall into three groups:

1. Those which provide variations on standard OIDC (e.g. google, gitlab, github, microsoft, linkedin etc)
2. Those which integrate with foreign identity protocols (saml)
3. Those which act as a password checker, where Dex itself prompts for username and password (e.g. ldap, mock, and passwordDB - although the latter isn't actually a connector at the moment)

Are you thinking of a plugin interface which is capable of all three, or a subset? As far as I can see, the code has different go interfaces for these cases:

                switch conn := conn.Connector.(type) {
                case connector.CallbackConnector: ...
                case connector.PasswordConnector: ...
                case connector.SAMLConnector: ...

My view is: the different flavours of OIDC are where the integration points are needed. SAML is such a different protocol that you need to take over the whole HTTP exchange.

Password checkers could be removed from Dex completely, by moving them into a separate IDP which you talk to using the OIDC connector. This would allow the IDP to implement 2FA or other logic which I don't think really belongs in Dex. A system which allows you to authenticate using just a password, with no form of 2FA, is legacy by design. I'd be very interested in a lightweight, standalone IDP which is backed by passwordDB or LDAP, and also integrates with a 2FA library such as linOTP/privacyIDEA.

But if you do want to keep password checking as a feature of Dex, the LDAP protocol already exists as an extension point; there's no need for a separate gRPC protocol.

[candlerb]

An intermediate option is to make the password exchange more generic, by turning it into a multi-step "challenge/response" - similar to a PAM conversation function. For example, Prompt() and Login() could be repeated multiple times with some state maintained in between, until Login says the exchange is successfully complete

That would allow both basic password checking and some forms of 2FA (e.g. TOTP) - but not modern ones like U2F which need to inject code into the web page.

[candlerb]

BTW, did you consider go native plugins as an alternative to gRPC? (blog)

Of course, it limits plugins to being written in go (although someone could still write a gRPC client as a plugin). It's also currently limited to Linux, FreeBSD and macOS. EDIT: maybe not such a great idea.

[sagikazarmark]

That's actually the reddit post I wanted to link :)

[sagikazarmark]

But if you do want to keep password checking as a feature of Dex, the LDAP protocol already exists as an extension point; there's no need for a separate gRPC protocol.

I wonder if implementing the LDAP protocol is complicated enough to warrant a simpler protocol.

[nabokihms]

Thinking about implementation, I wonder if we can use Hashicorp's go-plugin. It looks reliable and promising.

[the-maldridge]

Chipping in my 2c: I use go-plugin in some authentication software to do almost exactly what's being described here and it works well. Best practices dictate that you should sign your plugins though, which turns out to be annoyingly involved.

[xyslion]

How about wasm?

[candlerb]

How about wasm?

It depends on how much capability you want to give to plug-ins.

If you only want them to transform a request - without any side effects and without reading from the filesystem or communicating to external databases or such - then something like JSONnet would be fine.

If you want to write general-purpose scripts which are interpreted inside the Dex runtime, then something like LUA, an embedded Javascript engine or even wasm would do (see Greenspan's Tenth Rule). Speed will be limited by what that interpreter can do, and some things like crypto may be awkward and/or inefficient in an interpreted language.

If you want to extend Dex using compiled extensions written in Go or C, then you need to link to that code or call out to it via RPC.

[braunsonm]

I think the connectors fall into three groups:

1. Those which provide variations on standard OIDC (e.g. google, gitlab, github, microsoft, linkedin etc)
2. Those which integrate with foreign identity protocols (saml)
3. Those which act as a password checker, where Dex itself prompts for username and password (e.g. ldap, mock, and passwordDB - although the latter isn't actually a connector at the moment)

For me, I would like to extend Dex to include custom claims which I add to the token before it is returned during the auth code exchange. Doesn't quite fit in this model

[Eskpil]

I don't mean to be that guy who asks "any progress here"? But still I am doing it. This is something both my personal projects and projects at work would benefit from, Dex already provides the oauth/openid flow we want but we don't wanna do that our self, we much rather would like too expose a grpc or a http api. With this api we can then our own connector instead of our own oauth2/openid implementation. We are aware that this is not exactly what Dex is supposed to do, but we so no reason why it should not be able too.

I know my way around Go, so I would be open to personally make headlines on this but it seems we still need to decide on how it would work. There have already been discussed really good options how to do this, and I think something like expanding the gRPC api is a good idea.

[nabokihms]

@Eskpil, we are at the point of writing a design doc for the feature. I even tried to make a POC, but we still need to settle some things.

In my mind, the most complicated part is about exposing the callback connector's interface. Implementing password connector's customization seems straightforward yet should not make adding support for custom callback connectors more difficult in the future.

[Eskpil]

@nabokihms Thats awesome too hear, make sure too keep me updated if there is anything I can do! 😄

[ioneyed]

Another way to do plugins is to enable an interface that developers can code towards then have a mechanism to "register" the plugin in the init command. Micro (a go framework) leverages this pattern if you want to see an example

The RabbitMQ plugin has this init function

func init(){
cmd.DefaultBrokers["rabbitmq"] = NewBroker`
}

A plugins.go file is created and compile alongside main.go which pulls in the plugins.

package main

import (
	_ "github.com/go-micro/plugins/v4/broker/rabbitmq"
	_ "github.com/go-micro/plugins/v4/registry/kubernetes"
	_ "github.com/go-micro/plugins/v4/transport/nats"
)

The build command is now go build -o dex ./main.go ./plugins.go

Like the reddit post showed, go-plugins are so-so on capabilities due to the requirements but further is there a problem with go plugins because they add another go runtime to your runtime and tools like Dynatrace (used in many enterprises) blows up and won't instrument the binary because of go plugins.

ref: https://github.com/go-micro/plugins/tree/main/v4

[candlerb]

I'd still say Hashicorp's go-plugin is a better approach all round.

I also recently came across this wasm-based version (but have not tested it): https://github.com/knqyf263/go-plugin

But Hashicorp's is long-standing and likely to be extremely robust and battle-tested.

[ioneyed]

I read a bit on the go-plugin, hoping for you to shed some more light before a deeper rabbit hole.

Does the plugin-host spin up (effectively) multiple gRPC servers (one per plugin) as a sub-process to itself so it can do a go-routine style IPC (inter process communication) model?

Does it use goroutines or is it sub-processes that could be ran entirely on their own (with or without the host)?

I ask these questions because there is a bit of a movement to more controlled environments where sub-processing becomes very limited for security purposes (like using a unikernel such as NanoVMs - https://nanovms.com/security). The sub-processing of goroutines is still viable because it is just a thread to the main process but in a unikernel world there isn't the idea of multiple
individual processes running (unless its a thread bound to the parent process)

This would be a consideration for the plugin system as our industry should be diving deeper into the likes of Unikernels for security based applications to reduce the threat vectors and likelihood of compromise in a delicate system

[candlerb]

Hashicorp's go-plugin? Yes, it spins up a separate process and talks to it over gRPC or net/rpc. Plugins can be verified with an expected checksum and RPC communications can be configured to use TLS.

https://github.com/hashicorp/go-plugin#architecture

[sagikazarmark]

@jkl0898 this issue is about moving connectors out of tree. Please use the discussion forum if you have a question.

I believe in this case you can answer the documentation: https://dexidp.io/docs/connectors/oidc/

[jkl0898]

@jkl0898 this issue is about moving connectors out of tree. Please use the discussion forum if you have a question.

I believe in this case you can answer the documentation: https://dexidp.io/docs/connectors/oidc/

Thanks!