Heartbeat add NTLM to basic URL up/down testing

[mgevans-5]

This request is also described here: Beats Issue 5237 - Heartbeat support for NTLM auth

We have many internal Corp URLs that are single sign-on with Windows Authentication / NTLM. This is a pretty widespread use case for shops doing observability for internal applications. A good example is SharePoint but we have many apps that use NTLM auth.

thanks

[elasticmachine]

Pinging @elastic/uptime (Team:Uptime)

[paulb-elastic]

An alternative solution would be to implement this with the new Synthetics offering, where you have full JavaScript capabilities that you use can build the NTLM auth into the script yourself. You would then also have a full browser and user journey capabilities to test more than just the authentication, but continue interactions in the browser beyond the authentication.

[mgevans-5]

Thanks- I'm familiar with the full Synthetics component. However that is more of a scripting option not applicable to all of our distributed teams. The use case for Heartbeat/Uptime includes the simple 'mini' synthetics of availability - in many cases separate from performance. Think important internal applications across an org. This is where NTLM is most prevelant.

[andrewvc]

@mgevans-5 I'm wondering if there's a way you could generate simple synthetics scripts to do the NTLM auth in javascript?

The reason I ask is that this is honestly the first ask we've gotten around NTLM in years of the project. Given the niche nature of the request, it's not something likely to make it onto our roadmap short of significant additional feedback from others.

That said, we'd gladly accept a patch adding NTLM auth if you're interested in adding one, especially given that go doesn't support NTLM natively. If it were a simple matter of enabling an option that'd be one thing, but it looks like we'd have to customize our round tripper, write complex tests for various failure modes etc.

[mgevans-5]

Hi There Andrew. Thank you for the response.
You'll note that in the opening issue request i linked to another issue that requested NTLM in 2017.
both @brandonmensing and @gurumaia engaged in the request. The latter actually proposed a change that he couldn't quite get working.

I will reiterate that as Elastic moves further into Observability within enterprise walls you'll see more requests for basic URLs that are authenticated with windows logins via NTLM. I would think from a product standpoint you may want to incorporate this basic request. I would consider this an out-of-the-box feature from competing products.

I understand the challenge with writing new components - I do think it would be worth the time. The difference between a no-code heartbeat and a synthetic script is the world of difference in implementation time and costs to the operations folks running an observability platform.

[andrewvc]

@mgevans-5 thanks for the additional color. @paulb-elastic @drewpost curious as to your thoughts re: prioritization here?

[paulb-elastic]

Right now we’re currently focusing on the features needed to move Synthetics to beta, but will keep this on the backlog and review again early next year.

[gurumaia]

Just saw this and would like to say that this is indeed an useful feature for the enterprise folk.

@mgevans-5 Back in 2018 I ended up using a reverse proxy implemented in Python, that would perform the NTLM authentication. It's not pretty, but it works. Here's the code: https://github.com/enkelbr/ntlm-proxy and here's the docker image it that's your thing: https://hub.docker.com/r/enkelbr/ntlm-proxy

[cdavid15]

Just to chime in here but expanding on the heartbeat http auth support for enterprise logins such as NTLM and Kerberos would be a good and welcome addition.

I am currently trying to figure out how to add monitors to http services which are protected by Kerberos and NTLM and my current thinking is we are likely to need to send the requests via a proxy which is far from ideal.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[mgevans-5]

@drewpost is this something for the Synthetics environment?

[andrewvc]

@mgevans-5 unclear what you mean exactly, could you clarify?

This is still a very low priority item, we don't get many asks for it. I think the most likely way forward here would be scriptable API Journeys: elastic/synthetics#137

These aren't on our immediate roadmap, but are something we'd like to tackle in the med-long term

[mgevans-5]

@andrewvc Thanks for checking in.
The comment on the synthetics is to see if NTLM authentication would be something that synthetics (via playwright) - instead of Heartbeat - could accomplish. It may not be so much about the script, but perhaps about what context the script is executed in.
We have a lot of SSO and direct auto-login URLs that rely on Active Directory authentication so testing them as a user would require mimicking or utilizing AD credentials.
I would imagine this to be the case across most intra-corporate environments, as mentioned above.

[mgevans-5]

Hi Folks,
Now that we're beginning to deploy heartbeat across our Org this is becoming quite the topic. We have teams that do not have access to folks to build and maintain full-synthethics (playwright) but can set up simple heartbeats. Do we have any movement on adding NTLM support? (this was out of the box in our 'very old' solution prior to moving to Elastic)

[andrewvc]

Unfortunately it's not a focus for us at the moment, and probably won't be for a while. We'd gladly accept a patch here however! Our current focus is more on revamping our UI and building out our hosted service.

[fludo]

+1 Same interest in our Org.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[mgevans-5]

:)