director reference counting races

[nigoroll]

Expected Behavior

Director reference counting should avoid use-after-frees

Current Behavior

I believe there are still open races with our director reference counting

Simple director assignment

VCL: set bereq.backend = vmod.backend()

this basically translates to: VRT_Assign_Backend(&ctx->bo->director_req, vmod_function())

So what happens when the last reference goes away between the time of vmod_function() returning and VRT_Assign_Backend() referencing it?

Resolve function

VRT_DirectorResolve() does not take/release references. But even if it would, it still left a similar gap as in the previous case, the backend could go away between d->vdir->methods->resolve() looking it up and VRT_DirectorResolve() taking a reference.

(edit: VDI_Http1Pipe() and VDI_GetHdr() call VRT_DirectorResolve() via VDI_Resolve() and they do take references. So the window is larger than for the simple assignment case, but still comparably small)

Possible Solution

#2725 suggested to avoid this and similar problems by putting all to-be-deleted backends onto a cool list, which would be worked when the last request possibly referring any of the to-be-deleted backends went away.
With the current implementation, it would appear to me that any function returning a backend would need to provide the reference with it.

Steps to Reproduce (for bugs)

No response

Context

Staring at nigoroll/libvmod-dynamic#81

Varnish Cache version

No response

Operating system

No response

Source of binary packages used (if any)

No response

[nigoroll]

bugwash: we tend towards changing interfaces returning directors to always return with another reference held.
phk said he wanted to work on this.