Postgres Adapter

[GeorgeHahn]

Hi! This looks like a great library - thank you for publishing it.

I would like to use this library in a project that uses Postgres. I'm going to be pretty busy for the next few weeks, so it won't happen immediately, but I'd like to begin building a Postgres adapter. It looks like this should be a straightforward process, but if you have any thoughts or pieces of advice, I'd love to hear them.

Cheers,
George

[mrahhal]

Hi.

A few months ago I was going to focus on this exclusively to get the interfaces and the public API fixed and to include the major features that I'd like this to have so that a stable base can be worked from. But unfortunately I got busy.

I started this after getting upset whenever I saw how background processing was supported in frameworks like rails whereas Asp.Net was so lagging behind. We don't have an abstraction to work from and every other 3rd party solution was either more complex than necessary or not good enough for my needs.

But now, I think I'll be able to get back to this. I want to finalize the interfaces as I said to the point where I'm satisfied. After that I can focus on the adapters.

If you'd like to know, some points I want to resolve before I can really encourage custom adapters:

• The interfaces that an adapter must implement. There are the interfaces that are responsible for the real work and there are setup interfaces. I want to make sure they are appropriate in general and not just for the SQL adapter (in order to build custom adapters in a straightforward manner).
• Because the whole pipeline is asynchronous I'll need to review what I'm doing so that I'm not causing deadlocks or anything in edge cases.
• More features I absolutely want to support:
  • What happens if a job is failing always (for example due to a poor written job or even just because of a network issue)? Right now it will just retry directly and infinitely which is problematic. There should be a way to delay failed jobs until a certain retry limit is met and this behavior should be configurable (probably on a job bases).
• Move as much functionality as possible from the adapters to the core (through abstract classes, see BootstrapperBase for example).

When I started with this, I didn't really want a dashboard. After rethinking, I guess having an API that lets you query jobs and their states in the core services would be nice. That way I can build a dashboard service in a separate project, as I really want the core library to be simple.

I think I'll make a PR that finalizes the points I talked about so that you can follow the progress.
Feedback is appreciated and I'd be happy to help when you start building an adapter. Thanks!

[mrahhal]

@GeorgeHahn do you plan on using it on Linux/Mac? (so sorry, deleted your comment by mistake 😅)

[GeorgeHahn]

Oops :P

At the moment, I plan to host in azure. Flexibility for Linux would be nice, but isn't currently a requirement.

API capabilities to enable a dashboard or monitoring system would be pretty great, though it certainly isn't something I need right now.

Cheers,
George