Jobs Refactor

[adam-fowler]

This refactor is required as it has been pointed out accessing services from a job is quite hard with the current structure. The changes include a breaking up of what defines a job and an actual job instance with parameters. I've also added a JobIdentifier which also includes a reference to the parameter type used by the job to ensure we don't pass bad parameters to a job.

// define email job parameters
struct EmailJobParameters: Codable {
    let recipient: String
    let subject: String
    let message: String
}

// add identifier for email job and associate with EmailJobParameters type
extension HBJobIdentifier<EmailJobParameters> {
    static var email: Self { .init(name: "email-job") }
}

// Create job queue
let jobQueue = HBJobQueue(.redis(...), maxWorkers: 4, logger: logger)

let ses = SES()

// register email job, with job functionality
jobQueue.registerJob(.email) { parameters, context in
    try await ses.sendEmail(to: parameters.recipient, subject: parameters.subject, body: parameters.message)
}

//  Push email job to queue. Because we have the association between
// the identifier and the required parameters we can ensure the correct parameter type
// is passed in
jobQueue.push(
    id: .email, 
    parameters: .init(recipient: "adam@gmail.com", subject:"Test", message: "Testing, testing, 123")
)

[codecov]

Codecov Report

Attention: Patch coverage is 88.88889% with 17 lines in your changes are missing coverage. Please review.

Project coverage is 84.46%. Comparing base (8cb3e42) to head (f130b57).
Report is 2 commits behind head on 2.x.x.

Files	Patch %	Lines
Sources/HummingbirdJobs/JobRegistry.swift	80.00%	11 Missing ⚠️
Sources/HummingbirdJobs/JobQueueHandler.swift	88.46%	3 Missing ⚠️
Sources/HummingbirdJobs/JobIdentifier.swift	75.00%	1 Missing ⚠️
Sources/HummingbirdJobs/JobQueueError.swift	0.00%	1 Missing ⚠️
...obs/Utils/DecodableWithUserInfoConfiguration.swift	90.90%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##            2.x.x     #391      +/-   ##
==========================================
+ Coverage   84.39%   84.46%   +0.06%     
==========================================
  Files          94       99       +5     
  Lines        5282     5342      +60     
==========================================
+ Hits         4458     4512      +54     
- Misses        824      830       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[adam-fowler]

For reference this is what the Jobs framework did previously with comment indicating the issue I'm trying to fix

let see = SES()

struct EmailJob: HBJob {
    let recipient: String
    let subject: String
    let message: String

    func execute(logger: Logger) async throws {
        // How do I get access to ses to send the email here?
        // I can't include it in the init as the job is recreated by queue
        // at a later date, I can't use variable capture as I would in a closure
    }
}

EmailJob.register()

// Create job queue
let jobQueue = HBRedisJobQueue()

jobQueue.push(EmailJob(recipient: "adam@gmail.com", subject:"Test", message: "Testing, testing, 123"))

[Joannis]

Is there a reason we're using Data here, instead of ByteBuffer?

[adam-fowler]

Just the fact we are moving back and forward between the two types. Should probably change to ByteBuffer.

[Joannis]

Would it be possible to allow drivers to encode these parameters (or the request as a whole)? In MongoDB, it would be really unfortunate to encode this to JSON, then add the JSON into MongoDB. Since MongoDB already has (recursive) object support. So it would be able to store this as-is in BSON. Encoding to JSON would prevent indexes from being applies to id, and inspecting the data in MongoDB would require quite a lot of work from the developer.

[adam-fowler]

The idea is that the driver only stores data blobs. Do you see an advantage to storing the full object structure? I guess debugging job issues might be easier.

[Joannis]

Debugging mainly, yes

[adam-fowler]

I had moved away from the allowing drivers to do the encoding because I was repeating the same code across all of the drivers. It also meant the driver needed to know about the job registry so they could be sure to decode the correct job type. Data blobs meant the driver had a clean split from the job decoding. I'll have another look though.