A dev-friendly approach to handle background jobs in Magento 2

Our Sponsors

Overview 💭

Now and then we need to create processes that can take some time to execute, and that doesn't necessarily need to be done in real time. Like (but not limited to) third-party integrations.

For example, let's say you need to reflect product changes made by the storekeeper through the admin panel to their PIM/ERP. You can observe the catalog_product_save_after event and push the changes, but this would make the "Save" admin action become a hostage of the third-party system response time, potentially making the store admin reeealy slow.

But fear not citizens, because we are here!

Install 🔧

This module is compatible with both Magento 2.3 and 2.4, from PHP 7.3 to 8.1.

composer require discorgento/module-queue:^3 && bin/magento setup:upgrade

Usage ⚙️

💡 Tip: for 2.x version please refer to the old docs here. Just remember: the current version is 100% retrocompatible, so you can upgrade and use all the new features without breaking your existant code!

It's really simple, there's just two steps needed:

• append a job to the queue;
• create the job class itself (similar to Laravel).

Let's go back to the product sync example. You can now write the catalog_product_save_after observer like this:

<?php declare(strict_types=1);

namespace YourCompany\YourModule\Observer;

use Discorgento\Queue\Api\QueueManagementInterface;
use Magento\Framework\Event;

class ProductSaveAfter implements Event\ObserverInterface
{
    private QueueManagementInterface $queueManagement;

    public function __construct(
        QueueManagementInterface $queueManagement
    ) {
        $this->queueManagement = $queueManagement;
    }

    /** @inheritDoc */
    public function execute(Event\Observer $observer) {
        // append a job to the queue so it will run in background
        $this->queueManagement->append(
            // your job class, we'll create it later
            \YourCompany\YourModule\Job\SyncProduct::class,
            // a identifier of the entity we'll be working with
            $observer->getProduct()->getId(),
            // (optional) additional data for later usage
            ['foo' => $observer->getFoo()]
        );
    }
}

Now create the job itself, let's say app/code/YourCompany/YourModule/Job/SyncProduct.php:

<?php declare(strict_types=1);

namespace YourCompany\YourModule\Job;

use Discorgento\Queue\Api\JobInterface;
use Magento\Catalog\Api\ProductRepositoryInterface;
use YourCompany\YourModule\Gateway\ProductSyncer;

// the job MUST implement the JobInterface
class SyncProduct implements JobInterface
{
    private ProductRepositoryInterface $productRepository;
    private ProductSyncer $productSynchronizer;

    public function __construct(
        ProductRepositoryInterface $productRepository,
        ProductSyncer $productSynchronizer
    ) {
        $this->productRepository = $productRepository;
        $this->productSynchronizer = $productSynchronizer;
    }

    /** @inheritDoc */
    public function execute($target, $additionalData)
    {
        // retrieve the target product
        $product = $this->productRepository->getById($target);

        // optional additional data usage example
        $product->setFoo($additionalData['foo'] ?? null);

        // sync it to a third-party PIM/ERP
        $response = $this->productSynchronizer->sync($product);

        // NEW!! Now you can optionally return a string as the job "result".
        // This will be shown at admin in "System->(Tools) Queue Management"
        return "Synced. ID on PIM: {$response->pim_id}";
    }
}

And.. that's it! In the next cron iteration (which should be within five minutes) your job will be executed without compromising the performance of the store, assuring a smooth workflow for both your clients and their customers.

💡 Tip: any async process can benefit from this approach, your creativity is the limit.

Managing the queue 🆕

You can track the queued jobs status and their respective output with our brand new Queue Management grid, accessible through the "System->(Tools) Queue Management" menu (near to the native cache/index management entries):

💡 Tip: for more info about all the actions available in this grid please refer to its documentation.

Advanced features 🤖

Although this module was ported to Magento 2 due to its simplicity, over the time it also got some really neat tricks! So if want to do more with it, don't forget to check the official wiki.

Notes 🗒

• Magento can do this natively through Message Queues, but those are ridiculously verbose to use;
• issues and PRs are welcome in this repo;
• we want YOU for our community!