Webhooks throwing error when deserializing admin_graphql_api_id

[jeffse]

We started to receive webhook data including the new admin_graphql_api_id attribute today. This has a value of something akin to gid://shopify/Customer/61476333152.

The default WebhooksController is serializing the entire hash into the job request. When ActiveJob tries to deserialize the object, it's looking at these gid's and trying to deserialize them into ActiveRecord objects. Since these objects don't exist on our system, we get an ActiveJob::DeserializationError.

I think I can work around this by creating a custom WebhooksController that deletes all of the admin_graphql_api_id attributes, but I'd like to see a more long-term fix in shopify_app.

[EiNSTeiN-]

Which webhook event type is this happening for?

[edavis10]

I saw it on themes/publish:

ActiveJob::DeserializationError: Error while trying to deserialize arguments: uninitialized constant Theme

I ended up adding a fake Theme class that did nothing but if this happens in more places I'll have to be more heavy-handed with it.

[EiNSTeiN-]

Thanks. Looks like an unintended side effect caused by a change that was released this morning. I've passed this information on to the responsible team and they will look into it.

[adrianthedev]

More info and the fix here

https://stackoverflow.com/questions/50914718/shopify-api-error-while-trying-to-deserialize-arguments

[swalkinshaw]

@adrianthedev I've been investigating a fix for this and a custom locator was my first thought as well however it doesn't fix the problem for everyone.

Do you have a class defined in your application which matches up the resource of the webhook you're receiving? ie: if its a webhook from products/create, do you have a local Product class?

Another problem is here: https://github.com/rails/globalid/blob/5bfe23a8ab3ab49c9050bc3ad0668f5c01492cf6/lib/global_id/locator.rb#L17

gid.model_class gets called even if you have a custom locator and will cause an exception if that constant doesn't exist.

[adrianthedev]

yes. I have a class Product, but it's different for Order. It's namespaced. It started to fail on my end as well. Will update in a sec.

[adrianthedev]

The quickest quickfix is to have dummy (empty) models for everything you expect to receive in webhooks until this is fixed properly from shopify.

[swalkinshaw]

I'm working on a potential upstream fix to GlobalID which combined with a custom locator (that shopify_app will define) would fix this. It looks like the best solution so far.

[adrianthedev]

Sounds good!

[jeffse]

I ended up filtering out all of the Global IDs from the webhook data for now.

class ApplicationJob < ActiveJob::Base

  def deserialize(job_data)
    sanitized_data = recursive_delete_gids(job_data)
    super(sanitized_data)
  end

private

  def recursive_delete_gids(hash)
    hash.each do |key, value|
      case value
      when String
        hash.delete(key) if value.start_with? 'gid://'
      when Hash
        recursive_delete_gids(value)
      when Array
        value.each do |array_value|
          recursive_delete_gids(array_value) if array_value.is_a? Hash
        end
      end
    end
  end
end

[swalkinshaw]

There's another alternative solution which may be worthwhile regardless. Shopify webhooks take a fields param (https://help.shopify.com/api/reference/events/webhook#properties) which restricts what fields are sent in the payload.

If you only want a subset of the fields, it's a good practice to use anyway to reduce the payload size. In this case, it could be used to set a whitelist and not include admin_graphql_api_id. You can update your existing webhook subscriptions with that.

[edavis10]

This is what I'm using for the Theme one. If I had a Theme model then it'd be a much bigger issue

# Class used to prevent ActiveJob and GlobalId from erroring when
# the Shopify API sends GlobalID inside of the GraphQL API
#
# e.g. "admin_graphql_api_id":"gid://shopify/Theme/11810989"
#
class Theme
  include GlobalID::Identification

  # No-op
  def self.find(*args)
    nil
  end
end

[swalkinshaw]

I think you could combine two strategies like this:

# config/initializers/global_id.rb

GlobalID::Locator.use :shopify do |gid|
end

Theme = Class.new
# define others as needed

Then the class will exist, but locator should just return the string back.

[adrianthedev]

yes, but you still need to define all classes. Order, LineItem, Product, etc.

[talecK]

I am seeing a similar issue, woke up to thousands of failed ProductCreateJobs in resque.

We are seeing the errors in ProductsCreateJob, which is weird since we define product.rb < ActiveRecord::Base.

Any advice on how we should proceed in fixing this in the short term?