Middleware structuring to avoid encoding error

[thomasklemm]

Hey everyone,

I'm seeing an issue with swiftype-rb, a Ruby wrapper for the Swiftype hosted search engine built on faraday. On digging through the stack trace however and seeing #161 and #69, I assume though that the reason for the exception might have it's roots in the Faraday setup of swiftype-rb.

Here's a screenshot of the last stacktrack (done with better_errors).

After some experiments I can tell that having one of these characters ( ) , ? in the string will raise the shown exception, while an included . will do no harm.

After reading the mentioned issues I think the reason might lie somewhere in the middleware structuring. It would be nice if someone could have a look at this Connection class of swiftype-rb, especially at the used middleware.

module Swiftype
  module Connection
    include Swiftype::Request

    def connection
      raise(InvalidCredentials, "You must supply credentials to Swiftype.configure") unless (Swiftype.username && Swiftype.password ) || Swiftype.api_key

      @connection ||= begin
        conn = Faraday.new(Swiftype.endpoint) do |b|
          b.response :raise_error
          b.use Faraday::Request::UrlEncoded
          b.use FaradayMiddleware::ParseJson
          b.use FaradayMiddleware::Mashify
          b.use ApiResponseMiddleware
          b.adapter Faraday.default_adapter
        end

        conn.basic_auth Swiftype.username, Swiftype.password if Swiftype.username && Swiftype.password
        conn.headers['User-Agent'] = Swiftype.user_agent

        conn
      end
    end

    class ApiResponseMiddleware < Faraday::Response::Middleware
      def on_complete(env)
        case env[:status]
        when 200, 201, 204
          nil
        when 401
          raise InvalidCredentials
        when 404
          raise NonExistentRecord
        when 409
          raise RecordAlreadyExists
        else
          raise UnexpectedHTTPException, env[:body]
        end
      end

      def initialize(app)
        super
        @parser = nil
      end
    end
  end
end

Could the error be a result of the middleware structure, or would you recommend digging somewhere else?

Thanks for any feedback in advance!

Best regards,
Thomas

[technoweenie]

Faraday's builder uses a stack. Requests start at the top and work their way down. You typically put stuff that modifies the request first (anything that tweaks the headers, prepares the body, etc). Then the response middleware works with the adapter's response before Faraday returns it to your application.

Try re-arranging the stack to something like this:

conn = Faraday.new(Swiftype.endpoint) do |b|
  #b.use Faraday::Request::UrlEncoded
  b.request :url_encoded

  b.adapter Faraday.default_adapter

  # raise early...
  b.response :raise_error

  # then parse the body json to a hash...
  b.use FaradayMiddleware::ParseJson

  # then turn the hash into a mashify object
  b.use FaradayMiddleware::Mashify

  # not sure what this is, but I'm assuming it expects a mashify object
  b.use ApiResponseMiddleware
end

Make sense?

This has turned out to be a confusing aspect of Faraday, so we're likely changing it for Faraday v1.0 so you can add the middleware in any order.

[thomasklemm]

Thanks a lot, I'll give it a try!