ArgumentError: invalid byte sequence in US-ASCII

[jvenezia]

Hi there !

I get encoding errors with a website.

> url = 'http://www.lemagit.fr/actualites/4500244568/SentinelOne-mise-sur-le-poste-de-travail-pour-detecter-les-attaques?asrc=EM_MDN_42264256&utm_medium=EM&utm_source=MDN&utm_campaign=20150428_SearchSecurity.fr%20:%20toute%20l%27information%20S%e9curit%e9%20-%20Avril%202015_'
> MetaInspector.new(url, retries: 0)
ArgumentError: invalid byte sequence in US-ASCII
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/utils.rb:174:in `gsub'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/utils.rb:174:in `escape'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/parameters.rb:50:in `block in encode'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/parameters.rb:59:in `call'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/parameters.rb:59:in `block in encode'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/parameters.rb:57:in `each'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/parameters.rb:57:in `encode'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/utils.rb:153:in `to_query'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/connection.rb:407:in `build_exclusive_url'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/rack_builder.rb:191:in `build_env'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/rack_builder.rb:139:in `build_response'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/connection.rb:377:in `run_request'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/faraday-0.9.1/lib/faraday/connection.rb:140:in `get'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/metainspector-4.4.1/lib/meta_inspector/request.rb:59:in `fetch'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/metainspector-4.4.1/lib/meta_inspector/request.rb:37:in `response'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/metainspector-4.4.1/lib/meta_inspector/request.rb:21:in `initialize'
... 18 levels...
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.1/lib/active_support/dependencies.rb:268:in `load'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.1/lib/active_support/dependencies.rb:268:in `block in load'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.1/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.1/lib/active_support/dependencies.rb:268:in `load'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/commands/rails.rb:6:in `call'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/command_wrapper.rb:38:in `call'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application.rb:183:in `block in serve'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application.rb:156:in `fork'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application.rb:156:in `serve'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application.rb:131:in `block in run'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application.rb:125:in `loop'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application.rb:125:in `run'
    from /Users/jeremyvenezia/.rvm/gems/ruby-2.2.2/gems/spring-1.3.4/lib/spring/application/boot.rb:18:in `<top (required)>'
    from /Users/jeremyvenezia/.rvm/rubies/ruby-2.2.2/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /Users/jeremyvenezia/.rvm/rubies/ruby-2.2.2/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'

Any idea ?

Thanks !

[jaimeiniesta]

Thanks for the report!

No idea yet but will investigate. I've seen a similar error in other projects.

[jaimeiniesta]

Faraday (the gem we use to make the requests) seems to have trouble handling this URL, I've opened this issue upstream, let's continue the conversation there:

lostisland/faraday#476

[jvenezia]

Thanks @jaimeiniesta :)

[jaimeiniesta]

You're welcome @jvenezia :)

I'd like to investigate more, but definitely it seems a problem related to Faraday (we should help them solve it!) - for example, I tried fetching this URL using HTTParty and it went OK.

[jaimeiniesta]

It keeps failing in 5.0.1, needs some love:

http://www.lemagit.fr/actualites/4500244568/SentinelOne-mise-sur-le-poste-de-travail-pour-detecter-les-attaques?asrc=EM_MDN_42264256&utm_medium=EM&utm_source=MDN&utm_campaign=20150428_SearchSecurity.fr%20:%20toute%20l%27information%20S%e9curit%e9%20-%20Avril%202015_

[jaimeiniesta]

It keeps failing in 5.4.1

[jaimeiniesta]

Still failing in 5.6.0.