Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling connections with websocket client #222

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

Enabling connections with websocket client #222

wants to merge 27 commits into from

Conversation

kurotaky
Copy link
Collaborator

@kurotaky kurotaky commented Mar 8, 2023

Enabling connections via Websocket. Connect using only Ruby libraries.
fix: #86

Sample procedure

Eth::Client::Ws.new

[1] pry(main)> cli = Eth::Client::Ws.new('wss://eth-mainnet.g.alchemy.com/v2/demo')

Output

[2] pry(main)> -- websocket open ()

cli.send

[3] pry(main)> cli.send({"jsonrpc":"2.0","id": 2, "method": "eth_subscribe", "params": ["alchemy_pendingTransactions", {"toAddress": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", "0xdAC17F958D2ee523a2206206994597C13D831ec7"], "hashesOnly": false}]})
=> 223

Output

>> {"id":2,"result":"0x9b53d665c6f2fce805acfc4f8d5fc1ee","jsonrpc":"2.0"}
>> {"jsonrpc":"2.0","method":"eth_subscription","params":{"result":{"blockHash":null,"blockNumber":null,"from":"0xc313b5805c555a80f1d99775ad28c8c951eaeb72","gas":"0x12820","gasPrice":"0x153005ce00","hash":"0x283f3544322cdf4f430891c058f4ca3b931792f9790b4cf01bb87b73c779457a","input":"0xa9059cbb0000000000000000000000005655a6b14aa43de6c9ab3bf28f9a5ec55f34e8db0000000000000000000000000000000000000000000000000000000035a4e900","nonce":"0x1c","to":"0xdac17f958d2ee523a2206206994597c13d831ec7","transactionIndex":null,"value":"0x0","type":"0x0","chainId":"0x1","v":"0x26","r":"0xc46065d6e4fc0cdd5cab47c2b7318b0de955a83897e316f9e63a296b11c822dd","s":"0x740b4223e5aceb6b642ae3917d9ee5b29b2338cdae0d3878d1c264435e742d49"},"subscription":"0x9b53d665c6f2fce805acfc4f8d5fc1ee"}}
>> {"jsonrpc":"2.0","method":"eth_subscription","params":{"result":{"blockHash":null,"blockNumber":null,"from":"0x4e5b2e1dc63f6b91cb6cd759936495434c7e972f","gas":"0x15f90","gasPrice":"0x153005ce00","hash":"0x630a566d6a81804647f610885930f682a3c5bf5090ea57704bcc3ec66832d7bb","input":"0xa9059cbb0000000000000000000000004591d13a4e8253b50f93cd4c4d714fb5ca63f97b000000000000000000000000000000000000000000000000000000001ac272d8","nonce":"0x50eb7","to":"0xdac17f958d2ee523a2206206994597c13d831ec7","transactionIndex":null,"value":"0x0","type":"0x0","chainId":"0x1","v":"0x25","r":"0x60a6de68d3c4d08e35f465d526f5b9dd38c9e3f7a6d4fef782d65cf87648ba6a","s":"0x1f16a504ba55520b1ae41d8ad4cf238390a3d56a98963bfab8350cd794bd5f6"},"subscription":"0x9b53d665c6f2fce805acfc4f8d5fc1ee"}}
>> {"jsonrpc":"2.0","method":"eth_subscription","params":{"result":{"blockHash":null,"blockNumber":null,"from":"0xe8c6384ffa45274060707d483d72475cc6ffe760","gas":"0x10273","gasPrice":"0x12a05f2000","hash":"0x8a5675b84caea680c77f3a7e299950f1de451d59b4f783e66781b5861bfec7f4","input":"0xa9059cbb000000000000000000000000318b580fbb592903e5524234d27a05e885402d2a0000000000000000000000000000000000000000000000000000000165a0bc00","nonce":"0x2","to":"0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48","transactionIndex":null,"value":"0x0","type":"0x0","chainId":"0x1","v":"0x25","r":"0x616177e0c045c1588797cb85751382284c2e5b95b88e9811cda910c67fee59f6","s":"0x5a11d899ea2fe81a8749f65b3f043224cc22afeb03acb309bd7a85ef6106fdb9"},"subscription":"0x9b53d665c6f2fce805acfc4f8d5fc1ee"}}
>> {"jsonrpc":"2.0","method":"eth_subscription","params":{"result":{"blockHash":null,"blockNumber":null,"from":"0xcd0b8b0d3f79aa3206010b8106984a692e283820","gas":"0xf6e9","gasPrice":"0x143534c1ea","hash":"0x6a6a74d577df8e12a7fa61563859267e815c380ee5bc78b5ba140c562fb27aa5","input":"0xa9059cbb00000000000000000000000075a12da3374fdc56bc9cb247cf07e78d0eb532bc0000000000000000000000000000000000000000000000000000000009a7ec80","nonce":"0x4","to":"0xdac17f958d2ee523a2206206994597c13d831ec7","transactionIndex":null,"value":"0x0","type":"0x0","chainId":"0x1","v":"0x25","r":"0x53cab7a052afdf8a577ed666ca91eccabf4b272242b5844d6b370e2f881f0950","s":"0x7017b4115855f5dae65fe4dcd19b4bd01b93d5ed2fd226ede4e5623760948570"},"subscription":"0x9b53d665c6f2fce805acfc4f8d5fc1ee"}}
>> {"jsonrpc":"2.0","method":"eth_subscription","params":{"result":{"blockHash":null,"blockNumber":null,"from":"0x4097612be9032d240a94975d9a1a0611b815fe97","gas":"0x10e2b","gasPrice":"0x1a2897b01b","maxFeePerGas":"0x1a2897b01b","maxPriorityFeePerGas":"0x5f5e100","hash":"0xc252cd295969f21decad928dcb16896f644f5045da210770afc2c95030a102b6","input":"0xa9059cbb0000000000000000000000004897cb3c25e6a8f5f2fdf776aa82a572414c9f0c0000000000000000000000000000000000000000000000000000000013ab6680","nonce":"0x24","to":"0xdac17f958d2ee523a2206206994597c13d831ec7","transactionIndex":null,"value":"0x0","type":"0x2","accessList":[],"chainId":"0x1","v":"0x1","r":"0x7e25854f55816965a5eb2cc391e0cd98523bcc383e925208abebbe483976f990","s":"0x5ec3b4f6cecef9a785d5855b92e25518ff6e6472ab1316e3f8ed83406bb4238b"},"subscription":"0x9b53d665c6f2fce805acfc4f8d5fc1ee"}}
>> {"jsonrpc":"2.0","method":"eth_subscription","params":{"result":{"blockHash":null,"blockNumber":null,"from":"0x2b155df40d10445ca6539c05f61297b2311aae86","gas":"0x1726f","gasPrice":"0x1bb1debe79","maxFeePerGas":"0x1bb1debe79","maxPriorityFeePerGas":"0x5f5e100","hash":"0xcb9d60e8bc862cfee841a9c88675e2762237fd744567f47556accc767cb59cb5","input":"0xa9059cbb000000000000000000000000100ecc25feeb9e931403c06daaf2a82708a3cf9e0000000000000000000000000000000000000000000000000000000a8df274b0","nonce":"0xf","to":"0xdac17f958d2ee523a2206206994597c13d831ec7","transactionIndex":null,"value":"0x0","type":"0x2","accessList":[],"chainId":"0x1","v":"0x0","r":"0x594db7ce08853d618e37027af83735fac48ad1a9ec064c59f24dbbf913940153","s":"0x4498b8e6d2bcb22d38325e12a4919661a5b7c727750119c800dd5bba02a021e2"},"subscription":"0x9b53d665c6f2fce805acfc4f8d5fc1ee"}}

@codecov-commenter
Copy link

codecov-commenter commented Apr 30, 2023

Codecov Report

Attention: 39 lines in your changes are missing coverage. Please review.

Comparison is base (db8becd) 99.70% compared to head (4c8d760) 98.86%.

Files Patch % Lines
spec/eth/client_spec.rb 43.13% 29 Missing ⚠️
lib/eth/client/ws.rb 68.75% 10 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #222      +/-   ##
==========================================
- Coverage   99.70%   98.86%   -0.85%     
==========================================
  Files          77       78       +1     
  Lines        4477     4562      +85     
==========================================
+ Hits         4464     4510      +46     
- Misses         13       52      +39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kurotaky kurotaky marked this pull request as ready for review May 3, 2023 14:21
@@ -425,4 +437,8 @@
expect(geth_ipc.is_valid_signature(contract, hashed, signature)).to be true
end
end

describe ".send" do
pending("using the mock websocket server")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good so far. let me know when you are ready so I can review it.

@kurotaky
Copy link
Collaborator Author

📝 This test is failing on Ubuntu only. under investigation.
https://github.com/q9f/eth.rb/actions/runs/4974153459?pr=222

 1) Eth::Client.deploy .deploy_and_wait deploy the contract with constructor params
     Failure/Error: raise IOError, output["error"]["message"] unless output["error"].nil?

     IOError:
       invalid opcode: PUSH0
     # ./lib/eth/client.rb:482:in `send_command'
     # ./lib/eth/client.rb:393:in `block (2 levels) in <class:Client>'
     # ./lib/eth/client.rb:440:in `send_transaction'
     # ./lib/eth/client.rb:233:in `deploy'
     # ./lib/eth/client.rb:190:in `deploy_and_wait'
     # ./spec/eth/client_spec.rb:175:in `block (3 levels) in <top (required)>'

Copy link
Owner

@q9f q9f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, left a few comments!

lib/eth/client/ws.rb Outdated Show resolved Hide resolved
lib/eth/client/ws.rb Outdated Show resolved Hide resolved
#
# @param payload [Hash] the RPC request parameters.
# @return [Integer] Number of bytes sent by this method.
def send(payload)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We recently renamed send to send_request #201

Suggested change
def send(payload)
def send_request(payload)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the info, helpful. I fixed this here: fcaf5db


@ws.on :close do |e|
puts "-- websocket close (#{e.inspect})"
exit 1
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not exit here as we are only providing a library and not an application

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exit 1 has been deleted.

@ws = WebSocket::Client::Simple.connect @host

@ws.on :message do |msg|
puts ">> #{msg.data}"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not have stray puts in the code

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

puts method has been removed.


sleep 0.001
geth_dev_ws.send(payload)
sleep 0.001
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way to avoid sleeping here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look into ways to avoid using sleeping.

@kurotaky kurotaky self-assigned this May 24, 2023
@kurotaky
Copy link
Collaborator Author

@q9f Thanks for the review. I've just taken in the fixes, please look at it again!

@q9f
Copy link
Owner

q9f commented May 26, 2023

tests fail due to shanghai (or lack thereof)

Copy link
Owner

@q9f q9f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way we can abstract the way websockets work away from the client?

I.e., I would like to be able just to call chain_id and get a result.

lg = Logger.new(STDOUT, level: Logger::FATAL)
ws = Client.create("ws://127.0.0.1:8546", logger: lg)
ws.chain_id
# TypeError: no implicit conversion of Integer into String
# from /usr/lib/ruby/3.0.0/json/common.rb:216:in `initialize'
ws.default_account
# TypeError: no implicit conversion of Integer into String
# from /usr/lib/ruby/3.0.0/json/common.rb:216:in `initialize'

Do you think this is possible?

subject(:geth_ipc) { Client.create geth_ipc_path }
subject(:geth_http) { Client.create geth_http_path }
subject(:geth_http_authed) { Client.create geth_http_authed_path }

# it expects an $INFURA_TOKEN in environment
let(:infura_api) { "https://mainnet.infura.io/v3/#{ENV["INFURA_TOKEN"]}" }
subject(:infura_mainnet) { Client.create infura_api }
let(:logger) { Logger.new(STDOUT, level: Logger::FATAL) }
subject(:geth_dev_ipc) { Client.create geth_dev_ipc_path }
subject(:geth_dev_http) { Client.create geth_dev_http_path }
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where do these two lines come from?


describe ".send" do
it "should set up the WebSocket connection" do
expect(geth_dev_ws.instance_variable_get("@ws")).to be_instance_of(WebSocket::Client::Simple::Client)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we expose @ws with a getter?

break if geth_dev_ws.instance_variable_get("@ws").open? || (Time.now - start_time > 3)
end

geth_dev_ws.send_request(payload)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while this is fine for a test, it would be good if it supports all APIs out of the box (instead of using send_request)

# Wait for the response to be received
start_time = Time.now
loop do
break if received_data || (Time.now - start_time > 3)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be good to add a TIMEOUT to the client, so we don't have to deal with this externally

expect(received_data["result"]).to start_with("0x")

contract = Eth::Contract.from_file(file: "spec/fixtures/contracts/dummy.sol")
geth_http.deploy_and_wait(contract)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
geth_http.deploy_and_wait(contract)
geth_dev_ws.deploy_and_wait(contract)

@kurotaky
Copy link
Collaborator Author

kurotaky commented May 28, 2023

Is there a way we can abstract the way websockets work away from the client?

I.e., I would like to be able just to call chain_id and get a result.

lg = Logger.new(STDOUT, level: Logger::FATAL)
ws = Client.create("ws://127.0.0.1:8546", logger: lg)
ws.chain_id
# TypeError: no implicit conversion of Integer into String
# from /usr/lib/ruby/3.0.0/json/common.rb:216:in `initialize'
ws.default_account
# TypeError: no implicit conversion of Integer into String
# from /usr/lib/ruby/3.0.0/json/common.rb:216:in `initialize'

Do you think this is possible?

Okay, as a result, it appears that the number of bytes is returned.This is because the send method returns the number of bytes sent.Unlike HTTP, WebSocket is stateful in nature, so I may need to wait for the response before parsing it. I will think about it and try to implement it. 😄

From: /Users/kurotaki/ghq/github.com/q9f/eth.rb/lib/eth/client.rb:487 Eth::Client#send_command:

    479: def send_command(command, args)
    480:   args << "latest" if ["eth_getBalance", "eth_call"].include? command
    481:   payload = {
    482:     jsonrpc: "2.0",
    483:     method: command,
    484:     params: marshal(args),
    485:     id: next_id,
    486:   }
 => 487:   binding.pry
    488:   output = JSON.parse(send_request(payload.to_json))
    489:   raise IOError, output["error"]["message"] unless output["error"].nil?
    490:   output
    491: end

[1] pry(#<Eth::Client::Ws>)> payload
=> {:jsonrpc=>"2.0", :method=>"eth_chainId", :params=>[], :id=>1}

[2] pry(#<Eth::Client::Ws>)> output = JSON.parse(send_request(payload.to_json))
TypeError: no implicit conversion of Integer into String
from /Users/kurotaki/.rbenv/versions/3.2.0/lib/ruby/3.2.0/json/common.rb:216:in `initialize'

[3] pry(#<Eth::Client::Ws>)> send_request(payload.to_json)
=> 79

@kurotaky
Copy link
Collaborator Author

I'm back! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

eth/client: add ws/wss support
3 participants