Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Client-618] max connections enforced #114

Open
wants to merge 6 commits into
base: stage
Choose a base branch
from

Conversation

vmsachin
Copy link
Contributor

@vmsachin vmsachin commented Mar 3, 2023

This PR contains the following code change:

  1. Added a logic that enforces a hard-limit on max connections that can be spawned.
  2. Added an exception for MAX_CONNS

@vmsachin vmsachin requested a review from khaf March 3, 2023 22:41
Copy link
Collaborator

@khaf khaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, logging rescues should raise the exception afterwards.
The logic for max_connections_exceeded is faulty and will lead to an invalid state for the client in which it won't be able to create new connections.

lib/aerospike/cluster.rb Show resolved Hide resolved
lib/aerospike/cluster.rb Show resolved Hide resolved
lib/aerospike/task/index_task.rb Show resolved Hide resolved
@@ -487,6 +487,9 @@ def execute
@node = get_node
@conn = @node.get_connection(@policy.timeout)
rescue => e
if e.is_a?(Aerospike::Exceptions::MaxConnectionsExceeded)
Aerospike.logger.error("Maximum connections established. No new connection can be created. #{e}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue. Logging without re-raising the exception.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not addressed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have not re-raised exception here as there is a retry loop here

lib/aerospike/task/execute_task.rb Show resolved Hide resolved
lib/aerospike/task/udf_register_task.rb Show resolved Hide resolved
lib/aerospike/task/udf_remove_task.rb Show resolved Hide resolved
@@ -17,19 +17,27 @@
module Aerospike
class ConnectionPool < Pool

attr_accessor :cluster, :host
attr_accessor :cluster, :host, :number_of_creations
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I Believe we can use a better name. :total_conns, etc.

else
conn = cluster.create_connection(host)
if conn.connected?
@number_of_creations += 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this number decreased? You will not create new connections after max_size connections are disconnected and new connections are needed.

@@ -49,6 +51,17 @@
end
end

context "enforce max connections as a hard limit" do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is good for the expected behavior, but does not consider what happens when a few connections are closed. Will the client allow new connections to be created afterwards, or will be stuck at max_conns_exceeded forever? (Which from the code, I think it does)

@vmsachin vmsachin requested a review from khaf March 22, 2023 13:24
@@ -1,4 +1,4 @@
# encoding: utf-8
module Aerospike
VERSION = "2.26.0"
VERSION = "2.26.1"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be 2.27.0. Too many deep changes have been applied.

conn.write(@data_buffer, @data_offset)
conn.read(@data_buffer, HEADER_SIZE)
node.put_connection(conn)
rescue => e
conn.close if conn
node.close_connection(conn) if conn
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The better design should be adding a node attribute to the connection, adding the clean up code there if the node is set (node should be set after a successful connection is established to the database node), and then adding a self.finalize to the Connection class and setting it up in the initializer. That way, even a user takes a connection from the pool and forgets to put it back, the code will still work.

@@ -487,6 +487,9 @@ def execute
@node = get_node
@conn = @node.get_connection(@policy.timeout)
rescue => e
if e.is_a?(Aerospike::Exceptions::MaxConnectionsExceeded)
Aerospike.logger.error("Maximum connections established. No new connection can be created. #{e}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not addressed.

@@ -224,14 +224,18 @@ def refresh_reset
Node::Refresh::Reset.(self)
end

def close_connection(conn)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed if the suggested changes are applied.

end

# Destroys the connection pool and closes all the connections.
def self.finalize(id)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method should be set during initialization to be called on GC:
ObjectSpace.define_finalizer(self, self.class.method(:finalize))

Aerospike.logger.error("Error occurred while closing a connection")
raise e
end
@total_connections -= 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is buggy. I'll leave it as an exercise for you to find out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants