Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(pairing): port queries to xandra #834

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

noaccOS
Copy link
Contributor

@noaccOS noaccOS commented Aug 9, 2023

use xandra in place of cqex for all queries to the cluster.

feat: expose custom query API
feat: expose cluster name as an env var internally to the config
fix: the tests now wait for the cluster to be connected
fix: remove :autodiscovery key from xandra_options as it is deprecated
chore: update xandra to v0.17.0

closes #268

@codecov
Copy link

codecov bot commented Aug 9, 2023

Codecov Report

Merging #834 (eaa9900) into master (e0fd426) will increase coverage by 7.80%.
Report is 14 commits behind head on master.
The diff coverage is 79.51%.

❗ Current head eaa9900 differs from pull request most recent head 06b2a2a. Consider uploading reports for the commit 06b2a2a to get more accurate results

@@            Coverage Diff             @@
##           master     #834      +/-   ##
==========================================
+ Coverage   67.42%   75.23%   +7.80%     
==========================================
  Files         264      139     -125     
  Lines        6429     2588    -3841     
==========================================
- Hits         4335     1947    -2388     
+ Misses       2094      641    -1453     
Files Coverage Δ
apps/astarte_pairing/lib/astarte_pairing.ex 100.00% <ø> (ø)
apps/astarte_pairing/lib/astarte_pairing/config.ex 90.00% <100.00%> (+16.66%) ⬆️
...ne_api/lib/astarte_appengine_api/device/queries.ex 80.92% <0.00%> (ø)
apps/astarte_pairing/lib/astarte_pairing/engine.ex 80.76% <85.71%> (-4.48%) ⬇️
...astarte_pairing/lib/astarte_pairing/rpc/handler.ex 58.57% <25.00%> (-2.63%) ⬇️
...pps/astarte_pairing/lib/astarte_pairing/queries.ex 77.01% <82.25%> (+0.82%) ⬆️

... and 128 files with indirect coverage changes


defp use_realm(conn, realm) when is_binary(realm) do
with true <- Realm.valid_name?(realm),
{:ok, %Xandra.SetKeyspace{}} <- Xandra.execute(conn, "USE #{realm}") do
Copy link
Contributor Author

@noaccOS noaccOS Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For c42c32a I have opted for switching keyspace first and running the query second.

This is in contrast with #771 where we're currently using a query parameter and then manually replacing the string

get_pem_statement = @query_jwt_public_key_pem |> String.replace(":realm_name", realm_name)

While it is of course possible to do that here as well inside the prepare_query/3 function, I've found it overall cleaner, especially for error handling and query definition, to have the separate USE statement.

I'd like to discuss this as I think it is important to have coherent code in all our PRs

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason we did it with string interpolation is that there was an issue with how Xandra handled USE queries in older versions (see whatyouhide/xandra#180).
The PR took some time to be merged so we kept on using string interpolation, but now that it's upstreamed we should be fine using USE queries.

@noaccOS noaccOS force-pushed the pairing-xandra branch 3 times, most recently from 2e924f9 to 7399bb5 Compare August 9, 2023 14:43
@@ -65,17 +65,21 @@ defmodule Astarte.Pairing.Config do
Returns the cassandra node configuration
"""
@spec cassandra_node!() :: {String.t(), integer()}
def cassandra_node!, do: Enum.random(cqex_nodes!())
def cassandra_node!, do: Enum.random(xandra_nodes!())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function should be unused now, as it was required by CQEx but not by Xandra. It can be removed

Comment on lines 28 to 50
@typedoc """
A string representing the current context, to include in error messages.
Not included if nil.
Default: nil
"""
@type custom_context() :: String.t() | nil

@typedoc """
The type to cast the result in.
Unless otherwise stated, all results are returned in an {:ok, value} tuple.
- `:page`: default xandra return type
- `:list`: a list
- `{:first, default}`: only return the first element, `default` if empty
- `:first`: shorthand for `{:first, nil}`
- `{:first!, error}`: like `{:first, default}`, but returns `{:error, error}` if empty
- `:first!`: shorthand for `{:first!, :not_found}`
"""
@type custom_result() :: :page | :list | :first | :first! | {:first, any()} | {:first!, any()}

@type custom_opt() :: {:context, custom_context()} | {:result, custom_result()}
@type xandra_opt() :: {atom(), any()}

@type query_opt() :: custom_opt() | xandra_opt()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good effort to get to a safer typing of queries. However, I'm not sure that this would be the best place: this is something I would expect to find in a library (e.g. Astarte Data Access) rather than in an application module. For example, the custom_result() type is generic enough not to be related just to the queries in this module.
Nonetheless, this approach is interesting, so I say we should discuss it further.

@type custom_context() :: String.t() | nil

@typedoc """
The type to cast the result in.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that the type is casted is an implementation detail that could be hidden

end

defp database_error_message(%Xandra.Error{message: message, reason: reason}, nil = _context) do
%{message: "Database error #{reason}: #{message}", tag: "db_error"}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Xandra errors implement the Exception behaviour, you could use Exception.message/1 rather than manually getting into the struct and retrieving message and reason

Signed-off-by: Francesco Noacco <francesco.noacco@secomind.com>
use xandra in place of cqex for all queries to the cluster.

feat: expose cluster name as an env var internally to the config
fix: the tests now wait for the cluster to be connected
fix: remove :autodiscovery key from xandra_options as it is deprecated
chore: update xandra to v0.17.0

Signed-off-by: Francesco Noacco <francesco.noacco@secomind.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Port astarte_pairing to Xandra
3 participants