Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add elvis linter #502

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

ci: add elvis linter #502

wants to merge 3 commits into from

Conversation

hlolli
Copy link
Member

@hlolli hlolli commented Nov 22, 2023

No description provided.

@hlolli hlolli force-pushed the feature/elvis-linter branch 5 times, most recently from e68721f to 64fa2fc Compare December 8, 2023 13:49
parse_cli_args(Rest, C#config{ transaction_whitelist_files = [File | Files] });
parse_cli_args(["transaction_whitelist_url", URL | Rest],
C = #config{ transaction_whitelist_urls = URLs} ) ->
C = #config{transaction_whitelist_urls = URLs}) ->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we remove spaces here, but not here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is a good question, I believe the complaint was just about the parentheses and I did more than was needed to satisfy this lint rule

@@ -484,8 +482,7 @@ generate_block_data_segment_base(B) ->
integer_to_binary(ScheduledRateDividend),
integer_to_binary(ScheduledRateDivisor),
integer_to_binary(B#block.packing_2_5_threshold),
integer_to_binary(B#block.strict_data_split_threshold)
| Props
integer_to_binary(B#block.strict_data_split_threshold) | Props
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an elvis change? Personally I prefer to keep newlines consistent within an operator. For this operator we've made the choice to put each list element on its own line, so I would prefer to also put the | Props on its own line to be consistent. I've been bitten before by inconsistent new lines where only one element is put on the same line as the others and it gets missed when ssomeone is scannign the code because they expect all elements to be on new lines.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default rule says that there can't be space on the left side of an operator, this has the effect that we have

ValA +
ValB +
ValC.

and not

ValA
+ ValB
+ ValC

personally I'm not trained to read code like thie

{
  a: 1
  ,b: 2
  ,c: 3
 }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with

ValA +
ValB +
ValC.

or

ValA
+ ValB
+ ValC

and think in general commas should be treated differently from math operators for the reason you mention (commas generally bind to the left value, but it's less clear with binary math operators which value they more logically bind to - think of grammar school long addition, in the US we're taught to put the + at the beginning of every line)

But that said, I retract my original point on the | Elements and defer to the elvis rule

try ar_http_req:body(Req, SizeLimit) of
Term -> From ! {read_complete_body, Term}
catch
Exception:Reason -> From ! {read_complete_body, {Exception, Reason}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is a slight change in functionality. Previously Exception:Reason was sent, now {Exception, Reason} is sent. I'm not familiar with the code enough to know if it will break anything, given that I'd revert to the old behavior (unless you feel good that it won't break)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, I don't know what the old catch did when it hit, I don't have a good feeling myself, to be fair I asked chatgpt to keep the behaviour the same. But I also wouldn't have a good feeling with making the catch case a no-op. Happy for suggestions.

The receiver is in ar_http_req.erl I think

read_complete_body(Req, #{ acc := Acc, counter := C } = Opts) ->
	{MoreOrOk, Data, ReadReq} = cowboy_req:read_body(Req),
	DataSize = byte_size(Data),
	prometheus_counter:inc(
		http_server_accepted_bytes_total,
		[ar_prometheus_cowboy_labels:label_value(route, #{ req => Req })],
		DataSize
	),
	read_complete_body(MoreOrOk, Opts#{ acc := [Acc | Data],  counter := C + DataSize }, ReadReq).

read_complete_body(_, #{ counter := C, size_limit := SizeLimit }, _) when C > SizeLimit ->
	{error, body_size_too_large};
read_complete_body(more, Data, Req) ->
	read_complete_body(Req, Data);
read_complete_body(ok, #{ acc := Acc, start_time := StartTime }, Req) ->
	Body = iolist_to_binary(Acc),
	BodyReadTime = erlang:monotonic_time() - StartTime,
	{ok, Body, with_body_req_fields(Req, Body, BodyReadTime)}.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe @vird or @ldmberman could chip in here, what is returned in this loop code when an error is caught? Are we safe to just return {error, Reason} or should we return nothing?

try ar_http_req:read_body_chunk(Req, Size, Timeout) of
Term -> From ! {read_body_chunk, Term}
catch
Exception:Reason -> From ! {read_body_chunk, {Exception, Reason}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto above

* Ln2Divisor
* EXDividend
).
?GENESIS_TOKENS * ?WINSTON_PER_AR * EXDivisor * 2 * Ln2Dividend div (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -390,7 +390,7 @@ find_low_priority_txs(Iterator, {MempoolHeaderSize, MempoolDataSize})
when
MempoolHeaderSize > ?MEMPOOL_HEADER_SIZE_LIMIT;
MempoolDataSize > ?MEMPOOL_DATA_SIZE_LIMIT ->
{{_Utility, TXID, _Status} = _Element, Iterator2} = gb_sets:next(Iterator),
{{_Utility_, TXID, _Status} = _, Iterator2} = gb_sets:next(Iterator),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{_Utility_, TXID, _Status} = _, Iterator2} = gb_sets:next(Iterator),
{{_Utility_, TXID, _Status} = _Element, Iterator2} = gb_sets:next(Iterator),

_ ->
{error, not_found}
catch
_Class:_Exception ->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we lost the {error, timeout} return

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add it back, but it seems this can be any error, but I revert

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe take a look at the file now if it makes sense, I return error, timeout from the catch clause

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the issue is the formatting of returns. Previously only when we caught {'EXIT', {timeout, {gen_server, call, [ar_arql_db, _]}}} did we return {error, timeout} - now any error will trigger an {error, timeout}. I'm not sure if it will make a difference, but I'm a little wary of functionality changes in a large formatting PR.

If there aren't any callers of get_tx_confirmation_data that handle {error, timeout}, then it's fine and we can go with your earlier iteration of {error, Exception} (or whatever)

but if the callers of get_tx_confirmation_data handle {error, timeout} or handle {error, SOMETHING}, then this diff might break those callers handling logic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want any functional change, I'm aware that enforcing single case statement may be mutually exclusive with that goal. But let's look at this:

> git grep get_tx_confirmation_data                                           
apps/arweave/src/ar_header_sync.erl:250:        case ar_storage:get_tx_confirmation_data(TXID) of
apps/arweave/src/ar_http_iface_middleware.erl:1543:                                     case ar_storage:get_tx_confirmation_data(TXID) of
apps/arweave/src/ar_storage.erl:10:             read_tx/1, read_tx_data/1, update_confirmation_index/1, get_tx_confirmation_data/1,
apps/arweave/src/ar_storage.erl:301:get_tx_confirmation_data(TXID) ->
apps/arweave/test/ar_tx_blacklist_tests.erl:324:                        ?assertMatch({ok, {_, _}}, ar_storage:get_tx_confirmation_data(TXID))
apps/arweave/test/ar_tx_blacklist_tests.erl:351:                        ?assertMatch({ok, {_, _}}, ar_storage:get_tx_confirmation_data(TXID)

Excluding tests, we have callers in ar_http_iface_middleware.erl and ar_header_sync.erl

Looking at ar_http_iface_middleware.erl
it pattern matches on

{ok, {Height, BH}} ->
.... elided ...
						not_found ->
							{404, #{}, <<"Not Found.">>, Req};
						{error, not_found} ->
							{404, #{}, <<"Not Found.">>, Req};
						{error, timeout} ->
							{503, #{}, <<"ArQL unavailable.">>, Req}

and ar_header_sync.erl

which pattern matches on

		{ok, {Height, _BH}} ->
.....
		not_found ->
			ar_events:send(tx, {ready_for_unblacklisting, TXID}),
			{noreply, State};
		{error, Reason} ->
			?LOG_WARNING([{event, failed_to_read_tx_confirmation_index},
					{error, io_lib:format("~p", [Reason])}]),
			{noreply, State}

the former one only matches timeout errors, while the latter one matches any error. So any other error, Reason that isn't timeout would not fly with ar_http_iface_middleware but would be ok in ar_header_sync.

I can write up a ticket to make it consistent and allow any error reason to be sent and received, because surely timeout is only one of meany reasons.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given what the callers are doing, and what the behavior was before, it looks like what you have now is good. The only change in functionality is now it will return {error, timeout} for some scenarios where it crashed outright before. I think that change in functionality is okay. (assuming I understand the whole {'EXIT', {... match which I'm not sure i do)

@hlolli hlolli force-pushed the feature/elvis-linter branch 2 times, most recently from b2f0ee8 to ad0e12c Compare December 8, 2023 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants