Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache common scalar values #1875

Open
wants to merge 5 commits into
base: dev
Choose a base branch
from
Open

Conversation

sungam3r
Copy link
Contributor

@sungam3r sungam3r commented Mar 4, 2023

fixes #1777

Comment on lines -168 to +181
return new ScalarValue(value.ToString() ?? "");
var stringified = value.ToString();

if (stringified == null)
return ScalarValue.EmptyString; // ScalarValue.Null; ?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 ? Why so returning empty string? I tried to change it to return ScalarValue.Null but some tests failed. I have a suspicion that this was done by design a long time ago and now it can be dangerous to change behavior here. Honestly returning null here seems more reasonable.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this even possible? I suppose some class could override ToString method, but that seems silly. As an outsider, returning null seems more logical.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose some class could override ToString method, but that seems silly.

Not so silly. There are a lot of different situations. Nevertheless, the question is - why to swap null to empty string here? What is the purpose?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant was, it would be silly for a class to override ToString to return null, but I'm sure it's being done somewhere for a reason I am not aware of.

@nblumhardt
Copy link
Member

I just ran:

select
  sum(length(keys(@Properties))) as total,
  count(@Properties[?] = -1) as minus_1,
  count(@Properties[?] = 0) as zero,
  count(@Properties[?] = 1) as plus_1,
  count(@Properties[?] = '') as empty,
  count(@Properties[?] = null) as nul,
  count(@Properties[?] = true) as tru,
  count(@Properties[?] = false) as fals
from stream 

over our last week's raw prod log data, and got:

total minus_1 zero plus_1 empty nul tru fals
8486554 0 116 1975 0 10706 614 191

The logs are our own app logs (system is ~6 years old?) using ASP.NET Core.

Out of the 8.5M property values we've collected, shockingly, we don't have any -1 or '' values.

0, true, and false are so uncommon as to be practically absent, while the 1 case is still only around 1/4250 frequency.

Not sure how representative this is, but given the most frequent null case is only 10.7K values, perhaps we should reevaluate?

Anyone watching along keen to spin that query up over a few hours (or days) data in Seq and see whether the picture is similar?

@sungam3r
Copy link
Contributor Author

Not sure how representative this is

Yes, experience may vary.

@sungam3r
Copy link
Contributor Author

Rebased on dev. What is verdict here?

@nblumhardt
Copy link
Member

I'd be inclined to skip this one - doesn't seem compelling enough to justify including it.

@sungam3r
Copy link
Contributor Author

Some logs from our production for IsAuthenticated boolean property
изображение

As I said - experience may vary. In your case you have no enough logs with booleans, ints, etc., OK.

if (stringified == null)
return ScalarValue.EmptyString; // ScalarValue.Null; ?

if (stringified == string.Empty)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A similar check is already happening at line 138. IMO we should move the string checks to the bottom and consolidate to something like var stringified = value is string ? (string)value : value.ToString().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar but not the same. Check on line 138 serves as fast return before calling value.ToString().

@sungam3r
Copy link
Contributor Author

rebased on dev


if (value is int i)
{
if (i == 0) return ScalarValue.Zero;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use a switch expression here, that would eliminate two comparisons if i == -1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switch expression should return value for each case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cache common scalar values
3 participants