WIP: App specific transformations #4683
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Build FAILURE |
bazsi
force-pushed
the
app-specific-transformations
branch
from
7f9039d
to
76e9e64
Compare
|
Build FAILURE |
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
The entire app-parser() mechanism is undocumented as of now and I've looked at the users (our own scl and SC4S), neither is using it. I think this is a bad idea and just complicates the code anyway, so let's get rid of it. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
In preparation for adding a new root-level object type, extract the abstract base object from Application. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
…t present If the timestamp is not actually there in the message, we still tried to call convert_and_normalize_wall_clock_time_to_unix_time_with_tz_hint(), which in turn calls mktime(), which is a heavyweight operation. Let's not do that. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Without this patch, cached_mktime() produces a cached value for every second, and would call mktime() again as the second changes. This can be problematic when timestamps are parsed from different seconds in succession (e.g. when the syslog timestamp is 1 second away from a timestamp being parsed using strptime or date-parser). In these cases the cache was invalidated and mktime() was called again. mktime() is _slow_ as it even calls tzset() and validates that /etc/localtime is still pointing to the same timezone. This patch changes the caching strategy: - instead of specific seconds we calculate the cached value for the top of the hour (e.g. minutes==seconds==0) - timezones never change within the same hour, so as long as we are trying to use the cached value, we will do so as long as the year/month/day/hour value matches (plus isdst) - with this, mktime() will be called once every hour. If there's some fluctuation between subsequent timestamps at the turn of the hour, we can still exhibit limited caching (e.g. subsequent timestamps from the last hour and this one), but this is a lot better than the existing behaviour where we do that for every second. Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
bazsi
force-pushed
the
app-specific-transformations
branch
from
76e9e64
to
1b76a4b
Compare
|
Build FAILURE |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements app-transform() parser that turns an app's messages conform to a destination's requirements.
Example config: