We want to use the rfc3339 standard but manually change the timezone

[andreasfrischauf]

Hello everyone,

We receive a lot of Logs from Network Devices that don't have any timezone information but are sent in UTC time.
Our receiving syslog server is located in GMT+1 timezone and because of this writes all of our events as though they were in the timezone GMT+1. This results in our logs being offset by one hour. The tool which further reads the data is unable to account for this error in the logs correctly, which leads to mismatched times for our logs. I didn't find any good way to prevent this error from happening other than changing the server timezone to UTC but since it is located in GMT+1 and also has other services running on it, we don't want to risk breaking the server.

We are currently using rfc-3339 for the timestamp format since we need the microseconds, because of our log volume.

Another solution that I thought would work was that I recreate the rfc-3339 standard using templates and then statically adding the offset as +00:00 which would work, however I didn't find any way to add microseconds from the parsed "timereported" field:

template(name="testTemplate" type="list") {
property(name="timereported" dateFormat="year")
constant(value="-")
property(name="timereported" dateFormat="month")
constant(value="-")
property(name="timereported" dateFormat="day")
constant(value="T")
property(name="timereported" dateFormat="hour")
constant(value=":")
property(name="timereported" dateFormat="minute")
constant(value=":")
property(name="timereported" dateFormat="second")
constant(value=".")
# This is not working
property(name="timereported" dateFormat="microsecond")
constant(value="+00:00 ")
property(name="fromhost-ip") # can also be set to 'hostname'
constant(value=" ")
property(name="syslogtag")
property(name="msg" spifno1stsp="on")
property(name="msg" droplastlf="on")
constant(value="\n")
}

Expected behavior

We want to control the timezone manually only for rsyslog without having to change the linux server timezone.

Actual behavior

Rsyslog is not able to do this?

Steps to reproduce the behavior

Environment

rsyslogd 8.2306.0 (aka 2023.06) compiled with:
PLATFORM: x86_64-suse-linux-gnu

[rgerhards]

You can use "position.from" and "position.to" template options to obtain the subseconds - see template doc.

[davidelang]

In part this is a glibc limitation. They have internal functions that manipulate the time based on the timezones, but have so far refused to make those functions take the timezone as a parameter as opposed to an environment variable (that is not expected to change). As such, any timezone manipulations require re-implementing the timezone database rather than just using what is already in the system. several years ago I reached out to them to see if they would be willing to just expose their internal functions that already take the TZ as a parameter and hit a wall. David Lang

…

On Fri, 29 Mar 2024, andreasfrischauf wrote: Hello everyone, We receive a lot of Logs from Network Devices that don't have any timezone information but are sent in UTC time. Our receiving syslog server is located in GMT+1 timezone and because of this writes all of our events as though they were in the timezone GMT+1. This results in our logs being offset by one hour. The tool which further reads the data is unable to account for this error in the logs correctly, which leads to mismatched times for our logs. I didn't find any good way to prevent this error from happening other than changing the server timezone to UTC but since it is located in GMT+1 and also has other services running on it, we don't want to risk breaking the server. We are currently using rfc-3339 for the timestamp format since we need the microseconds, because of our log volume. Another solution that I thought would work was that I recreate the rfc-3339 standard using templates and then statically adding the offset as +00:00 which would work, however I didn't find any way to add microseconds from the parsed "timereported" field: template(name="testTemplate" type="list") { property(name="timereported" dateFormat="year") constant(value="-") property(name="timereported" dateFormat="month") constant(value="-") property(name="timereported" dateFormat="day") constant(value="T") property(name="timereported" dateFormat="hour") constant(value=":") property(name="timereported" dateFormat="minute") constant(value=":") property(name="timereported" dateFormat="second") constant(value=".") # This is not working property(name="timereported" dateFormat="microsecond") constant(value="+00:00 ") property(name="fromhost-ip") # can also be set to 'hostname' constant(value=" ") property(name="syslogtag") property(name="msg" spifno1stsp="on") property(name="msg" droplastlf="on") constant(value="\n") } ### Expected behavior We want to control the timezone manually only for rsyslog without having to change the linux server timezone. ### Actual behavior Rsyslog is not able to do this? ### Steps to reproduce the behavior ### Environment rsyslogd 8.2306.0 (aka 2023.06) compiled with: PLATFORM: x86_64-suse-linux-gnu