Add functionality: log messages to a file ?

[0x003e]

Hi @lpereira ,

Do you think create a "log file" functonality (release and debug) is a good idea ? to write on a file (specified in .conf) all messages of this flowing functions:

• lwan_status_info
• lwan_status_warning
• lwan_status_error
• lwan_status_perror
• lwan_status_critical
• lwan_status_critical_perror
• lwan_status_debug

I can implement it with a pull request.

[0x003e]

To that I need to remove each

#ifdef NDEBUG

in the source code.

[lpereira]

Logging is indeed important, but it has to be done carefully as to not impact performance that much. I've been postponing this since the project started since I haven't found yet a way I'm comfortable with.

Do you have anything in mind?

[0x003e]

I began implement this functionality:

Log sample file:

Errors handling:

This functionality can be deactivate if neither file are specified in conf file

However, it remains for me to do some benchmark to evaluate performance impact and optimize syscall when the buffer are flushed...

[lpereira]

Hm, I'm not sure if I like Lwan writing directly to a file. I've been more keen to implement logging using the same approach as Varnish rather than files.

[0x003e]

You are right:

GET 100.html 5M request 1K connexions:

• with log disabled ~312 000 request/s
• with log enabled ~100 000 request/s
• redirect stdout to a file ~110 000 request/s

conclusion: log file it's not the best way.

I've been more keen to implement logging using the same approach as Varnish rather than files.

Like this ?: https://github.com/varnishcache/varnish-cache/blob/a50c99f6b3883d1a58cedfe26511bfc0d30d50bb/bin/varnishtest/vtc_log.c

[lpereira]

Like this: https://www.varnish-cache.org/docs/trunk/reference/vsm.html

[0x003e]

Thank's
However, for the moment I not completely understand how Varnish export the log message stored in shared memory: https://github.com/varnishcache/varnish-cache/blob/a50c99f6b3883d1a58cedfe26511bfc0d30d50bb/lib/libvarnishapi/vsm.c#L215

I fond this sample: https://github.com/rohitsinha54/shared-memory-producer-consumer/blob/master/prodcon.c.

What do you think of this @lpereira ?:
lwan server is a producer and a another process is a consumer.
The another process that will wait the log message send by lwan server in a shared memory, and write the log message to a file ?

[lpereira]

That's the idea, @0x003e. In fact, if we could get the SHM format and protocol the same as Varnish's, then all the tools developed for Varnish can be used as well with little to no modification.

[mxh69420]

I made a change where it logs 4xx and 5xx as warnings, and leaves everything else fast and unlogged. I like it because I can now see if there's problems with my site, like if I'm missing files, or if one of my custom handlers are failing, or a hacker is trying to probe my server.

I haven't benchmarked this change at all yet, but it should still run just as fast for 2xx and 3xx.
I'm willing to make this an actual config option if anyone wants this ability too.

This project is amazing, I really like being able to extend it like this because the code is that simple.

[lpereira]

Good to know it's working well for you, @samiam308!