-
Notifications
You must be signed in to change notification settings - Fork 16
/
Changes
708 lines (423 loc) · 18.5 KB
/
Changes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
This file describes the changes made in each version of the tail_n_mail program.
Please see http://bucardo.org/tail_n_mail for full documentation.
Version 3.3.0, released February 23, 2021
* Support the log_line_prefix variables %x and %v
[Torsten Förtsch]
* Fix printf wrong format bug
[Github issue #37]
[Jon Jensen]
Version 3.2.0, released December 9, 2020
* Force dryrun mode if the config file has the string 'dryrun' inside it
[Greg Sabino Mullane]
* Add argument "pgflatten" to allow Postgres flattening for non-Postgres files
[Greg Sabino Mullane]
* Flattening improvements with LIKE and ILIKE
[Greg Sabino Mullane]
Version 3.1.0, released September 30, 2020
* Minimum Perl version is now 5.14.0
* Add new argument "show_file_counter" to allow turning off automatic numbering of items.
[Greg Sabino Mullane]
* Prevent calling pg_stat_dir on rotated file unless pg_ls_dir states it is there
[Greg Sabino Mullane]
* Query flattening improvements
Version 3.0.0, released July 25, 2020
* Major version bump!
* Minimum Perl version is now 5.8.0
* Config files are now static, and will not get rewritten. Instead,
we start using "state" files to track information that changes
each run. With this change, config files to be easily tracked in git.
[Greg Sabino Mullane]
* Show a breakdown of how many matches came from each file on multi-file matches
Controlled with the "show_file_numbers" variable (defaults to 1)
[Greg Sabino Mullane]
* Allow service file to have the "latest"
Format is: SERVICE=name LATEST
[Greg Sabino Mullane]
* Redo the 'duration' type system entirely.
[Greg Sabino Mullane]
* Improve sorting of items to use the timestamp before the line number.
[Greg Sabino Mullane]
* Add new config file option "NO_INHERIT" to prevent the inherited
files from setting things (esp. 'include' and 'exclude')
* New command line option "--start_time", in the format YYYY-MM-DD HH:MI:SS,
which ignores existing offsets and tries to do things by date only.
Requires your log files start with the same format.
[Greg Sabino Mullane]
* Allow 'all' as an option to the --skip_report_failing_host option.
[Greg Sabino Mullane]
* Throw an exception if duplicate FILE entries are loaded
[Greg Sabino Mullane]
* Minor adjustments to folding in failed connection attempts, and in deadlock errors.
[Greg Sabino Mullane]
* Better flattening of EXECUTE arguments, lines with PID arguments
[Greg Sabino Mullane]
Version 2.8.1, released April 10, 2020
* Fix usage of log_line_prefix in RC files
[Greg Sabino Mullane]
* Better folding of timestamps
[Greg Sabino Mullane]
Version 2.8.0, released March 16, 2020
* Added the LOG_LINE_PREFIX_REGEX option, to allow changing the
log_line_prefix for files on the fly. Format is regex = log_line_prefix
and the 'regex' must start with a letter. Example:
LOG_LINE_PREFIX_REGEX: warehouse = '%m %t [%p]'
[Greg Sabino Mullane]
Version 2.7.1, released March 11, 2020
* Allow find_line_number to be set from the command line. It defaults to
true, so typical use is --find_line_number=0
[Greg Sabino Mullane]
* Add TMPDIR => 1 to all tempfile() calls, use better template names.
[Greg Sabino Mullane]
Version 2.7.0, released December 12, 2019
* Add new EXCLUDE_VIA_FILE option, to narrow exclusions to specific files.
Usage is EXCLUDE_VIA_FILE: exclusion_regex ~ filename_regex
[Greg Sabino Mullane]
* Add new INCLUDE_VIA_FILE option, to narrow inclusions to specific files.
This will override generic INCLUDE entries.
Usage is INCLUDE_VIA_FILE: inclusion_regex ~ filename_regex
[Greg Sabino Mullane]
* Add 'yesfile' and 'nofile' arguments to allow quick FILE filtering from the command line
[Greg Sabino Mullane]
* Better flattening of array literals and date/time errors
[Greg Sabino Mullane]
Version 2.6.0, released November 17, 2019
* Add --connect_timeout option (defaults to 15 seconds)
[Greg Sabino Mullane]
* Add --skip_report_failing_host option, so reports are not sent if the only
problem is a failed connection to a known down host.
[Greg Sabino Mullane]
* Allow options to contain underscores or dashes
[Greg Sabino Mullane]
* More flattening improvements
[Greg Sabino Mullane]
Version 2.5.0, released August 28, 2019
* Do not show passwords that appear in the logs as "dsn was .. password="
[Greg Sabino Mullane]
* Allow log line prefix timezone to be numeric (e.g. -05:00) or character (e.g. EST)
[Github issue #13]
[Greg Sabino Mullane]
* Use Net::SMTP directly instead of Net::SMTP::SSL
[Git hub pull request #30 by KrzysztofCzajkaTURCOM]
* Better handling of case where the rotated file has been removed (e.g. replica promotion)
[Greg Sabino Mullane]
* Handle case where log_line_prefix has both %p and %c inside it
[Github issue #29]
[Greg Sabino Mullane]
* Better flattening of WAL, pg_hba, and function errors
[Greg Sabino Mullane]
Version 2.4.0, released April 22, 2019
* Support for pgbouncer logs. Just give your config file the string 'bouncer' in it to activate.
[Greg Sabino Mullane]
* Handle cases in which the log files contain null bytes.
[Greg Sabino Mullane]
* Do not attempt to failed service twice, if host is the same but file is different.
[Greg Sabino Mullane]
* Output why some files were skipped in the header for serious issues.
[Greg Sabino Mullane]
* Add flattening for non-Postgres files
[Greg Sabino Mullane]
* Allow custom flattening rules via FLATTEN: in the config file
[Greg Sabino Mullane]
* Rather than [A], [B], etc., show the service names themselves in the output if all
of the files are using service names (unless the name is the same).
Can be forced to the old behavior with --force-letter-output.
[Greg Sabino Mullane]
* Better flattening of queries
[Greg Sabino Mullane]
* Do not show any file summaries if --mailzero was triggered
[Greg Sabino Mullane]
* Fix tracking of line number position
[Greg Sabino Mullane]
* Report pg_read_file failures rather than exiting the whole program
[Greg Sabino Mullane]
* Allow pg_read_file to handle very large files smoothly
[Greg Sabino Mullane]
* Add 'hide_conflict_error_details' option to merge all "conflict with recovery" strings
[Greg Sabino Mullane]
Version 2.3.1, released July 3, 2018
* Add new "bonus include" feature to further control what lines match via INCLUDE.
After the INCLUDE string, you can tell it to only include if the prefix matches
the given string, to only match if it does NOT match the given string, and the
same for the filename. Examples:
INCLUDE: ERROR: +FILE:foo
INCLUDE: syntax error: -PREFIX:@devbox
[Github issue #25]
[Greg Sabino Mullane]
* Better flattening of queries
[Greg Sabino Mullane]
* Use the second-arg-true version of pg_stat_file for potential log rotated files
[Greg Sabino Mullane]
* Make sure all output in --html mode is escaped
[Greg Sabino Mullane]
Version 2.2.0, released May 14, 2018
* Add ---html option, to send text/html emails with <pre> for nicer alignments.
[Greg Sabino Mullane]
* Support for handling files moved by log rotation.
[Greg Sabino Mullane]
* Add new argument --skipfilebyregex to allow quick and easy skipping of downed nodes.
[Greg Sabino Mullane]
* Add new argument, --nonparsed_statement_size, to control the maximum display size
of "nonparsed" log lines (e.g. OS errors). Defaults to 400 characters.
[Greg Sabino Mullane]
* Make sure we do not duplicate include/exclude information from inherited files.
[Greg Sabino Mullane]
* Do not write LASTFILE information to the config if using service files
[Greg Sabino Mullane]
* Bunch all non-parsed lines together when possible (e.g. failing archive_command output)
[Greg Sabino Mullane]
* Fix minor bug with display of autovac entries with a single matched file
[Greg Sabino Mullane]
Version 2.1.0, released March 8, 2018
* Support grabbing log files via Postgres connections, by using service files
and the pg_read_file() function.
[Greg Sabino Mullane]
* Allow different log files to have different log_line_prefixes.
[Greg Sabino Mullane]
Version 1.32.0, released February 12, 2018
* Support Nagios-like exit statuses with new arguments --critical=# and --warning=#.
If the number of matches found is greater than equal to the number given, then
the program will exit with a 2 (for a critical match) or 1 (for a warning match).
Otherwise, it will exit with 0.
[Github issue #26]
[Joshua Tolley]
* Add support for more custom timestamps, especially for non-Postgres logs.
Add %ts in the log_line_prefix to match things like: Sun Oct 29 08:50:53.691196 2017
[Greg Sabino Mullane]
* Better handling of continuation lines after an ignored LOG: line
[Greg Sabino Mullane]
* Improve log_line_prefix parsing, especially those using %q
[Greg Sabino Mullane]
Version 1.31.3, released October 27, 2017
* When logrotate makes a file we are watching zero-size, make sure we
store the new offset, even if it is zero.
(Github issue #21)
[Greg Sabino Mullane]
Version 1.31.2, released October 25, 2017
* Turn the %a parsing in the log_line_prefix into a "non-greedy dot" regex when
possible, which fixes the problem of whitespace inside of application names.
Note: this requires some other constant string after the %a in your log_line_prefix.
(Github issues #22 and #23)
[Greg Sabino Mullane]
Version 1.31.1, released October 24, 2017
* Fix bug where LOG_LINE_PREFIX can get "quoted" on config rewrite, but the
parser was only removing 'single' quotes.
[Greg Sabino Mullane] (thanks to Don Seiler for spotting and helping debug)
Version 1.31.0, released October 24, 2017
* Better flattening of UPDATE ... SET statements
[Greg Sabino Mullane]
* Better flattening of 'invalid input syntax' queries
[Greg Sabino Mullane]
Version 1.30.0, released October 5, 2017
* Allow the "pglog" argument to be put inside the configuration files
[Greg Sabino Mullane]
Version 1.29.0, released August 10, 2017
* Add new pglog argument, syslog2, to account for syslogs that use [\d+] instead of [\d+\-\d+]
[Greg Sabino Mullane]
* Add 'statement_size' to items we rewrite in config file
(Github issue #20)
[Greg Sabino Mullane]
* Make the escaping of regex metacharacters happen earlier on
[Greg Sabino Mullane]
Version 1.28.0, released February 8, 2016
* Better flattening of "deadlock detected" and duplicate key detail strings
[Greg Sabino Mullane]
* Add new "canceled_autovac" option to summarize those noisy events
[Greg Sabino Mullane]
* Teach about %e in log_line_prefix
[Greg Sabino Mullane]
* Better whitespace handling
[Greg Sabino Mullane]
* Walk backwards 60 days max, not 30, when trying to find last restart point.
[Greg Sabino Mullane]
Version 1.27.0, released April 18, 2014 (commit 493418bc95f878cf2de2349082c5fe3abc214c5e)
* Allow dots in user and database names of log_line_prefix.
(Github issue #9) [Markus Zimmermann]
* Allow hyphens in user and database names of log_line_prefix.
(Github issue #12) [Jan Urbański]
* Add new option EXCLUDE_NON_PARSED, so non-parsed lines (e.g. notices from
the OS) can be excluded without the heavy hammer of skip_non_parsed.
(Github issue #15) [Stephen Frost]
* Attempt to allow %q in log_line_prefix
[Greg Sabino Mullane]
* More minor improvements to similar-error-code squashing
[Greg Sabino Mullane]
Version 1.26.3, released September 24, 2013
* Minor improvements to similar-error-code squashing
[Greg Sabino Mullane]
Version 1.26.2, released September 3, 2013
* Ignore duplicate config entries, as they can cause exponential
comment growth, per bug #125.
[Greg Sabino Mullane, reported by paulh@instructure.com]
Version 1.26.1, released August 29, 2012
* Allow for spaces in the database name, per bug #110
[Greg Sabino Mullane]
Version 1.26.0, released July 10, 2012
* Added the 'sqlstate' argument, to handle cases where log_error_verbosity
is set to verbose and the SQLSTATE codes show up. If enabled, codes
after FATAL and ERROR will be stripped for easier regex matching.
Version 1.25.2, released May 23, 2012
* Fix bug that prevented the --file argument from working
[Greg Sabino Mullane]
* Handle new %a log_line_prefix escape, per bug #102
[Greg Sabino Mullane]
Version 1.25.1, released May 12, 2012
* Better spacing of the FILEx entries
[Greg Sabino Mullane]
Version 1.25.0, released April 30, 2012
* Allow multiple FILE entries per configuration file!
[Greg Sabino Mullane]
Version 1.24.0, released March 19, 2012
* Allow LOG_LINE_PREFIX in config files
[Arsen Stasic]
* Handle cases in which something is in the middle of writing a
log file line, and thus it does not end with a newline.
[Greg Sabino Mullane]
Version 1.23.0, released February 1, 2012
* Add new EXCLUDE_PREFIX option
[Bryan Montgomery]
* Add mailing option to append signatures from one or more files
to the emailed reports.
[Greg Sabino Mullane, per idea from D. Joseph Creighton]
* Fix error in prefix parsing.
[Greg Sabino Mullane]
Version 1.22.1, released November 10, 2011
* Add grand totals and better displays to tempfile output.
[Greg Sabino Mullane]
Version 1.21.1, released October 26, 2011
* Show the total tempfile size per statement at top of page.
[Greg Sabino Mullane]
Version 1.21.0, released October 19, 2011
* Added tempfile mode
[Greg Sabino Mullane]
* Fixed a bug where non-existent files still reports 0 matches.
[Greg Sabino Mullane]
Version 1.20.2, released August 31, 2011
* Make the %r escape like %h
[Keith Fiske]
Version 1.20.1, released August 26, 2011
* Switch from sysread to read
[David Christensen]
Version 1.20.0, released July 21, 2011
* Allow statement_size to be changed via config file
[Greg Sabino Mullane]
Version 1.19.4, released July 18, 2011
* Make sure we unlink the temporary file when done
[Greg Sabino Mullane]
Version 1.19.3, released July 17, 2011
* Check for LOG: files with nothing but a leading timestamp
[Greg Sabino Mullane]
Version 1.19.2, released July 7, 2011
* Do not assume the host in log_line_prefix will always have a length
[Greg Sabino Mullane]
* More minor improvements to the SQL 'flattening' process
[Greg Sabino Mullane]
Version 1.19.1, released June 16, 2011
* Minor improvements to the SQL 'flattening' process
[Greg Sabino Mullane]
Version 1.19.0, released May 12, 2011
* Allow log_line_prefix to be specified in include files.
[Greg Sabino Mullane]
* Fix problem with --debug output.
[Greg Sabino Mullane]
Version 1.18.1, released April 15, 2011
* Try harder to find inherited file, using the base of the config file
[Greg Sabino Mullane]
* Allow large files to be chunked so as not to overload outgoing mail programs.
[Greg Sabino Mullane]
* Add 'skip_non_parsed' option, to ignore non-standard log lines.
[Greg Sabino Mullane]
Version 1.17.6, released March 17, 2011
* Expand tildes to ENV{HOME} for the 'file' argument.
* Make main regex .* not .+ to accommodate syslog entries better.
[Greg Sabino Mullane]
Version 1.17.5, released January 25, 2011
* Added check for MAXSIZE setting in configuration file
[Selena Deckelmann]
* Better flattening of single-quoted constants
[Greg Sabino Mullane]
* Added the 'hideflatten' argument (primarily for debugging)
[Greg Sabino Mullane]
Version 1.17.4, released December 28, 2010
* Added more tests.
[Greg Sabino Mullane]
* Major fixes for CSV mode.
[Greg Sabino Mullane]
Version 1.17.3, released December 21, 2010
* Fixed non-functioning --file argument.
[Greg Sabino Mullane]
* Better syslog matching: fixed error in prefix number detection.
[Greg Sabino Mullane]
* Do not set a default log_line_prefix for syslog.
[Greg Sabino Mullane]
Version 1.17.2, released November 12, 2010
* Better flattening: ignore empty () and count(*)
[Greg Sabino Mullane]
* Handle all OS-generated entries as a match: allows catching of things
like out of disk space errors when archiving.
[Greg Sabino Mullane]
Version 1.17.1, released November 11, 2010
* Allow FILE, MAILSUBJECT, and DURATION_LIMIT inside of inherited config files.
[Greg Sabino Mullane]
* Various improvements to syslog output handling.
[Greg Sabino Mullane]
* Add 'nolastfile' argument for rare case of changing log_filename
[Greg Sabino Mullane]
Version 1.16.7, released September 30, 2010
* Add QUERY to list of continuation key words.
[Greg Sabino Mullane]
Version 1.16.6, released September 29, 2010
* Allow null and '[unknown]' for %u and %d in log_line_prefix.
[Greg Sabino Mullane]
Version 1.16.5, released September 17, 2010
* Allow %c as a surrogate PID, cleanup the havetimestamp and havepid logic.
[Greg Sabino Mullane]
Version 1.16.4, released September 16, 2010
* Show generated pgpidre in verbose mode, other minor fixes.
[Greg Sabino Mullane]
Version 1.16.3, released August 11, 2010
* Allow for four character time zones in header.
[Greg Sabino Mullane, spotted by Johan]
* Skip cluster-wide log entries (e.g. autovacuum)
[Greg Sabino Mullane]
Version 1.16.2 of tail_n_mail, released August 5, 2010
* Strip opening whitespace so we can effectively anchor user regexes
[Greg Sabino Mullane]
Version 1.16.1 of tail_n_mail, released August 5, 2010
* Separate the collapsing of SELECT func() vs VALUES/REPLACE
* Add negative lookahead so we don't flatten IN (SELECT)
* Move all args to new system.
* Flatten common SELECT func() calls.
* Don't show uniques for durations, because they are all unique.
* Add %c and %h to log_line_prefix handling
Version 1.15.1 of tail_n_mail, released July 23, 2010
* Handle %c and %h in log_line_prefix
Version 1.15.0 of tail_n_mail, released July 23, 2010
* Remove the old pgformat system, parse a log_line_prefix variable instead.
[Greg Sabino Mullane]
* Major rewrite of the regex code. Display the prefix better.
[Greg Sabino Mullane]
* Show an example query when outputting flattened examples.
[Greg Sabino Mullane]
* Always show non-flattened version when a single item.
[Greg Sabino Mullane]
* Make POSIX a requirement
[Greg Sabino Mullane]
* Never flatten in duration mode.
[Greg Sabino Mullane]
* Show the timezone in the Date output
[Greg Sabino Mullane]
* Never offset when using a custom file
[Greg Sabino Mullane]
Version 1.14.0 of tail_n_mail, not officially released
* Change the name from tail_n_mail.pl to simply tail_n_mail
[Greg Sabino Mullane]
* Add new pgformat to support microsecond logging
[Greg Sabino Mullane]
* Add statement_size option (defaults to 1000 characters)
[Greg Sabino Mullane]
* Change the default subject line to have UNIQUE : NUMBER
[Greg Sabino Mullane]
* Make all numbers "pretty"
[Greg Sabino Mullane]
Version 1.13.0 of tail_n_mail, released June 26, 2010
* Initial public release