Log refactor - Table logging features #2984

arng40 · 2024-02-09T15:46:29Z

This PR is the first one of a work in collaboration with @MelReyCG that aim to standardize logs and make them more readable.
Today's logs are complicated to read:

Data is output in different formats
Alignments are random

This PR aim to help to format the logs in a clearer way by providing table drawing :

Separation of : the collect, the format and dump.
Compile time verification of value added
Dynamic table and column size
Optional title
Optional columns
Custom column lignment
Margin between columns sides

This feature will be used in a first time for the well data output. According to the well control log level, the column of previous and next element can be displayed.

An example of the refactored Well Log:

Dumping 5 Elements into Log and CSV

1. Data collection

TableData tableData;
loop (over data)
{
tableData.addRow(val1,val42val3,val4,val5);
}

2. Format the data

TableLayout tableLayout( {"Element no.",  "CordX", "CoordY", " CoordZ",  "Prev\nelement", "Next\nelement"} );

3. Dumping the data

Using TableTextFormatter

TableTextFormatter tableText(tableLayout);

or dumping to a csv file

std::ofstream os( joinPath( OutputBase::getOutputDirectory(), filename + ".csv" ) );
TableCSVFormatter csvFormat( tableLayout );
os << csvFormat.headerToString();
os << csvFormat.dataToString( tableData );

Log Output

An example of current well log, 3 elements displayed

++++++++++++++++++++++++++
InternalWellGenerator = well_injector1
MPI rank = 0

Number of well elements = 30
Well element #0
Coordinates of the element center: { 9.5, 0.2, 11.9583 }
No next well element
Previous well element #1
First well node: #0
Second well node: #1
Well element #1
Coordinates of the element center: { 9.5, 0.2, 11.875 }
Next well element # = 0
Previous well element #2
First well node: #1
Second well node: #2
Well element #2
Coordinates of the element center: { 9.5, 0.2, 11.7917 }
[...]

With the new log

--------------------------------------------------------------------------------------
|                        Well 'well_injector1' Element Table                         |
--------------------------------------------------------------------------------------
|  Element no.  |  CoordX  |  CoordY  |              CoordZ  |     Prev  |     Next  |
|               |          |          |                      |  Element  |  Element  |
--------------------------------------------------------------------------------------
|            0  |     9.5  |     0.2  |  11.958333333333334  |        1  |           |
|            1  |     9.5  |     0.2  |              11.875  |        2  |        0  |
|            2  |     9.5  |     0.2  |  11.791666666666666  |        3  |        1  |
|            3  |     9.5  |     0.2  |  11.708333333333334  |        4  |        2  |
|            4  |     9.5  |     0.2  |              11.625  |        5  |        3  |
--------------------------------------------------------------------------------------

…e-layout

MelReyCG

First wave

src/coreComponents/codingUtilities/StringUtilities.hpp

src/coreComponents/codingUtilities/Table.cpp

MelReyCG

second wave

src/coreComponents/codingUtilities/tests/testTable.cpp

src/coreComponents/codingUtilities/Table.cpp

src/coreComponents/codingUtilities/Table.hpp

src/coreComponents/codingUtilities/Table.cpp

src/coreComponents/codingUtilities/Table.hpp

…dudes/table-layout

…feature/dudes/table-layout" This reverts commit 8f74cfa.

…nto feature/dudes/table-layout

codecov · 2024-02-23T17:38:48Z

Codecov Report

Attention: Patch coverage is 83.46883% with 61 lines in your changes missing coverage. Please review.

Project coverage is 53.70%. Comparing base (79856a0) to head (5b46e14).

Files	Patch %	Lines
...reComponents/mesh/generators/WellGeneratorBase.cpp	4.00%	24 Missing ⚠️
src/coreComponents/fileIO/Table/TableFormatter.cpp	86.70%	23 Missing ⚠️
src/coreComponents/common/DataTypes.hpp	0.00%	4 Missing ⚠️
src/coreComponents/fileIO/Table/TableData.cpp	87.09%	4 Missing ⚠️
...oreComponents/fileIO/Table/unitTests/testTable.cpp	96.73%	3 Missing ⚠️
src/coreComponents/fileIO/Table/TableLayout.cpp	92.00%	2 Missing ⚠️
src/coreComponents/fileIO/Table/TableFormatter.hpp	66.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2984      +/-   ##
===========================================
+ Coverage    53.56%   53.70%   +0.14%     
===========================================
  Files         1003     1010       +7     
  Lines        85295    85637     +342     
===========================================
+ Hits         45685    45992     +307     
- Misses       39610    39645      +35

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/coreComponents/fileIO/Table/unitTests/testTable.cpp

…e-layout

MelReyCG

New batch! I hope it will be the last one.

src/coreComponents/fileIO/Table/TableFormatter.cpp

src/coreComponents/codingUtilities/TableFormatter.hpp

MelReyCG

All good to me!

MelReyCG

All good to me!

TotoGaz · 2024-05-31T17:25:30Z

src/coreComponents/fileIO/Table/TableData.cpp

-  {
-    m_errorsMsg.push_back( splitErrors );
-  }
+  m_errorsMsg.push_back( msg );


Maybe inline it now? (Or even remove it: it seems to be used once for a push_back...)

function deleted

TotoGaz · 2024-05-31T17:30:29Z

src/coreComponents/fileIO/Table/TableFormatter.hpp

+  static constexpr char m_sideCross = '+';
  /// symbol to delimit a table column
-  static constexpr string_view innerCross = "+";
+  static constexpr char m_innerCross = '+';


Where are m_sideCross and m_innerCross used?

They aren't used anymore, they are just for customisation purpose
if we want to have a table with this style +-----+, we can use them.

As there is no longer code that use these contants, you must remove it. If someone will need to change the style of the tables, he will re-write them.
Keeping them would be confusing.

OK, it's removed

…e-layout

…DEV/GEOS into feature/dudes/table-layout

Table function creation

751a919

arng40 requested a review from MelReyCG February 9, 2024 15:46

arng40 self-assigned this Feb 9, 2024

arng40 added 7 commits February 9, 2024 17:19

add title off case

c84a06e

setMargin function added

57bc055

add comments + test for tiny table

6255f9b

value default optionnal variable changed

601a2c5

Merge remote-tracking branch 'origin/develop' into feature/dudes/tabl…

59a9bc8

…e-layout

add log to wellGeneratorBase

677fdb6

uncrustify + missing doc

623b6ad

MelReyCG requested changes Feb 14, 2024

View reviewed changes

first review with correction

37dc5dd

MelReyCG requested changes Feb 14, 2024

View reviewed changes

arng40 added 2 commits February 15, 2024 13:50

constness modification, doc/test updated

8678fc0

const declaration updated

9475286

MelReyCG requested changes Feb 15, 2024

View reviewed changes

src/coreComponents/codingUtilities/Table.cpp Outdated Show resolved Hide resolved

src/coreComponents/codingUtilities/Table.hpp Outdated Show resolved Hide resolved

doc + minor restructuring

1320651

MelReyCG requested changes Feb 19, 2024

View reviewed changes

arng40 added 8 commits February 20, 2024 11:08

doc + method for row from vector

f78bd83

clean code + doc + add test for rows from vector

294f075

doc updated

36dd7a4

Merge commit 'c11ad8ecd2d0d0aba5cd0661eb6e72951252409d' into feature/…

8f74cfa

…dudes/table-layout

Revert "Merge commit 'c11ad8ecd2d0d0aba5cd0661eb6e72951252409d' into …

ebb9fc0

…feature/dudes/table-layout" This reverts commit 8f74cfa.

revert last merge and Merge remote-tracking branch 'origin/develop' i…

cb41359

…nto feature/dudes/table-layout

CI correction ( uncrustify, missing doc )

e11a2bf

optionnal added to datatype

a8f9271

update doxygen for ci

9ff17b9

This was referenced Feb 27, 2024

PVT Tables logging improvements #2996

Closed

PVT Tables logging improvements #3003

Closed

TotoGaz reviewed May 4, 2024

View reviewed changes

src/coreComponents/fileIO/Table/unitTests/testTable.cpp Show resolved Hide resolved

arng40 added 2 commits May 13, 2024 14:07

review correction #1

fe80c59

variables renamed

abb994e

herve-gross approved these changes May 22, 2024

View reviewed changes

arng40 added 2 commits May 23, 2024 17:10

uncrustify + doc

c851b38

doc

0ae0c74

arng40 added the ci: run CUDA builds Allows to triggers (costly) CUDA jobs label May 23, 2024

Merge remote-tracking branch 'origin/develop' into feature/dudes/tabl…

86c0130

…e-layout

arng40 requested review from TotoGaz and MelReyCG May 24, 2024 07:41

remove unused code

e9639ea

MelReyCG requested changes May 28, 2024

View reviewed changes

arng40 mentioned this pull request May 29, 2024

Log refactor - Output PVT #3149

Open

arng40 added 2 commits May 30, 2024 10:52

doc + code clarification

d6415a7

more code simplification

eaa8919

MelReyCG approved these changes May 31, 2024

View reviewed changes

arng40 requested a review from ryar9534 May 31, 2024 14:06

ryar9534 approved these changes May 31, 2024

View reviewed changes

TotoGaz approved these changes May 31, 2024

View reviewed changes

arng40 and others added 5 commits June 3, 2024 15:49

remove addErrorMsg function

138541b

merge

3f74746

Merge branch 'develop' into feature/dudes/table-layout

d360dcf

Merge remote-tracking branch 'origin/develop' into feature/dudes/tabl…

d34603c

…e-layout

Merge branch 'feature/dudes/table-layout' of https://github.com/GEOS-…

324fb24

…DEV/GEOS into feature/dudes/table-layout

paveltomin removed the flag: ready for review label Jun 4, 2024

Merge branch 'develop' into feature/dudes/table-layout

5b46e14

paveltomin merged commit ce4e169 into develop Jun 4, 2024
26 checks passed

paveltomin deleted the feature/dudes/table-layout branch June 4, 2024 17:57

paveltomin added the flag: no rebaseline Does not require rebaseline label Jun 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Log refactor - Table logging features #2984

Log refactor - Table logging features #2984

arng40 commented Feb 9, 2024 •

edited

MelReyCG left a comment

MelReyCG left a comment

codecov bot commented Feb 23, 2024 •

edited

MelReyCG left a comment

MelReyCG left a comment

MelReyCG left a comment

TotoGaz May 31, 2024

arng40 Jun 3, 2024

TotoGaz May 31, 2024

arng40 Jun 3, 2024

MelReyCG Jun 4, 2024

arng40 Jun 4, 2024

Log refactor - Table logging features #2984

Log refactor - Table logging features #2984

Conversation

arng40 commented Feb 9, 2024 • edited

An example of the refactored Well Log:

1. Data collection

2. Format the data

3. Dumping the data

Log Output

MelReyCG left a comment

Choose a reason for hiding this comment

MelReyCG left a comment

Choose a reason for hiding this comment

codecov bot commented Feb 23, 2024 • edited

Codecov Report

MelReyCG left a comment

Choose a reason for hiding this comment

MelReyCG left a comment

Choose a reason for hiding this comment

MelReyCG left a comment

Choose a reason for hiding this comment

TotoGaz May 31, 2024

Choose a reason for hiding this comment

arng40 Jun 3, 2024

Choose a reason for hiding this comment

TotoGaz May 31, 2024

Choose a reason for hiding this comment

arng40 Jun 3, 2024

Choose a reason for hiding this comment

MelReyCG Jun 4, 2024

Choose a reason for hiding this comment

arng40 Jun 4, 2024

Choose a reason for hiding this comment

arng40 commented Feb 9, 2024 •

edited

codecov bot commented Feb 23, 2024 •

edited