Set per record log to debug level

[itschrispeck]

Per record level logging is pretty expensive, this log line emits 98% of our logs. I believe changing it to debug level is reasonable.

tags: refactor

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.17%. Comparing base (59551e4) to head (e711ebf).
Report is 290 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #12909      +/-   ##
============================================
+ Coverage     61.75%   62.17%   +0.42%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2502      +66     
  Lines        133233   136390    +3157     
  Branches      20636    21106     +470     
============================================
+ Hits          82274    84797    +2523     
- Misses        44911    45309     +398     
- Partials       6048     6284     +236     

Flag	Coverage Δ
custom-integration1	<0.01% <0.00%> (-0.01%)	⬇️
integration	<0.01% <0.00%> (-0.01%)	⬇️
integration1	<0.01% <0.00%> (-0.01%)	⬇️
integration2	0.00% <0.00%> (ø)
java-11	62.11% <100.00%> (+0.41%)	⬆️
java-21	62.05% <100.00%> (+0.42%)	⬆️
skip-bytebuffers-false	62.14% <100.00%> (+0.40%)	⬆️
skip-bytebuffers-true	62.01% <100.00%> (+34.28%)	⬆️
temurin	62.17% <100.00%> (+0.42%)	⬆️
unittests	62.16% <100.00%> (+0.42%)	⬆️
unittests1	46.76% <0.00%> (-0.13%)	⬇️
unittests2	27.91% <100.00%> (+0.18%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gortiz]

You don't need to modify the source code to change a log. Instead I would recommend to modify the log4j2.xml file you are using to disable the logging on this class (or change it to warn).

For example, adding the following logger:

  <!-- The rest of your log4j2.xml file -->
  <Loggers>
    <Root level="info" additivity="false">
      <AppenderRef ref="pinotLog"/>
    </Root>
    <Logger name="org.apache.pinot.segment.local.recordtransformer.SpecialValueTransformer" level="warn" additivity="false">
      <AppenderRef ref="pinotLog"/>
    </Logger>
  </Loggers>

[itschrispeck]

Is info the right default level for this log? My thinking was debug might be more apt as default given the potentially high volume, and users could override the level if needed.

[Jackie-Jiang]

I think we may introduce an API reportStats() to the RecordTransformer to report the stats collected over all records. We shouldn't log record level info