Skip to content
This repository has been archived by the owner on Sep 21, 2022. It is now read-only.

Make all logging consistent and now only using the log kit pkg #389

Merged
merged 2 commits into from Feb 5, 2021

Conversation

TheRocketCat
Copy link
Contributor

@TheRocketCat TheRocketCat commented Jan 26, 2021

fixes: #185

When changing from fmt.Pr and Logrus to the go-kit logger i tried to follow the flow (with minimal code change), in terms of editing comments and making code changes.

There was one fmt.Pr i did not change, it was in the cmd/telliot/cmd.go Run function (i think), because it seemed to actually want to print info about the programs version.

Also by the end of this i had an extra file, that i have no idea if it actually is part of the project, it seemed to appear after i pulled some changes to make sure my code was auto mergable.

I did not remove it (even tho i cant find it in your repo), because when i realised it seemed entierly new i had already editeded its fmt.Pr* and did not have the heart to potentially discard my extra work. File in question pkg/db/localDataProxy.go

Copy link
Collaborator

@krasi-georgiev krasi-georgiev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it is starting to look much better 👍

There was a little bit of oversight on my end. We want to have proper logging only for the mine and dataserver command as these are the only long running ones. For the other cmds that just execute an action and exit would be better to keep the use of fmt.Print as this will present the result in a cleaner way than with a log line.

With this PR we want to also remove cfg.LogLevel from the config.
so instead of createLogger we will have 2 functions NewLogger and ApplyFilter

func NewLogger() log.Logger{
	logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
	return log.With(logger, "ts", log.TimestampFormat(func() time.Time { return time.Now().UTC() }, "Jan 02 15:04:05.99"), "caller", log.DefaultCaller)
}

func ApplyFilter(logLevel string) (log.Logger,error) {
	lvl := level.AllowInfo()
	if level,ok:= cfg.Logger[ComponentName];ok{
		switch level {
		  case "error":
			  lvl = level.AllowError()
		  case "warn":
			  lvl = level.AllowWarn()
		  case "info":
			  lvl = level.AllowInfo()
		  case "debug":
			  lvl = level.AllowDebug()
		  default:
			  return nil,errors.Errorf("unexpected log level:%v",level)
		  }
	}

	return level.NewFilter(logger, lvl),nil
}

basically each component in the cli has its own log level set by the config file.

In each major component constructor that takes a logger as a parameter should have something like:

const ComponentName = "trackers"
New(logger log.Logger, ctx...., cfg){
	logger,err = ApplyFilter(cfg,ComponentName,logger)
}

This granular setup is important only for the dataserver and mine command as these are the only long-running commands, all the others just execute a single action and exit.

tl-dr - all commands should use NewLogger in their Run()
Additionaly mine and dataserver cmds should also use ApplyFilter in the constructors of some internal components where the logging might need more granular filtering.

At the end the logrus import should be removed "github.com/sirupsen/logrus"

pkg/db/db.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@krasi-georgiev krasi-georgiev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid circular imports create a package logger and put ApplyFilter and New in it so we have logger.ApplyFilter and logger.New

the logger and config should be passed to the component constructors use global constant for the ComponentName

So in tl-dr in cmds.go use lgr:=util.NewLogger() and cfg:=config.GetConfig

Then pass these when you initialize a component. for example on package ops

package ops
New(lgr,cfg){
	lgr = logger.ApplyFilter(cfg, ComponentName, lgr)
	lgr = log.With(lgr, "component", ComponentName)
}

cmd/telliot/cmds.go Outdated Show resolved Hide resolved
cmd/telliot/cmds.go Outdated Show resolved Hide resolved
cmd/telliot/cmds.go Outdated Show resolved Hide resolved
cmd/telliot/main.go Outdated Show resolved Hide resolved
docs/README.md Outdated Show resolved Hide resolved
pkg/apiOracle/valueOracle.go Outdated Show resolved Hide resolved
pkg/dataServer/dataServer.go Outdated Show resolved Hide resolved
pkg/db/db.go Outdated Show resolved Hide resolved
pkg/db/localDataProxy.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@krasi-georgiev krasi-georgiev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also update the configs/config.json

and also need to remove the logLevel flag from the cli flags.

pkg/apiOracle/valueOracle.go Outdated Show resolved Hide resolved
pkg/apiOracle/valueOracle.go Outdated Show resolved Hide resolved
pkg/apiOracle/valueOracle.go Outdated Show resolved Hide resolved
pkg/db/proxy.go Outdated Show resolved Hide resolved
pkg/rpc/mock.go Show resolved Hide resolved
pkg/tracker/psrs.go Show resolved Hide resolved
pkg/tracker/psrs.go Outdated Show resolved Hide resolved
pkg/rpc/mock.go Outdated Show resolved Hide resolved
pkg/util/logConfig.go Outdated Show resolved Hide resolved
pkg/util/logger.go Outdated Show resolved Hide resolved
@coveralls
Copy link

coveralls commented Feb 4, 2021

Pull Request Test Coverage Report for Build 540039912

  • 167 of 473 (35.31%) changed or added relevant lines in 36 files are covered.
  • 11 unchanged lines in 7 files lost coverage.
  • Overall coverage increased (+0.4%) to 38.659%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pkg/config/testing.go 0 1 0.0%
pkg/ops/TxnSubmitter.go 0 1 0.0%
pkg/rpc/contractRPC.go 0 1 0.0%
pkg/tracker/index.go 6 7 85.71%
pkg/dataServer/dataServer.go 3 5 60.0%
pkg/ops/dataServerOps.go 10 12 83.33%
pkg/tracker/gas.go 2 4 50.0%
pkg/pow/miningGroup.go 13 16 81.25%
pkg/tracker/disputeChecker.go 8 12 66.67%
pkg/tracker/runner.go 3 7 42.86%
Files with Coverage Reduction New Missed Lines %
pkg/db/remoteRequest.go 1 61.67%
pkg/ops/miningManager.go 1 0%
pkg/rest/server.go 1 0%
pkg/db/proxy.go 2 47.04%
pkg/ops/disputeOps.go 2 0%
pkg/ops/transferOps.go 2 0%
pkg/rpc/mock.go 2 0%
Totals Coverage Status
Change from base Build 534879226: 0.4%
Covered Lines: 1522
Relevant Lines: 3937

💛 - Coveralls

pkg/ops/miningManager.go Outdated Show resolved Hide resolved
pkg/util/httpRetriever.go Outdated Show resolved Hide resolved
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
@krasi-georgiev krasi-georgiev changed the title This is for fixing the #185 in regards to the gitcoin bounty Make all logging consistent and now only using the log kit pkg Feb 5, 2021
@krasi-georgiev krasi-georgiev merged commit 3300e3a into tellor-io:master Feb 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use consisent logging instead of printf everywhere.
3 participants