ABC metrics

[boyter]

Discussion point for ABC metrics

[boyter]

The way I see it working is as an option, as an opt in feature as I suspect that it would otherwise slow the tool down. Users can then select to flip into this mode getting better calculations where there is a language that supports it. Ideally with the ability for more parsers to be built into scc over time growing its footprint. What you have suggested above however looks about right, as you could add some new definitions into the languages.json to deal with this.

[xxxserxxx]

Thanks for your email. A ticket is perfect!

I agree that an opt-in is the right approach, if for no other reason than preserving backwards compatability in the output.

Can we agree on the schema changes before I start? I want to make sure you don't have a better idea before I start. If you prefer, we can use json-schema.

ABC needs to recognize and tally three categories of logic: (1) assignment; (2) branches; and (3) conditionals. languages.json contains most of these tokens but doesn't have metadata to assign categories, so this would be added. One solution would add this metadata:

  "C": {
    "complexitychecks": [
      { "assignment": [
         "=", 
         "*=",
         "/=",
         "%=",
         "+=",
         "<<=",
         ">>=",
         "&=",
         "!=",
         "^=",
         "++",
         "--"
      ] },
      { "conditional": [
         "<",
         ">",
         "<=",
         ">=",
         "==",
         "!="",
         ""<",
         ">",
         "<=",
         ">=",
         "==",
         "!="
      ] }, 
      { "branch": [ 
      	"(",
      	"goto"
      ] },
      { "keyword": [
         "for ",
         "for(",
         "if ",
         "if(",
         "switch ",
         "while ",
         "else ",
         "|| ",
         "&& ",
         "!= ",
         "== "
      ] }
    ],
    "extensions": [
      "abap",
      "ABC"
    ],
    "line_comment": [
      "*",
      "\\\""
    ],
    "multi_line": [],
    "quotes": []
  },

This change minimizes structural changes, as opposed to -- say -- making each keyword a struct of its own. However, it does embed ABC metrics metadata into the structure, which is simple, but dangerous. Internally, I'd probably flatten this back out to the original data structure, and build a separate map for the metric categories that is only used by the ABC code; this would minimize changes and have a one-time startup cost, leaving the performance (hopefully) flat. This may require separating the marshalling structure from the processing structure.

The biggest problem I forsee is that the branch rules are less simple than token recognition. For example, for C the rule is that a branch is any function call, or goto at a deeper level of nesting than the goto. So scc would need to recognize function calls and also track scope depth, which may be complex. abcgo used Go's internal ast parser and so had an easier job, but this wouldn't work for many languages -- and I wonder whether you want to add the complexity of a new metric only for Go source trees? From what I've seen, scc isn't building a parse tree, so I could have challenges here.

For the UI, ABC can report on different scopes and the user would need to choose from them. ABC is nominally a vector of the three categories, and commonly a scalar (summary) score is calculated. These scores are reported per function or package, or for the entire project. A flag to enable the metric could take function or package -- the default would be the whole project. Scope would be further limited by the existing file/dir argument.

[boyter]

Still digesting this.

I think I would prefer if the complexity check was kept as is and the new rules put as another field inside the language, just to ensure that its 100% backwards compatible, and I can see some cases where you might want both. It means there are less changes inside the application itself as well which is always a plus.

[boyter]

Yep I think other than splitting the complexity out this seems good to me.