You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
It would be great to have built into MLJ a way for searching measures suitable to a given predictions yhat and target y, similar to the model searching functionality already implemented.
Describe the solution you'd like
So I propose something along these lines, measures(matching(yhat, y)), which would return a list of measures that can be used suitable for predictions yhat and target y.
The text was updated successfully, but these errors were encountered:
Okay this design won't do because of ambiguity with models(matching(X, y)). Also I don't like the fact that matching is forcing us to couple model search code with measure search code. Measures will eventually be independent of the whole stack, except StatisticalTraits.jl.
One suggestion is to deprecate predicates as arguments of models and do something like this:
models() acts as currently
models(str) acts as currently, for str a string or regex
models((X, y)) acts like models(matching(X, y)) does now.
models(p) falls back to assume p is a predicate (such as matching(X, y)) but this behaviour is deprecated, as not needed (we have filter for that).
measures((yhat, y)) finds all measures compatible with (yhat, y)
A breaking alternative is to modify the third item above and allow models(X, y) to behave as models(matching(X, y)) does now, but that would disallow the possibility that X is a string, say, and we would have to disallow predicates immediately.
Another nice enhancement would be to allow kwargs for searching over trait values, as in models(package_name="ScikitLearn", is_supervised=true). Currently we have the ugly
models() do m
m.package_name(m) ==ScikitLearn" && m.is_supervisedend
Is your feature request related to a problem? Please describe.
It would be great to have built into MLJ a way for searching measures suitable to a given predictions
yhat
and targety
, similar to the model searching functionality already implemented.Describe the solution you'd like
So I propose something along these lines,
measures(matching(yhat, y))
, which would return a list of measures that can be used suitable for predictionsyhat
and targety
.The text was updated successfully, but these errors were encountered: