Add new algorithms - GATE and TabR

[strukevych]

There are two 2023 models that outperform MLP on all or almost all datasets:
GATE https://arxiv.org/abs/2207.08548
TabR https://arxiv.org/abs/2307.14338

They also outperformed CatBoost/XGBoost/LightGBM on most datasets.

TabR is also a KNN on steroids, so it could be a good replacement for the NeuralNetwork algorithm in that repository.

We can drop NeuralNetwork or/and KNN from the algorithms after implementation.

On some datasets, they achieved even a 10% boost in accuracy metric on the tuned model compared to other tuned alternatives.

They are suitable for both classification and regression problems, but the best performance boost they achieving on the classification datasets

[pplonski]

Amazing results for GATE and TabR. Do they have github repos available with implementation? Have you tried those algorithms?

[strukevych]

Amazing results for GATE and TabR. Do they have github repos available with implementation? Have you tried those algorithms?

Yes, TabR has a single benchmark repo (with implementation included)

As I understood, TabR is done by Yandex (same as CatBoost), and it's a kind of replacement for CatBoost, KNN and MLP at the same time.
They also have a very cool feature named context freezing. It can speed up learning up to x100 for a small cost of accuracy so that it can be used even for such things as an exploration on the data.

GATE has repositories both with benchmarks and several implementations

[strukevych]

I will do a prototype with a benchmark on my dataset and will place a PR for these algorithms.

But I will take that only after improving handle_imbalance and feature selection for fair results.

[pplonski]

Here is TabR repo https://github.com/yandex-research/tabular-dl-tabr

[strukevych]

Here is TabR repo https://github.com/yandex-research/tabular-dl-tabr

Yes, exactly. It's a benchmarking repo, basically :)

But we can still take implementation from it

I just meant that GATE has an implementation that you can install from pip (not a git URL) and has active support