Symbolic Formula predicts worse than model itself

[seyidcemkarakas]

Hi

I have been practicing about KAN. I have made regression implementation. My output layer has 1 node.

After implementation I got pretty good R2 and MAE on my dataset (including train val test). I wanted to get symbolic formula and I got it according to => https://kindxiaoming.github.io/pykan/Examples/Example_3_classfication.html

After that I created a code which gets the symbolic formula of KAN and calculate given inputs according to given formula.
The function is below:

def kan_symbolic_formula_prediction(formula, X):
    batch = X.shape[0]
    predictions = []  # Empty list for keeping predictions

    for i in range(batch):
        # Evaluation on symbolic formula on every single row
        expression = formula
        for j in range(X.shape[1]):
            expression = expression.subs(f'x_{j+1}', X[i, j])
        
        # Get output of formula
        predicted = float(expression.evalf())
        
        predictions.append(predicted)
    
    return predictions

Then I get prediction by using formula like that:

# Get results using symbolic formula
preds_from_kan_formula = kan_symbolic_formula_prediction(formula, X_train.to_numpy())

and here is metrics model.forward() and symbolic formula respectively:

print("MAE from formula on train data",mean_absolute_error(train_labels.numpy(),preds_from_kan_formula))
print("R2 from formula on train data",r2_score(train_labels.numpy(), preds_from_kan_formula))
MAE from formula on train data 0.15750130512335783
R2 from formula on train data -0.47894340657227064

print("MAE from model.forward() on train data",mean_absolute_error(train_labels.numpy(), train_preds.numpy()))
print("R2 from model.forward() on train data",r2_score(train_labels.numpy(), train_preds.numpy()))
MAE from model.forward() on train data 0.04164282
R2 from model.forward() on train data 0.8345471009872176

• As you see symbolic formula performs badly. What do you think about it ?

Here is my full code => https://www.kaggle.com/code/seyidcemkarakas/kan-regression-graduate-admissions

[KindXiaoming]

Hi, since only a limited library of symbolic formulas is provided, it could be that the real symbolic formula is not supported in the library, or even the formula is not symbolic at all. It might be helpful to stare at the learned KAN plot a bit by calling model.plot(), trying to get a sense of what's going on. For example, are there any activation functions that look particularly suspicious?