invalid results on during evaluation

[riyaj8888]

Hi,

I am following this tutorial https://www.tensorflow.org/recommenders/examples/basic_retrieval .
during model.fit and model.evaluate i am passing the same dataset but i am getting very different numbers like below.

after model.evaluate results:
{'factorized_top_k/top_1_categorical_accuracy': 6.989097164478153e-05, 'factorized_top_k/top_5_categorical_accuracy': 0.0002096729149343446, 'factorized_top_k/top_10_categorical_accuracy': 0.0003494548436719924, 'factorized_top_k/top_50_categorical_accuracy': 0.000629018759354949, 'factorized_top_k/top_100_categorical_accuracy': 0.0011182555463165045, 'loss': 14674.890625, 'regularization_loss': 0, 'total_loss': 14674.890625}

during training results:
Epoch 3/10 1/2 [==============>...............] - ETA: 2s - factorized_top_k/top_1_categorical_accuracy: 0.4688 - factorized_top_k/top_5_categorical_accuracy: 0.4733 - factorized_top_k/top_10_categorical_accuracy: 0.4733 - factorized_top_k/top_50_categorical_accuracy: 0.4745 - factorized_top_k/top_100_categorical_accuracy: 0.4752 - loss: 19850.7344 - regularization_loss: 0.0000e+00 - total_loss: 19852/2 [==============================] - ETA: 0s - factorized_top_k/top_1_categorical_accuracy: 0.6085 - factorized_top_k/top_5_categorical_accuracy: 0.6209 - factorized_top_k/top_10_categorical_accuracy: 0.6273 - factorized_top_k/top_50_categorical_accuracy: 0.6323 - factorized_top_k/top_100_categorical_accuracy: 0.6348 - loss: 17292.4678 - regularization_loss: 0.0000e+00 - total_loss: 17292/2 [==============================] - 8s 6s/step - factorized_top_k/top_1_categorical_accuracy: 0.6085 - factorized_top_k/top_5_categorical_accuracy: 0.6209 - factorized_top_k/top_10_categorical_accuracy: 0.6273 - factorized_top_k/top_50_categorical_accuracy: 0.6323 - factorized_top_k/top_100_categorical_accuracy: 0.6348 - loss: 16439.7122 - regularization_loss: 0.0000e+00 - total_loss: 16439.7122 - val_factorized_top_k/top_1_categorical_accuracy: 0.0000e+00 - val_factorized_top_k/top_5_categorical_accuracy: 0.0000e+00 - val_factorized_top_k/top_10_categorical_accuracy: 0.0000e+00 - val_factorized_top_k/top_50_categorical_accuracy: 0.0000e+00 - val_factorized_top_k/top_100_categorical_accuracy: 0.0000e+00 - val_loss: 14709.7910 - val_regularization_loss: 0.0000e+00 - val_total_loss: 14709.7910

i kept all train-val-test dataset as same .

`batch_size = config.BATCH_SIZE
S = num_samples

train_sz = int(S*0.8)
val_sz = S - train_sz

shuffled_train = ratings.shuffle(S ,seed=seed, reshuffle_each_iteration=True)
train = shuffled_train.take(train_sz)
val = shuffled_train.skip(train_sz).take(val_sz)

cached_train = train.shuffle(S).batch(batch_size).cache()
cached_val = val.shuffle(S).batch(batch_size).cache()
cached_test = ratings_test.batch(batch_size).cache()`

model.fit(cached_train, epochs=config.EPOCHS,validation_data=cached_train)
model.evaluate(cached_train, return_dict=True)

i would appreciate your help
thanks

[riyaj8888]

{'factorized_top_k/top_1_categorical_accuracy': [0.04586755111813545, 0.012231347151100636, 0.014677616767585278, 0.01895858906209469, 0.02192905731499195, 0.04359601438045502, 0.038528744131326675, 0.03363620489835739, 0.03302463889122009, 0.035558272153139114], 'factorized_top_k/top_5_categorical_accuracy': [0.04752752184867859, 0.02341429330408573, 0.04298444837331772, 0.027782632037997246, 0.03791717812418938, 0.058011531829833984, 0.07233968377113342, 0.05626419559121132, 0.06526297330856323, 0.06141883507370949], 'factorized_top_k/top_10_categorical_accuracy': [0.04883802309632301, 0.03616984188556671, 0.0710291787981987, 0.03529617190361023, 0.056351564824581146, 0.06613664329051971, 0.1050148531794548, 0.07120391726493835, 0.10274332016706467, 0.08640573173761368], 'factorized_top_k/top_50_categorical_accuracy': [0.05198322609066963, 0.21160230040550232, 0.19622576236724854, 0.1167220026254654, 0.21212650835514069, 0.09278350323438644, 0.20548664033412933, 0.11174209415912628, 0.2239210158586502, 0.16512319445610046], 'factorized_top_k/top_100_categorical_accuracy': [0.05626419559121132, 0.41280797123908997, 0.2412196397781372, 0.2149222493171692, 0.3554953634738922, 0.1028306856751442, 0.23248296976089478, 0.12484710663557053, 0.24384064972400665, 0.1822470724582672], 'loss': [714189.8125, 2223343.5, 3307595.25, 1418317.25, 927078.75, 725588.375, 566257.1875, 506635.71875, 408394.125, 331931.03125], 'regularization_loss': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'total_loss': [714189.8125, 2223343.5, 3307595.25, 1418317.25, 927078.75, 725588.375, 566257.1875, 506635.71875, 408394.125, 331931.03125], 'val_factorized_top_k/top_1_categorical_accuracy': [0.002795248059555888, 0.004542278125882149, 0.003144653979688883, 0.010482179932296276, 0.007337526418268681, 0.010482179932296276, 0.01292802207171917, 0.010132774710655212, 0.01187980454415083, 0.007686932105571032], 'val_factorized_top_k/top_5_categorical_accuracy': [0.003144653979688883, 0.006289307959377766, 0.006638714112341404, 0.016422081738710403, 0.016771487891674042, 0.016072675585746765, 0.018169112503528595, 0.013277428224682808, 0.016072675585746765, 0.012578615918755531], 'val_factorized_top_k/top_10_categorical_accuracy': [0.003494060132652521, 0.008385743945837021, 0.009783368557691574, 0.019566737115383148, 0.02341020293533802, 0.02131376601755619, 0.02375960908830166, 0.016072675585746765, 0.019916143268346786, 0.016072675585746765], 'val_factorized_top_k/top_50_categorical_accuracy': [0.005590496119111776, 0.011530398391187191, 0.019916143268346786, 0.035290006548166275, 0.04751921817660332, 0.03668763116002083, 0.04297693818807602, 0.029350105673074722, 0.03598881885409355, 0.03144654259085655], 'val_factorized_top_k/top_100_categorical_accuracy': [0.006289307959377766, 0.015024458058178425, 0.026205450296401978, 0.04332634434103966, 0.05730258673429489, 0.04437456279993057, 0.05310971289873123, 0.03668763116002083, 0.04332634434103966, 0.03808525577187538], 'val_loss': [3966347.25, 3077428.5, 3170788.75, 2097657.5, 1538916.5, 1169540.625, 974531.375, 816660.1875, 701207.0625, 591413.8125], 'val_regularization_loss': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'val_total_loss': [3966347.25, 3077428.5, 3170788.75, 2097657.5, 1538916.5, 1169540.625, 974531.375, 816660.1875, 701207.0625, 591413.8125]}

[riyaj8888]

any update?

[riyaj8888]

?