Describe the bug

I am trying to reproduce the LiteralE results mentioned in the original paper, starting from DistMult+LiteralE-g_lin. The hyperparameter values that I used are the ones mentioned in the paper.

After running the pipeline mentioned in the next section, the both.realistic metrics that I get are:

"adjusted_arithmetic_mean_rank": 0.9079933755507523,
"adjusted_arithmetic_mean_rank_index": 0.09201952015962944,
"adjusted_geometric_mean_rank_index": 0.22190754750602726,
"adjusted_hits_at_k": 0.0032136119490491994,
"adjusted_inverse_harmonic_mean_rank": 0.001488301332888998,
"arithmetic_mean_rank": 6479.14013671875,
"count": 40876.0,
"geometric_mean_rank": 4080.906982421875,
"harmonic_mean_rank": 454.56912614955536,
"hits_at_1": 9.785693316371466e-05,
"hits_at_10": 0.003914277326548586,
"hits_at_3": 0.0010030335649280752,
"hits_at_5": 0.0017858890302377924,
"inverse_arithmetic_mean_rank": 0.00015434146916959435,
"inverse_geometric_mean_rank": 0.00024504357133992016,
"inverse_harmonic_mean_rank": 0.0021998854354023933,
"inverse_median_rank": 0.00015867978800088167,
"median_absolute_deviation": 5447.080550789581,
"median_rank": 6302.0,
"standard_deviation": 4214.4306640625,
"variance": 17761424.0,
"z_arithmetic_mean_rank": 32.18230897434722,
"z_geometric_mean_rank": 44.94204728253047,
"z_hits_at_k": 24.49747148157558,
"z_inverse_harmonic_mean_rank": 28.025614680547044

As it can be seen, the metric values are orders of magnitude worse than the results of the original paper. For example, the hits_at_10 is 2 orders of magnitude lower than the one reported in the paper (see paper's table 4).

How to reproduce

The steps to reproduce the issue are the following:

1. Download the numeric literal triples file from the official LiteralE repo.

2. Download the official FB15k-237 triples.

3. Place test.txt, train.txt, valid.txt and numerical_literals.txt in a directory named fb15k237 inside your project's sources directory.

4. In the above directory, create a file named __init__.py with the following contents:

   # -*- coding: utf-8 -*-
   
   """Get triples from the FB15k-237 dataset with literals."""
   
   import pathlib
   
   __all__ = [
       "FB15K237_TRAIN_PATH",
       "FB15K237_TEST_PATH",
       "FB15K237_VALIDATE_PATH",
       "FB15K237_LITERALS_PATH",
       "FB15K237Literal",
   ]
   
   from pykeen.datasets import NumericPathDataset
   from pykeen.triples import TriplesNumericLiteralsFactory
   
   HERE = pathlib.Path(__file__).resolve().parent
   
   FB15K237_TRAIN_PATH = HERE.joinpath("train.txt")
   FB15K237_TEST_PATH = HERE.joinpath("test.txt")
   FB15K237_VALIDATE_PATH = HERE.joinpath("valid.txt")
   FB15K237_LITERALS_PATH = HERE.joinpath("numerical_literals.txt")
   
   
   class FB15K237Literal(NumericPathDataset):
       training: TriplesNumericLiteralsFactory
   
       def __init__(self, create_inverse_triples: bool = False, **kwargs):
           super().__init__(
               training_path=FB15K237_TRAIN_PATH,
               testing_path=FB15K237_TEST_PATH,
               validation_path=FB15K237_VALIDATE_PATH,
               literals_path=FB15K237_LITERALS_PATH,
               create_inverse_triples=create_inverse_triples,
               **kwargs,
           )
   
   
   if __name__ == "__main__":
       FB15K237Literal().summarize()

5. Specify the FB15K237Literal class as a component for the pykeen.datasets entry point in your project's config file (as proposed in pykeen.datasets.__init__.py).
   For example, if you are using poetry, then the following should be added in pyproject.toml:

   [tool.poetry.plugins."pykeen.datasets"]
   fb15k237literal = "fb15k237:FB15K237Literal"   

6. Run the following code:

   result = pipeline(
       model='DistMultLiteral',
       dataset="fb15k237literal",
       epochs=100,
       stopper='early',
       stopper_kwargs=dict(metric='inverse_harmonic_mean_rank', frequency=3),
       result_tracker='console',
       model_kwargs=dict(embedding_dim=200, input_dropout=0.2),
       loss='BCEWithLogitsLoss',
       training_kwargs=dict(batch_size=128, label_smoothing=0.1),
       optimizer_kwargs=dict(lr=0.001),
       training_loop='LCWATrainingLoop'
   )
   result.save_to_directory('./distmult_literal_e')

Environment

Additional information

No response

Issue Template Checks

• This is not a feature request (use a different issue template if it is)
• This is not a question (use the discussions forum instead)
• I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed