Perform fully-inductive OR semi-inductive link prediction using my own dataset

[serenalotreck]

Hi All,

I have a dataset of scientific abstracts manually labeled with gold standard entities and relations, which can be used to build a TriplesFactory.

Separately, I have an information extraction model that identifies entities and relations in new abstracts from the same scientific discipline.

I would like to use a PyKEEN model trained on my existing gold standard data to make predictions of links between the predictions on a new abstract(s). In essence, I'd like to write a piece of code that takes a set of triples from a new abstract as an input (a "mini-graph"), and outputs a set of the highest-scoring predicted links between entities in the mini-graph.

The mini-graph may or may not have entities that overlap with those in the original training set, so it could be either semi-inductive or fully-inductive, depending on the abstract. I would also like to be able to do this iteratively; that is, re-apply the same pre-trained model on new abstracts as I receive them.

I've read the introduction to inductive predictions in the docs; however, I can't quite figure out how to adapt it to my use case.

Specifically speaking, I'm confused about how to generate all four of the triples factories indicated in the docs: transductive_training, inductive_inference, inductive_validation, inductive_test. Right now, I have my gold standard dataset set up as transductive_training, and the abstract of interest's triples as inductive_inference.

As of right now, I have the following implementation:

from pykeen.triples import TriplesFactory
from pykeen.training import SLCWATrainingLoop
from pykeen.models.inductive import InductiveNodePiece
from pykeen.losses import NSSALoss
from torch.optim import Adam
from pykeen.predict import predict_all

# Read in data as lists of 3-tuples, where each 3-tuple is a triple
gold_trips = ...
one_abstract_trips = ...

tf_gold = TriplesFactory.from_labeled_triples(trips, create_inverse_triples=True)
tf_one_abstract = TriplesFactory.from_labeled_triples(one_abstract_trips, create_inverse_triples=True)

model = InductiveNodePiece(
    triples_factory=tf_gold,  # training factory, used to tokenize training nodes
    inference_factory=tf_one_abstract,  # inference factory, used to tokenize inference nodes
    num_tokens=12,  # length of a node hash - how many unique relations per node will be used
    aggregation="mlp",  # aggregation function, defaults to an MLP, can be any PyTorch function
    loss=NSSALoss(margin=15),  # dummy loss
    random_seed=42,
)

optimizer = Adam(params=model.parameters(), lr=0.0005)

training_loop = SLCWATrainingLoop(
    triples_factory=tf_gold,  # training triples
    model=model,
    optimizer=optimizer,
    mode="training",   # necessary to specify for the inductive mode - training has its own set of nodes
)

training_loop.train(
    triples_factory=tf_gold,
    # stopper=early_stopper,
    num_epochs=100,
)

predictions = predict_all(model=model)

It all runs, except for the last line. predict_all only runs if I pass mode=training, but then it's operating on my gold standard triples, not the one abstract. Additionally, the only other options to pass for mode are validation and testing -- which to my undestanding, still wouldn't be making predictions on my one abstract even if I had provided them.

I have a feeling I'm missing some fundamental concept about inductive predictions here; would be very appreciative of any and all feedback/advice!

[mberr]

Hi @serenalotreck, the inductive model was mostly coded with the "standard" benchmark setting of a fixed pre-defined validation/test split in mind, rather than the open-ended, continuous prediction setting.

• transductive_training: the training portion
• inductive_inference: the triples available at inference (validation/test time)
• inductive_validation / inductive_test: the triples to score

It is not yet fully integrated with the prediction workflow we developed for the transductive setting. So, you would need to set, e.g., the validation factory to the one you want to run prediction with, e.g.:

model = InductiveNodePiece(
    ...,
    inference_factory=tf_one_abstract,
    validation_factory=tf_one_abstract, # <-
)

When running this code, I found another issue where the number of entities was not properly set; this will be fixed with #1320

[serenalotreck]

EDIT: I just noticed that inference_factory alone should already be enough.

The model training does run if I only pass inference_factory in addition to the training triples, but there isn't an option to pass mode=inference at prediction time -- doesn't that mean I do still need validation_factory?

Okay, this looks to be a bit more involved than I initially thought it was; I'll see whether we can create a utility method here.

Thanks so much, that would be awesome!

[mberr]

The model training does run if I only pass inference_factory in addition to the training triples, but there isn't an option to pass mode=inference at prediction time -- doesn't that mean I do still need validation_factory?

By default, if no validation factory is provided, the entity representations for validation are created based on the inference factory; thus, you should be able to specify mode="validation".

[mberr]

Thanks so much, that would be awesome!

cf. #1322

[serenalotreck]

By default, if no validation factory is provided, the entity representations for validation are created based on the inference factory; thus, you should be able to specify mode="validation".

I get the same error about shape, so this seems correct!

cf. #1322

Thanks for jumping on that! Looking forward to seeing these integrated & using them!

[mberr]

#1322 is merged into master and will be part of our next release (soon)