Creating multiple relationships and nodes from 1 file

[Kamalika17]

Hi,

Is it possible to create multiple relationships and nodes from 1 file or is it necessary to have separate files for each node type and edge type for a knowledge graph. eg: I have 1 data frame where all information is given:
Person, City, Country, Hobby
A, Bern, Switzerland, Painting

Each column is a node type and relationships - A "lives-in" Bern, Bern "capital-of" Switzerland, A "likes" Painting

Thanks for your help!

[andyfengHKU]

Hi @Kamalika17,

Yes if you want to load them into different relationship tables then they should be in different files.

However, if your dataset is not too large. It should also work fine if you make lives-in, capital-of, likes as a relationship property (of type string). So effectively you only have one node table and one relationship table. Everything else is modelled as a property on these two tables.

[prrao87]

Kùzu is different from labelled property graph systems in that you require a primary key column in every node table (it's mandatory), so, in essence, you'd end up taking care of the duplicates yourself from the raw data source. Because primary key checks are always done during import into Kùzu, you don't run the risk of creating duplicates.

1. If importing records one at a time, you could choose between CREATE or MERGE statements. Using CREATE will throw a RuntimeError if you try to import any record that has the same primary key as an existing record (very similar to relational systems like Postgres). MERGE, on the other hand, will first check for the presence of the primary key value via the hash index, and if it exists, overwrites the existing node with the new node's properties. No duplicates :)
2. If bulk-importing records using COPY FROM, the system once again checks for primary key constraint violations and throws a RuntimeError if there are any duplicate records with the same primary key.

Because the decision of what the primary key is (and how it's created) sits in the hands of the user, you can think of Kùzu's data model as a structured property graph model, where everything sits in (node or relationship) tables, so you'd approach the data preparation step in a very similar way to how you would in any RDBMS. Hope that makes sense!

[prrao87]

I'd also add a couple more points here regarding performance: As mentioned in the docs, the recommended way is always to perform bulk imports rather than record-at-a-time, for performance reasons. If you really want to use CREATE and MERGE (say for real-time updates when you don't have pre-gathered records), bear in mind that MERGE would be slightly slower than CREATE, due to the added lookups involved, but both are slower than bulk-imports.

[Kamalika17]

Thanks a lot! Is it possible to import in bulk directly from a pandas data frame?
https://docs.kuzudb.com/import/

[andyfengHKU]

Yes I guess you could do something around

COPY <table_name> FROM ( LOAD FROM <data_frame> RETURN <columns> )

[prrao87]

Yup, the example snippet you're looking for is here. The link to this section was broken, it's been fixed now :).

Basically you'd do something along these lines:

# Assumes that you have a Kùzu connection object named `conn`
# Also assumes that you created a node table named `Person` with columns `name` and `age`
import pandas as pd

df = pd.DataFrame({
    "name": ["Adam", "Karissa", "Zhang", "Noura"],
    "age": [30, 40, 50, 25]
})

conn.execute("COPY Person FROM (LOAD FROM df WHERE age < 30 RETURN *)")

You can scan the Pandas DataFrame using the LOAD FROM df WHERE age < 30 RETURN * -- note that you can be selective and specify filter conditions, and whatever's found in Pandas will be returned by the RETURN * portion of the subquery (you can also return only specific columns of the Pandas DataFrame if desired). The results of the subquery are then passed to COPY FROM as shown above.