Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Why do you use Apache AGE? #1705

Open
markgomer opened this issue Mar 18, 2024 · 37 comments
Open

Why do you use Apache AGE? #1705

markgomer opened this issue Mar 18, 2024 · 37 comments
Labels
question Further information is requested

Comments

@markgomer
Copy link
Contributor

markgomer commented Mar 18, 2024 •
edited

Hello Apache AGE users!

I am curious to see use cases of Apache AGE to better understand and support our community.
Whether it's for data analysis, application development, college task or anything else, we'd love to hear how you're using AGE.

Please share any of the following:

  • Your project or application type;
  • Any integrations with other technologies;
  • Challenges solved with Apache AGE;
  • Unique or interesting use cases; or
  • Anything else you think it's worth sharing.

Why? These insights help us improve AGE and guide our development priorities.

How to Share: Simply comment below with details about your use of Apache AGE, also provide a link to your project if you have any.

Thank you for contributing to our community!
@markgomer markgomer added the question Further information is requested label Mar 18, 2024
@mingfang
Copy link

mingfang commented Mar 20, 2024 •
edited

My number one reason is because AGE is built on top of Postgres.
Most of our relational data is already in Postgres so AGE enables the combination of SQL and Cypher, very powerful.

The biggest challenge I'm having with AGE is trying to extend AGE with search capability.
I'm using ParadeDB's pg_bm25 extension but it can't index AGE's agtype data.
My ultimate goal is to integrate AGE with LLM but it needs searching capability(both bm25 and vector search).

For reference, these are the use cases I need.
https://neo4j.com/blog/unifying-llm-knowledge-graph/

@KamalAman
Copy link

Essentially that ^, hopefully Age development can get some boosters it to catch up to neo4j.

@MajorMask
Copy link

I am working with a firm that is using AGE for our PostgreSQL layer for authentication purposes in our app and taking care of multiple instances in a graph-based mode to develop the solution with a metered connection. Cypher and PostgreSQL combination would be of good use to me there.

@aked21 aked21 pinned this issue Mar 25, 2024
@diangamichael
Copy link

In bioinformatics and genomics research, Apache Age is used to analyze genetic data and intricate biological networks. Apache Age makes it easier to identify genes linked to disease, analyze biological pathways, and uncover the functions of genes by visualizing genetic and protein interactions as graphs. Precision agriculture, tailored medicine, and drug development are all benefited by this use case. With its ability to handle and analyze graph data, Apache Age is a potent tool that spurs innovation in a variety of industries and provides insightful information. Apache Age shows itself to be a flexible solution for utilizing the potential of graph data, regardless of the application—social network analysis, recommendation systems, fraud detection, knowledge graph management, network analysis, or biology. These are the use cases that i know of and am confident Apache Age can do

@dpdjvhxm dpdjvhxm mentioned this issue Mar 26, 2024
Merged
@kpushpankar2
Copy link

I use Apache AGE because of the following Reason:

  1. Social Networks : Social networking platforms are inherently graph-based, with users, connections, groups, and interactions forming a complex web of relationships. Apache AGE can be used to efficiently manage and query these relationships, enabling features like friend recommendations, content personalization, and network analysis.

  2. Recommendation Systems : E-commerce websites, streaming services, and content platforms often rely on recommendation systems to suggest products, movies, or articles to users. Apache AGE can help in modeling the relationships between users and items, as well as between the items themselves, to generate personalized recommendations based on user behavior and preferences.

  3. Fraud Detection : In the financial and e-commerce sectors, detecting and preventing fraud is crucial. Apache AGE can be used to analyze transaction networks, identify unusual patterns, and flag potential fraudulent activities by examining the relationships and behaviors across a network of transactions and accounts.

  4. Network and IT Operations : Companies managing large IT infrastructures can use Apache AGE to model their networks, servers, and devices as a graph. This can help in optimizing network topology, troubleshooting issues by analyzing dependencies and impacts, and enhancing security through network analysis.

  5. Knowledge Graphs : Organizations that need to manage complex sets of data and relationships, such as in research, legal, or medical fields, can benefit from using Apache AGE to create knowledge graphs. These graphs help in organizing information in a way that reflects real-world entities and their interconnections, making it easier to derive insights and relationships.

  6. Supply Chain Management : In supply chain management, understanding the relationships between suppliers, manufacturers, distributors, and retailers is crucial. Apache AGE can model these relationships as a graph, providing insights into efficiency, dependency, and risk analysis, which can help in optimizing the supply chain.

  7. Bioinformatics : In the field of bioinformatics, graph databases can be used to model complex biological systems, including genetic, protein, and metabolic interactions. Apache AGE can facilitate the analysis of these interactions, contributing to research in genetics, drug discovery, and personalized medicine.

@markgomer
Copy link
Contributor Author

@diangamichael @kpushpankar2
do you know which companies are using Apache AGE like this? Or are you personally using Apache AGE for those use cases? We’d love to hear about any real-world examples you have. Thanks! 😃

@SaurabhPrakash2020
Copy link

Imagine a social networking site where users can connect with each other, follow each other, and share content. In such a scenario, the relationships between users can be modeled as a graph, where users are nodes and connections (e.g., friendships, follows) are edges.

With Apache Age, you could store this graph in a PostgreSQL database, using its graph capabilities to efficiently query and analyze the relationships between users. You could use graph queries to find mutual friends, recommend new connections based on common interests, or analyze the network structure to identify influential users.

Additionally, Apache Age's integration with Apache Hadoop could be used for large-scale analysis of the social network data, such as identifying trends, clustering users based on behavior, or detecting anomalies. This combination of PostgreSQL with Apache Hadoop through Apache Age provides a powerful platform for building and analyzing complex social networks.

@markgomer companies which uses Apache Age :- Boeing.com, Jacobs.com, Zapata Technology, michigan.gov etc.

@kpushpankar2
Copy link

@markgomer The company uses apache age

1.Zepta Technology utilizes Apache AGE within their platform to enhance their data analysis capabilities by leveraging graph database functionality integrated with PostgreSQL. By incorporating Apache AGE, Zepta Technology can handle complex connected graph data, perform graph query modeling, and access graph-specific operations within the PostgreSQL environment. This integration allows Zepta Technology to delve into scenarios where graph data structures are crucial, such as relationship analysis, social networks, recommendation systems, and knowledge graphs. Apache AGE enables Zepta Technology to seamlessly combine relational and graph data models, empowering users to perform graph data analytics efficiently with minimal effort, making it an ideal choice for organizations seeking cohesive and easy-to-implement multi-model database.

  1. 3H System US based company uses apache age for making different project because its a consulting services.

  2. EXMC canada based flower delivery website that uses apache age for maintaining database of flowers of various variety.

4.Boeing, a renowned company in the aviation and aerospace industry, might use Apache AGE in their platform to enhance their data analysis capabilities, particularly in handling complex data relationships and graph structures

@MironAtHome
Copy link

MironAtHome commented Mar 28, 2024 •
edited

In bioinformatics and genomics research, Apache Age is used to analyze genetic data and intricate biological networks. Apache Age makes it easier to identify genes linked to disease, analyze biological pathways, and uncover the functions of genes by visualizing genetic and protein interactions as graphs. Precision agriculture, tailored medicine, and drug development are all benefited by this use case. With its ability to handle and analyze graph data, Apache Age is a potent tool that spurs innovation in a variety of industries and provides insightful information. Apache Age shows itself to be a flexible solution for utilizing the potential of graph data, regardless of the application—social network analysis, recommendation systems, fraud detection, knowledge graph management, network analysis, or biology. These are the use cases that i know of and am confident Apache Age can do

Could you please refer to anything specific, such as github, article describing specific use case, without revealing what your company doesn't allow you to?
It would really help my case.

@M-Bostone
Copy link

Heey AGE users!
I have been uing AGE for 9 months now and here is what i've seen so far;
I like Age because of its graph database functionality that offers the capability to store query, and analyze graph data directly within an existing PostgreSQL database systems.
AGE allows us to to use the power of graph databases without needing to set up or maintain a separate graph database system by integrating graph functionality seamlessly into the PostgreSQL environment.
AGE's perfomance can not be forgotten as it utilizes the power of PostgreSQL to the fullest, uses the scalable nature of PostgreSQL hence making more efficient to handle more complex graph queries with ease and more efficient.

@Emmanuelkiplangatbett
Copy link

Apache AGE is a handy tool that lets us work with graph data directly in PostgreSQL. Graph data represents connections between things, like social networks or networks of computer systems.

Instead of setting up a separate database just for graph data, Apache AGE lets us add graph features right into our existing PostgreSQL setup. This means we can manage all our data - both relational and graph data - in one place, making it simpler to handle.

Plus, because it's built on PostgreSQL, we can take advantage of all the tools and extensions available for PostgreSQL, like security features and performance optimization tools.

In short, Apache AGE helps us manage and analyze our graph data more efficiently without needing to complicate our technology setup.

@M-Bostone
Copy link

M-Bostone commented Apr 3, 2024 via email

@markgomer
Copy link
Contributor Author

@M-Bostone @Emmanuelkiplangatbett, are you using Apache AGE in any project that you can share with us here? We would love to know more about it!

@M-Bostone
Copy link

M-Bostone commented Apr 3, 2024 via email

@noahxzhu
Copy link

noahxzhu commented Apr 6, 2024

I'm recently working on a project with LlamaIndex, there're several kinds of storage can integrate with different kinds of databases, like mongodb for doc store, index store, postgres with pgvector extension for vector store, and neo4j, falkordb, neptune, nebula for graph store.

Currently I'm using neo4j community version which has some restrictions like no multiple databases support in one instance, the data migration is difficult. Nebula looks good, I do considered to move to it.

I'm a pg fan, after some google search, I found this apache age, it looks incredibly awesome, so I want to contribute the graph store integration to LlamaIndex. That's the beginning.

Thanks for your work for this project.

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@noahxzhu
Copy link

noahxzhu commented Apr 6, 2024

Hi @M-Bostone

For sure, I'm encountering some problem with age that the behavior is little different with the Neo4j in some specific case, I've already created a issue for this #1722

I'd appreciate some help. Also I want to contribute to this project, but I need to learn more in advance.

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@noahxzhu
Copy link

noahxzhu commented Apr 6, 2024

have you tried escaping the single quotes by doubling the single quotes like; MERGE (n2:Entity {id: 'It''s better for technology companies to be run by product people'}) which will tell your parser to treat the single quote in your string as part of the string
…
On Sat, Apr 6, 2024 at 3:10 PM Noah Zhu @.> wrote: Hi @M-Bostone https://github.com/M-Bostone For sure, I'm encountering some problem with age that the behavior is little different with the Neo4j in some specific case, I've already created a issue for this #1722 <#1722> I'd appreciate some help. Also I want to contribute to this project, but I need to learn more in advance. — Reply to this email directly, view it on GitHub <#1705 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE . You are receiving this because you were mentioned.Message ID: @.>
-- men

I'm using the docker image which is provided on the github README.md for some testing.

I do can put the completed query and add like backslash, use double quotes to wrap the string if the string contains single quotes or use single quotes to wrap the string if the string contains double quotes. But most of my time, I need to parameterize the queries that looks buggy, especially with the python driver.

And for now the problem from my side is all the strings are from some documents, that means I need to process the string in advance to put it as the parameters into the query, but that doesn't work for now. There's no this kinds of problems when I use the Neo4j.

Also I can't create a vertex with a label contain single quotes which works in Neo4j.

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@noahxzhu
Copy link

noahxzhu commented Apr 6, 2024

okay let me check your documents and see where the problem is do you mind if i see your work
…
On Sat, Apr 6, 2024 at 3:56 PM Noah Zhu @.> wrote: have you tried escaping the single quotes by doubling the single quotes like; MERGE (n2:Entity {id: 'It''s better for technology companies to be run by product people'}) which will tell your parser to treat the single quote in your string as part of the string … <#m_6476712402078747728_> On Sat, Apr 6, 2024 at 3:10 PM Noah Zhu @.> wrote: Hi @M-Bostone https://github.com/M-Bostone https://github.com/M-Bostone https://github.com/M-Bostone For sure, I'm encountering some problem with age that the behavior is little different with the Neo4j in some specific case, I've already created a issue for this #1722 <#1722> <#1722 <#1722>> I'd appreciate some help. Also I want to contribute to this project, but I need to learn more in advance. — Reply to this email directly, view it on GitHub <#1705 (comment) <#1705 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE . You are receiving this because you were mentioned.Message ID: @.> -- men I'm using the docker image which is provided on the github README.md for some testing. I do can put the completed query and add like backslash, use double quotes to wrap the string if the string contains single quotes or use single quotes to wrap the string if the string contains double quotes. But most of my time, I need to parameterize the queries that looks buggy, especially with the python driver. And for now the problem from my side is all the strings are from some documents, that means I need to process the string in advance to put it as the parameters into the query, but that doesn't work for now. There's no this kinds of problems when I use the Neo4j. Also I can't create a vertex with a label contain single quotes which works in Neo4j. — Reply to this email directly, view it on GitHub <#1705 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 . You are receiving this because you were mentioned.Message ID: @.**>

I'm working with the llamaindex, use LLM(OpenAI Model) to extract some contents to triplets like [subj, rel, obj], the contents are from any pdfs or txts, so that means I can't fully control what the LLM extract for me, maybe the subj contains some special chars, or rel and obj as well.

this is my python code

    def upsert_triplet(self, subj: str, rel: str, obj: str) -> None:
        """Add triplet."""
        rel = rel.replace(" ", "_").upper()
        prepared_statement = f"""
            MERGE (n1:`{self.node_label}` {{id: %s}})
            MERGE (n2:`{self.node_label}` {{id: %s}})
            MERGE (n1)-[:`{rel}`]->(n2)
        """

        self._age.execCypher(prepared_statement, params=(subj, obj))
        self._age.commit()

the extracted data could be:

["I", "Learned", "It's better for technology companies to be run by product people"]


or maybe the subj or rel also includes single quotes or double quotes
["I", "include any '", "It's better for technology companies to be run by product people"]

There's no problem with this kind of data in Neo4j.

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@noahxzhu
Copy link

noahxzhu commented Apr 6, 2024 •
edited

sorry it took me this long to reply but i've got one of the slowest machines there is but try this: def upsert_triplet(self, subj: str, rel: str, obj: str) -> None: """Add triplet.""" rel = rel.replace(" ", "").upper() prepared_statement = f""" MERGE (n1:{self.node_label} {{id: $subj}}) MERGE (n2:{self.node_label} {{id: $obj}}) MERGE (n1)-[:{rel}]->(n2) """ self.age.execCypher(prepared_statement, params={"subj": subj, "obj": obj}) self.age.commit() try this On Sat, Apr 6, 2024 at 4:44 PM Moturi Bostone @.> wrote:
…
let me take a look On Sat, Apr 6, 2024 at 4:35 PM Noah Zhu @.> wrote: > okay let me check your documents and see where the problem is do you mind > if i see your work > … <#m_7435983581932783017_m_8141234859897622336> > On Sat, Apr 6, 2024 at 3:56 PM Noah Zhu @.*> wrote: have you tried > escaping the single quotes by doubling the single quotes like; MERGE > (n2:Entity {id: 'It''s better for technology companies to be run by product > people'}) which will tell your parser to treat the single quote in your > string as part of the string … <#m_6476712402078747728> On Sat, Apr 6, > 2024 at 3:10 PM Noah Zhu @.> wrote: Hi @M-Bostone > https://github.com/M-Bostone https://github.com/M-Bostone > https://github.com/M-Bostone https://github.com/M-Bostone > https://github.com/M-Bostone https://github.com/M-Bostone > https://github.com/M-Bostone For sure, I'm encountering some problem with > age that the behavior is little different with the Neo4j in some specific > case, I've already created a issue for this #1722 > <#1722> <#1722 > <#1722>> <#1722 > <#1722> <#1722 > <#1722>>> I'd appreciate some help. > Also I want to contribute to this project, but I need to learn more in > advance. — Reply to this email directly, view it on GitHub <#1705 > <#1705> (comment) <#1705 (comment) > <#1705 (comment)>>>, or > unsubscribe > https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE > https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE > https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE > https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE > . You are receiving this because you were mentioned.Message ID: @.> -- men > I'm using the docker image which is provided on the github README.md for > some testing. I do can put the completed query and add like backslash, use > double quotes to wrap the string if the string contains single quotes or > use single quotes to wrap the string if the string contains double quotes. > But most of my time, I need to parameterize the queries that looks buggy, > especially with the python driver. And for now the problem from my side is > all the strings are from some documents, that means I need to process the > string in advance to put it as the parameters into the query, but that > doesn't work for now. There's no this kinds of problems when I use the > Neo4j. Also I can't create a vertex with a label contain single quotes > which works in Neo4j. — Reply to this email directly, view it on GitHub > <#1705 (comment) > <#1705 (comment)>>, or > unsubscribe > https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 > https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 > . You are receiving this because you were mentioned.Message ID: @.***> > > I'm working with the llamaindex, use LLM(OpenAI Model) to extract some > contents to triplets like [subj, rel, obj], the contents are from any pdfs > or txts, so that means I can't fully control what the LLM extract for me, > maybe the subj contains some special chars, or rel and obj as well. > > this is my python code > > def upsert_triplet(self, subj: str, rel: str, obj: str) -> None: > """Add triplet.""" > rel = rel.replace(" ", "").upper() > prepared_statement = f""" > MERGE (n1:{self.node_label} {{id: %s}}) > MERGE (n2:{self.node_label} {{id: %s}}) > MERGE (n1)-[:{rel}]->(n2) > """ > > self._age.execCypher(prepared_statement, params=(subj, obj)) > self._age.commit() > > the extracted data could be: > > ["I", "Learned", "It's better for technology companies to be run by product people"] > > > or maybe the subj or rel also includes single quotes or double quotes > ["I", "include any '", "It's better for technology companies to be run by product people"] > > There's no problem with this kind of data in Neo4j. > > — > Reply to this email directly, view it on GitHub > <#1705 (comment)>, or > unsubscribe > https://github.com/notifications/unsubscribe-auth/BFO2YCAJJG2OD53YGL57NX3Y372ZLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA4DGOBRG4 > . > You are receiving this because you were mentioned.Message ID: > @.***> >

It doesn't work for me

def upsert_triplet_test(subj: str, rel: str, obj: str) -> None:
    prepared_statement = f"""
        MERGE (n1:`Entity` {{id: $subj}})
        MERGE (n2:`Entity` {{id: $obj}})
        MERGE (n1)-[:`{rel.replace(" ", "_").upper()}`]->(n2)
    """

    ag.execCypher(prepared_statement, params={"subj": subj, "obj": obj})
    ag.commit()


upsert_triplet_test(
    "I",
    "Learned aa",
    "s 's better for technology companies to be run by product people",
)

I got this:

Traceback (most recent call last):
  File "/Users/noah/Projects/Python/age-test/.venv/lib/python3.11/site-packages/age/age.py", line 138, in execCypher
    cursor.execute(stmt)
psycopg2.errors.UndefinedParameter: parameters argument is missing from cypher() function call
LINE 1: SELECT * from cypher(NULL,NULL) as (v agtype);
                                        ^


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/noah/Projects/Python/age-test/main.py", line 142, in <module>
    upsert_triplet_test(
  File "/Users/noah/Projects/Python/age-test/main.py", line 138, in upsert_triplet_test
    ag.execCypher(prepared_statement, cols=[], params={"subj": subj, "obj": obj})
  File "/Users/noah/Projects/Python/age-test/.venv/lib/python3.11/site-packages/age/age.py", line 198, in execCypher
    return execCypher(self.connection, self.graphName, cypherStmt, cols=cols, params=params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noah/Projects/Python/age-test/.venv/lib/python3.11/site-packages/age/age.py", line 145, in execCypher
    raise SqlExecutionError("Execution ERR[" + str(cause) +"](" + stmt +")", cause)
age.exceptions.SqlExecutionError: ('Execution ERR[parameters argument is missing from cypher() function call\nLINE 1: SELECT * from cypher(NULL,NULL) as (v agtype);\n                                        ^\n](SELECT * from cypher(NULL,NULL) as (v agtype);)', UndefinedParameter('parameters argument is missing from cypher() function call\nLINE 1: SELECT * from cypher(NULL,NULL) as (v agtype);\n                                        ^\n'))

I can find the document mentioned that we can use the $ sign for the prepared statement, but it's for postgres procedure. From my understand of the python driver that it is not using procedure.

https://age.apache.org/age-manual/master/advanced/prepared_statements.html

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@M-Bostone
Copy link

M-Bostone commented Apr 6, 2024 via email

@M-Bostone
Copy link

M-Bostone commented Apr 7, 2024 via email

@noahxzhu
Copy link

noahxzhu commented Apr 8, 2024 •
edited

did you manage to solve it On Sat, Apr 6, 2024 at 11:33 PM Moturi Bostone @.> wrote:
…
are you trying to compare between Apache AGE and neo4j,for you to use Apache AGE, PostgreSQL has to be your primary SQL database On Sat, Apr 6, 2024 at 9:45 PM Moturi Bostone @.> wrote: > make sure that the argument parameters are correctly included in Apache > AGE when calling the cypher function > > On Sat, Apr 6, 2024 at 9:43 PM Moturi Bostone @.> > wrote: > >> i have encountered this once and the problem turned out to be a missing >> parameters in the cypher function call >> >> On Sat, Apr 6, 2024 at 9:29 PM Noah Zhu @.> >> wrote: >> >>> sorry it took me this long to reply but i've got one of the slowest >>> machines there is but try this: def upsert_triplet(self, subj: str, rel: >>> str, obj: str) -> None: """Add triplet.""" rel = rel.replace(" ", " >>> >>> ").upper() prepared_statement = f""" MERGE (n1:{self.node_label} {{id: >>> $subj}}) MERGE (n2:{self.node_label} {{id: $obj}}) MERGE >>> (n1)-[:{rel}]->(n2) """ self.age.execCypher(prepared_statement, >>> params={"subj": subj, "obj": obj}) self.age.commit() try this On Sat, Apr >>> 6, 2024 at 4:44 PM Moturi Bostone @.> wrote: … >>> <#m_5946786648096769430_m_463412608644053527_m_821820109361710640_m_-5735751191410968667_> >>> let me take a look On Sat, Apr 6, 2024 at 4:35 PM Noah Zhu @.> wrote: > >>> okay let me check your documents and see where the problem is do you mind > >>> if i see your work > … <#m_7435983581932783017_m_8141234859897622336> > On >>> Sat, Apr 6, 2024 at 3:56 PM Noah Zhu @.> wrote: have you tried > escaping >>> the single quotes by doubling the single quotes like; MERGE > (n2:Entity >>> {id: 'It''s better for technology companies to be run by product > >>> people'}) which will tell your parser to treat the single quote in your > >>> string as part of the string … <#m_6476712402078747728> On Sat, Apr 6, > >>> 2024 at 3:10 PM Noah Zhu @.> wrote: Hi @M-Bostone >>> https://github.com/M-Bostone > https://github.com/M-Bostone >>> https://github.com/M-Bostone https://github.com/M-Bostone >>> https://github.com/M-Bostone > https://github.com/M-Bostone >>> https://github.com/M-Bostone https://github.com/M-Bostone >>> https://github.com/M-Bostone > https://github.com/M-Bostone >>> https://github.com/M-Bostone https://github.com/M-Bostone >>> https://github.com/M-Bostone > https://github.com/M-Bostone >>> https://github.com/M-Bostone For sure, I'm encountering some problem with >>> > age that the behavior is little different with the Neo4j in some specific >>> > case, I've already created a issue for this #1722 >>> <#1722> > <#1722 >>> <#1722>> <#1722 >>> <#1722> > <#1722 >>> <#1722>>> <#1722 >>> <#1722> > <#1722 >>> <#1722>> <#1722 >>> <#1722> > <#1722 >>> <#1722>>>> I'd appreciate some help. > >>> Also I want to contribute to this project, but I need to learn more in > >>> advance. — Reply to this email directly, view it on GitHub <#1705 >>> <#1705> > <#1705 >>> <#1705>> (comment) <#1705 >>> <#1705> (comment) > <#1705 (comment) >>> <#1705 (comment)>>>>, or >>> > unsubscribe > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> https://github.com/notifications/unsubscribe-auth/BFO2YCEMHQXK6WMEXQ25LTTY37Q2HAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3DKNBTGE >>> > . You are receiving this because you were mentioned.Message ID: @.> -- >>> men > I'm using the docker image which is provided on the github README.md >>> for > some testing. I do can put the completed query and add like >>> backslash, use > double quotes to wrap the string if the string contains >>> single quotes or > use single quotes to wrap the string if the string >>> contains double quotes. > But most of my time, I need to parameterize the >>> queries that looks buggy, > especially with the python driver. And for now >>> the problem from my side is > all the strings are from some documents, that >>> means I need to process the > string in advance to put it as the parameters >>> into the query, but that > doesn't work for now. There's no this kinds of >>> problems when I use the > Neo4j. Also I can't create a vertex with a label >>> contain single quotes > which works in Neo4j. — Reply to this email >>> directly, view it on GitHub > <#1705 >>> <#1705> (comment) > <#1705 (comment) >>> <#1705 (comment)>>>, or > >>> unsubscribe > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 >>> https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 >>> > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 >>> https://github.com/notifications/unsubscribe-auth/BFO2YCCQ7PVYHFUFGPDN44LY37WGLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA3TIOBZG4 >>> > . You are receiving this because you were mentioned.Message ID: @.> > >>> > I'm working with the llamaindex, use LLM(OpenAI Model) to extract some > >>> contents to triplets like [subj, rel, obj], the contents are from any pdfs >>> > or txts, so that means I can't fully control what the LLM extract for me, >>> > maybe the subj contains some special chars, or rel and obj as well. > > >>> this is my python code > > def upsert_triplet(self, subj: str, rel: str, >>> obj: str) -> None: > """Add triplet.""" > rel = rel.replace(" ", "").upper() >>> > prepared_statement = f""" > MERGE (n1:{self.node_label} {{id: %s}}) >>> > MERGE (n2:{self.node_label} {{id: %s}}) > MERGE (n1)-[:{rel}]->(n2) >>> > """ > > self._age.execCypher(prepared_statement, params=(subj, obj)) > >>> self._age.commit() > > the extracted data could be: > > ["I", "Learned", >>> "It's better for technology companies to be run by product people"] > > > >>> or maybe the subj or rel also includes single quotes or double quotes > >>> ["I", "include any '", "It's better for technology companies to be run by >>> product people"] > > There's no problem with this kind of data in Neo4j. > >>> > — > Reply to this email directly, view it on GitHub > <#1705 >>> (comment) >>> <#1705 (comment)>>, >>> or > unsubscribe > >>> https://github.com/notifications/unsubscribe-auth/BFO2YCAJJG2OD53YGL57NX3Y372ZLAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGA4DGOBRG4 >>> > . > You are receiving this because you were mentioned.Message ID: > >>> @.> > >>> >>> It doesn't work for me >>> >>> def upsert_triplet_test(subj: str, rel: str, obj: str) -> None: >>> prepared_statement = f""" >>> MERGE (n1:Entity {{id: $subj}}) >>> MERGE (n2:Entity {{id: $obj}}) >>> MERGE (n1)-[:{rel.replace(" ", "_").upper()}]->(n2) >>> """ >>> >>> ag.execCypher(prepared_statement, params={"subj": subj, "obj": obj}) >>> ag.commit() >>> >>> >>> upsert_triplet_test( >>> "I", >>> "Learned aa", >>> "s 's better for technology companies to be run by product people", >>> ) >>> >>> I got this: >>> >>> Traceback (most recent call last): >>> File "/Users/noah/Projects/Python/age-test/.venv/lib/python3.11/site-packages/age/age.py", line 138, in execCypher >>> cursor.execute(stmt) >>> psycopg2.errors.UndefinedParameter: parameters argument is missing from cypher() function call >>> LINE 1: SELECT * from cypher(NULL,NULL) as (v agtype); >>> ^ >>> >>> >>> During handling of the above exception, another exception occurred: >>> >>> Traceback (most recent call last): >>> File "/Users/noah/Projects/Python/age-test/main.py", line 142, in >>> upsert_triplet_test( >>> File "/Users/noah/Projects/Python/age-test/main.py", line 138, in upsert_triplet_test >>> ag.execCypher(prepared_statement, cols=[], params={"subj": subj, "obj": obj}) >>> File "/Users/noah/Projects/Python/age-test/.venv/lib/python3.11/site-packages/age/age.py", line 198, in execCypher >>> return execCypher(self.connection, self.graphName, cypherStmt, cols=cols, params=params) >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> File "/Users/noah/Projects/Python/age-test/.venv/lib/python3.11/site-packages/age/age.py", line 145, in execCypher >>> raise SqlExecutionError("Execution ERR[" + str(cause) +"](" + stmt +")", cause) >>> age.exceptions.SqlExecutionError: ('Execution ERR[parameters argument is missing from cypher() function call\nLINE 1: SELECT * from cypher(NULL,NULL) as (v agtype);\n ^\n](SELECT * from cypher(NULL,NULL) as (v agtype);)', UndefinedParameter('parameters argument is missing from cypher() function call\nLINE 1: SELECT * from cypher(NULL,NULL) as (v agtype);\n ^\n')) >>> >>> >>> I can find the document mentioned that we can use the $ sign for the >>> prepared statement, but it's for postgres procedure. From my understand of >>> the python driver that it is not procedure. >>> >>> >>> https://age.apache.org/age-manual/master/advanced/prepared_statements.html >>> >>> — >>> Reply to this email directly, view it on GitHub >>> <#1705 (comment)>, >>> or unsubscribe >>> https://github.com/notifications/unsubscribe-auth/BFO2YCFZOWPG27XHNFOIDUTY4A5JRAVCNFSM6AAAAABE4GDGHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGE3DCMJTG4 >>> . >>> You are receiving this because you were mentioned.Message ID: >>> @.**> >>> >>

not yet, this solution is not faisable for me due to I'm going to contribute to this to a open source project. And I really don't understand what your mean, I went through all the documents, didn't see related descriptions.

@M-Bostone
Copy link

okay, what is your primary database

@noahxzhu
Copy link

noahxzhu commented Apr 8, 2024

okay, what is your primary database

I'm working on a project with LlamaIndex for RAG, in LlamaIndex we can integrate a graph store with Neo4j, Nebula etc. Current I'm using the Neo4j community edition, but there're some restrictions with it, so I'm seeking an alternative for it. After some investigations, I found this age extension PG, we can use this as a graph store, so I want to move to this, but current LlamaIndex doesn't support it, so I want to contribute this to LlamaIndex. I've already implemented this integration, before I create the pull request, I need to do some tests for it, then I encountered this problems(like the single quote problem which doesn't exist in Neo4j) with age during the testing.

@stdweird
Copy link

@noahxzhu thanks for looking into it. i would also like to settle on one db tenchnology for rag. also not sure you are aware of langchain-ai/langchain#20582

@noahxzhu
Copy link

@stdweird Cool! I'm using both langchain and llamaindex for my project, just replaced the rag part with llamaindex. Thank you for your link, I'll take a look into that langchain PR, would be helpful for me.

@markgomer
Copy link
Contributor Author

markgomer commented Apr 25, 2024 •
edited

@noahxzhu thanks for looking into it. i would also like to settle on one db tenchnology for rag. also not sure you are aware of langchain-ai/langchain#20582

This is awesome. Thank you for sharing @stdweird

@noahxzhu
Copy link

@noahxzhu thanks for looking into it. i would also like to settle on one db tenchnology for rag. also not sure you are aware of langchain-ai/langchain#20582

@stdweird I checked the source code, and this implementation has made a tricky workaround: it removed all the special characters from the label using re.compile("[^0-9a-zA-Z]+"). Yes, that will work, but I think AGE should support special characters in labels. @markgomer, at least I think the python driver is buggy for this kind of special cases.

@stdweird
Copy link

@noahxzhu ofcourse it should support all characters in the labels. it is up to you to either fix this first and then make the llamaindex PR, or make a PR with this known limitation, and make another PR to fix it in the (near) future. i can't estimate how much owrk and/or how long this might take.

personally i want to use AGE in the future, but i can continue my tests with neo4j

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests