opal-fetcher-postgres: opal-server data update verify

[rahulaga01]

I am running OPAL server with backend PostgreSQL and also with PostgreSQL fetcher using the following docker-compose file.
https://github.com/permitio/opal-fetcher-postgres/blob/master/docker-compose.yml

From the logs every thing is working fine, but I have some queries about OPAL with PostgreSQL.
If I make a change in PostgreSQL (tables or any thing) How can we verify the changes in OPAL server?

In this Docker-Compose file
OPAL_BROADCAST_URI=postgres://postgres:postgres@broadcast_channel:5432/postgres is Postgres, How can we verify it using Postgres?

[orweis]

Hi @rahulaga01 :)

If I make a change in PostgreSQL (tables or any thing) How can we verify the changes in OPAL server?

You need to trigger the update event to OPAL yourself - from the code that made the change, from a webhook from the DB or a DAL, or via polling on it.

is Postgres, How can we verify it using Postgres?

This is the broadcast channel (for inter-server communication) - if it doesn't work you would usually get an error, also random messages sent won't reach their clients , and you can also listen on the Listen/Notify itself and see if messages are going.
But mostly it just works if you were able to connect.

[rahulaga01]

Thank you for your reply.

Using the Docker-compose file, PostgreSQL create some tables with data.
When we run Postgres fetcher first time it will get the data or we need to trigger webhook or polling?

Can you please suggest best way to trigger opal server, when we made some changes in PostgreSQL (API endpoint or some other way, if API, please share API endpoint).

When we hit this url: http://localhost:8181/v1/data, it shows all the data that is coming from the git repo in data.json file,
But how can we verify the data that is coming from postgres fetcher?

[orweis]

Please read the docs :)
On the first run OPAL-client will be instructed by the server to load all the data defined in OPAL_DATA_CONFIG_SOURCES

Can you please suggest best way to trigger opal server, when we made some changes in PostgreSQL

https://docs.opal.ac/tutorials/trigger_data_updates

But how can we verify the data that is coming from postgres fetcher?

Load unique data, or load it into a seperate path on OPA

[rahulaga01]

@orweis

is Postgres, How can we verify it using Postgres?

This is the broadcast channel (for inter-server communication) - if it doesn't work you would usually get an error, also random messages sent won't reach their clients , and you can also listen on the Listen/Notify itself and see if messages are going.
But mostly it just works if you were able to connect.

We deploy the opal-server with the redis server, opal-server is running fine but there is no data inside the redis.

One more thing, it seems like that the opal-server is not storing anything in redis or postgres, right? if yes, then it means that our opal-server will also not store any policy inside redis or postgres ?

Is this the default behaviour of opal-server or are we missing any configuration?

[orweis]

How can we verify it using Postgres?

See reply above

it seems like that the opal-server is not storing anything in redis or postgres

Correct they are just used for the backbone pub/sub between the OPAL server (aka broadcaster)