Egeria Workshop topics (Bucharest, 24-26 April)

[lpalashevski]

Starting new discussion to gather ideas/topics for the next workshop taking place in Bucharest from 24th to 26th April 2023.

Please share your proposals so we can gather and propose agenda for the three days.

[davidradl]

I suggest:

• an update on cloud native workgroup including discussions on the next steps
• an update on the consolidated type script UI, including documentation, views and status for v4.

[sarbull]

not much has been changed since last workshop, i suggest to focus on documentation and a demo server(with or without api-mocks)

also adding:

• independent module documentation using GitHub pages for each UI component(i presented last time something in this direction, currently work in backlog for happi-graph)
• video documentation on how to start things (UI)
• demo server that should always be maintained, maybe ask on LFAI group for resources under DigitalOcean(i saw they have sponsorship)

[planetf1]

I suggest that if we need any more than a simple start of a process, we consider a specific helm chart for any demo as it means we can then move/maintain/redeploy in the easiest way possible.

One concern of any hosted service is maintainance -- but the other key aspect is security, so we should be certain we need/want this.. and if we do, do we want to go further than just the ui

[mandy-chessell]

I would like to talk about:

• building a glossary using a controlled workflow process
• new Atlas integration connector
• Using XTDB with postgreSQL as the repository
• Dynamic configuration of the integration daemon
• Secrets connector

[planetf1]

I presume you mean postgres for the transaction log & document store? We should also consider how the index store is handled. Either this needs to be persistent. OR we let it rebuild each time. If we go for rebuild that node should not have requests routed until the index rebuild is complete (I think?)

[mandy-chessell]

We have been trying to get as many of the stores used by XTDB onto postgres as possible

[sarbull]

Removing sonatype from code quality checks and replace it with alternatives

• UI: eslint
• Backend: ?

[planetf1]

• lift allows tools to be disabled, or customized. We did try this for eslint8, but hit issues - we could followup.
• Lift only runs within PR. If we can simply add additional tests directly into the build - or via a custom action, that may add more clarity

I presume you are referring to the UI -- but perhaps we could discuss this more broadly.

[planetf1]

My topics

• Review from the cloud native workgroup
• Release 4 - followups
  • deprecated function
  • XTDB connector
  • JDBC connector
  • egeria samples/dojo (ie egeria-samples-api ..., file repo connector @davidradl )
  • event schema, strimzi, lineage @juergenhemelt
• How do we better minimize ongoing maintainance
• repo owner assignment/responsibilities
• consolidation/refactoring/common build processes/automation

[planetf1]

The last two points are about scalability - we have other 30 repos now, so each needs management or automation, or will languish and get out of date/insecure.

[mandy-chessell]

We need to discuss the javadoc warnings that are coming out of the gradle build and what we are going to do about them.

[mandy-chessell]

Also a date for 4.1

[ColesnicencuM]

Topics :
• possibility for configuration improvement: add a new way to configure Egeria besides API calls; for example have a config file for simple basic settings and use API calls for more complex settings;
• develop and improve Egeria deployment using containers; right now there are external dependencies which make container deployment in production not feasible;
• add new connectors for graph db backends in the future besides JanusGraph and XTDB; right now JanusGraph development is not very active and XTDB may not be production ready; a discussion about possible better alternatives may help.

[lpalashevski]

I am interested in following topics:

• Regular component owners and component status review
• Cloud native work-group update: POC plan and next steps
• UI platforms consolidation: open discussion, agree on concrete design and work plan (as much as possible possible)
• ING Data Lineage Import Preview (demo)
• Overview on the new Lineage Types: Solution Ports and Wires vs Information Supply Chains
• (?) ING Data Mesh work-group / Egeria open discussion (if we can get someone from ING work-group on Wednesday)

[lpalashevski]

Consolidated agenda (for the record)

Day 1 Egeria workshop

Morning session:

9:00-9:30 Arrive at the location and coffee.
9:30
• Integration with third-party technologies
o new Atlas integration connector
o Secrets connector

• ING Data Lineage Import Preview (demo)

12:00 - 13:00 Lunch
Afternoon session:

• UI platforms consolidation: open discussion, agree on concrete design and work plan (as much as possible possible)

• Overview on the new Lineage Types: Solution Ports and Wires vs Information Supply Chains
• Using XTDB with postgreSQL as the repository

Day 2 Egeria workshop

Morning session:

9:00-9:15 Arrive at the location and coffee.
9:15
• Cloud native: an update on cloud native workgroup including discussions on the next steps.
• Addressing feedback from ING product team:
o possibility for configuration improvement: add a new way to configure Egeria besides API calls; for example, have a config file for simple basic settings and use API calls for more complex settings.
o develop and improve Egeria deployment using containers; right now, there are external dependencies which make container deployment in production not feasible.
o add new connectors for graph db backends in the future besides JanusGraph and XTDB; right now, JanusGraph development is not very active and XTDB may not be production ready; a discussion about possible better alternatives may help.

12:00 - 13:00 Lunch
Afternoon session:

• Pipelines and Code quality checks

• UI JavaScript/TypeScript (eslint)
• Use of Sonatype Lift scans
• Javadoc build warnings - what should we do, what are our standards?

• Release process activities

• Release 4 - follow-ups
• deprecated function
• XTDB connector
• JDBC connector
• egeria samples/dojo (ie egeria-samples-api ..., file repo connector @davidradl )
• event schema, strimzi, lineage @juergenhemelt

• How do we better minimize ongoing maintenance:
o repo owner assignment/responsibilities
o consolidation/refactoring/common build processes/automation

Day 3 Egeria workshop

Morning session:

9:00-9:15 Arrive at the location and coffee.
9:15

• Project health: Regular component owners and component status review
• UI platforms consolidation: open discussion, agree on concrete design and work plan (as much as possible possible)

12:00 - 12:30 Lunch
Afternoon session:

• UI topics
o an update on the consolidated type script UI, including documentation, views and status for v4.
o Independent module documentation using GitHub pages for each UI component (presented last time something in this direction, currently work in backlog for happi-graph)
o video documentation on how to start things (UI)
o demo server that should always be maintained, maybe ask on LFAI group for resources under DigitalOcean (they have sponsorship)

• Live coding session

Optional/if time permits
• Building a glossary using a controlled workflow process
• Dynamic configuration of the integration daemon - if we need to understand more.
• (?) ING Data Products workgroup / Egeria open discussion (if we can get someone from ING workgroup on Wednesday)