Prototype proposal for 2a

[davidradl]

@lpalashevski @planetf1 (@juergenhemelt I cant see Taylans id to add him here)

Relating to https://wiki.lfaidata.foundation/pages/viewpage.action?pageId=70648184 item 2a. I have been thinking about the prototype:

Phase 1:

• Make changes in the main Egeria repo.
• Create a new application similar to server chassis, called server-standalone (fyi I create the ui chassis so have some familiarity with this approach)
  • Change the grade file to not being in any admin projects. So that we have no admin endpoints picked up by spring.
  • Duplicate the code in https://github.com/odpi/egeria/blob/main/open-metadata-implementation/admin-services/admin-services-server/src/main/java/org/odpi/openmetadata/adminservices/server/OMAGServerOperationalServices.java into a new project maybe called standaloneServerLifecycle. This new project would pick up the config from the file system, based on an environment variable. It will assume there is a data folder with a server and platforms folders already there.
  • Need to ensure that the server is created after what ever initialisation has occurred when the process starts
  • Decide if we want to separate the read only storage from the cohort registry storage writeable.
  • We could assume that the config is encrypted (or plain text) , or pass the config store connector name through to use in an environment variable . I think for a prototype using the encrypted default would make sense for now - as it is a default. We can change as required.
  • Assumptions
    • No new rest calls
    • No platform changes
    • No change to tenancy

Phase 2:
- Investigate removing the platform services and feeding through the platform information using the file system or environment variables.

Phase 3:
- investigate / agree whether config maps or environment variables are better ways to pass config and platform information

Phase 4:
- add shutdown and status rest endpoints. Hopefully we understand more about what the status will / should mean after having done phase 1 and 2.
-
Phase 5:
- remove the need for any tenancy logic.

I can have a play with phase 1 early next week, if we think this is a reasonable plan.

[juergenhemelt]

fyi @T4ylan

[juergenhemelt]

@davidradl
I suggest the server configuration to be part of the Spring application properties. So you could use Spring mechanisms to pass the configuration - e. g. environment variables, properties/yaml-file, command line arguments.

[davidradl]

Thanks @juergenhemelt . Yes we have some properties that are passed through to the server chassis from applications.properties. Are you thinking that we would pass down an environment variable with a value of the json file that initially would be an OMAG configuration json? Either we get it from the file system or an environment variable then we can deserialise the json into the configuration object. I think you are saying you prefer the environment variable(s) (application properties) rather than the file system - have I understood correctly.

[davidradl]

I was thinking some more on this. If the state comes in an environment variable, it is not persisted; so we are stateless with respect to the config; which I think is clean. It would seem to me that the metadata collectionId and the the cohort registry are tied, we will need more environment variables to pass the platform keys and the disk location to write the cohort registry to. I think we should do this after phase 1. I will add this as phase 3.

[juergenhemelt]

If we start with an integration connector the cohort registry is not relevant because only metadata server register in a cohort, right?

[davidradl]

If we start with an integration connector the cohort registry is not relevant because only metadata server register in a cohort, right?

yes that is true

[davidradl]

My initial thinking of phase 1 was to prove that we could run with one server; less about how we take in the server config. As is it should work for all server types - with my testing outside of K8s.

[davidradl]

Feedback from initial phase 1 prototyping:

• I created a new 'standalone server' which was identical to the server chassis. Except for :
  • it did not bring in the platform or admin services spring dependancies
  • the OMAGServerPlatform was renamed to OMAGServerStandalone
  • OMAGServerStandalone in method applicationReady:
    - reads a file whose location is in an environment variable. The file is a plain text version of OMAGServerConfig.
    - the configuration is passed to a new method in operationalServices

activateWithSuppliedConfig(String userId, OMAGServerConfig configuration). 

This method does the same as

activateWithSuppliedConfig(String userId, String serverName, OMAGServerConfig configuration)  

but gets the server from the config and does not write back to the store.

The above application starts with no errors.

This shows that with a minimal change we can create a standalone server. #7513. There is some unused admin and platform code. So this could be opimized to have a smaller footprint.

Originally I had thought we could remove most of the admin services, but the admin services server code contains platform , common services, tenancy, server instance and validation logic. To create a smaller footprint, we would need a standalone version of the admin-services server; a spent a small amount of time playing with doing this; I felt there would be a lot of duplication of code - ideally we would refactor to have common modules that are used between the platform case and the standalone case. Or for now we leave this extra unused admin code in the application.

• we suggest we manage the rest calls exposed with a new 'standalone' spring module exposing only what is required for the stand alone.