Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correlate Database audit log entries with the asset id of the data link used for the connection, if available #9869

Open
3 tasks
radeusgd opened this issue May 7, 2024 · 0 comments
Assignees
Labels
-libs Libraries: New libraries to be implemented l-cloud-integration Enso Cloud integration work l-db-read Libraries: database reader l-postgresql

Comments

@radeusgd
Copy link
Member

radeusgd commented May 7, 2024

If a database connection was opened through a data link, it is audited. If the data link was from the Enso Cloud, we want to add its assetId to the metadata of each log entry.

For data links that are not in the Cloud (local file system etc.) we cannot have a stable identifier like the assetId, so we don't add anything similar.

  • The data link instance fetched from the Cloud needs to be aware of its asset id - modify the data link read logic to be able to supply optional id.
  • That ID needs to be passed to the CloudAuditedConnection, probably through additional JDBC properties.
  • Tests need to be written
    • The challenge is: our Cloud mock does not support reading files/datalinks, but we wanted to rely on the mock for testing the logs. Some solution needs to be picked:
      1. Add basic datalink support to the mock, and run the tests on there.
      2. Run the tests on the real cloud deployment (in local testing the stability seemed okayish - but hard to tell how it will be on CI).
@radeusgd radeusgd self-assigned this May 7, 2024
@radeusgd radeusgd added -libs Libraries: New libraries to be implemented l-postgresql l-db-read Libraries: database reader l-cloud-integration Enso Cloud integration work labels May 7, 2024
radeusgd added a commit that referenced this issue May 7, 2024
mergify bot pushed a commit that referenced this issue May 11, 2024
- Closes #9599
- Implemented API for sending audit logs to the cloud on a background thread.
- If the Postgres connection is opened through a datalink, its internal JDBC connection is replaced by a wrapper that reports executed queries to the audit log.
- Also introduces `EnsoMeta` - a helper Java class that can be used in our helper libraries to access Enso types.
- I have replaced the common pattern scattered throughout the codebase with calls to this 'library' to avoid repetitive code.
- Refactored `Table.display` to share code between in-memory and DB - it was needed as the function stopped working for `DB_Table` after adding making the `Table` constructor `private`.
- Clearer error when reading a SQLite database from a remote file (tells the user to download it first).
- Follow up - correlate asset id of the data link:
#9869
- Follow up - include project name (once bug is fixed):
#9875
- Some problems/improvements of the audit log:
- The audit log system is not yet ready for high throughput of logs
#9870
- The logs may be lost if `System.exit` is used
#9871
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
-libs Libraries: New libraries to be implemented l-cloud-integration Enso Cloud integration work l-db-read Libraries: database reader l-postgresql
Projects
Status: 📤 Backlog
Development

No branches or pull requests

1 participant