Skip to content

Commit

Permalink
Merge branch 'main' into pr-205
Browse files Browse the repository at this point in the history
  • Loading branch information
@orishoshan
orishoshan committed May 6, 2024
2 parents 97a7cf1 + 442f647 commit a71fbe8
Show file tree
Hide file tree
Showing 14 changed files with 151 additions and 157 deletions.
18 changes: 9 additions & 9 deletions docs/features/istio/reference.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ spec:

### Helm Chart options

| Key | Description | Default |
|---------------------------------|-----------------------------------------|--------------------------------|
| `istiowatcher.enable` | Enable Istio watcher deployment (beta). | `false` |
| `istiowatcher.image.repository` | Istio watcher image repository. | `otterize` |
| `istiowatcher.image.image` | Istio watcher image. | `network-mapper-istio-watcher` |
| `istiowatcher.image.tag` | Istio watcher image tag. | `latest` |
| `istiowatcher.pullPolicy` | Istio watcher pull policy. | `(none)` |
| `istiowatcher.pullSecrets` | Istio watcher pull secrets. | `(none)` |
| `istiowatcher.resources` | Resources override. | `(none)` |
| Key | Description | Default |
|----------------------------|-----------------------------------------|--------------------------------|
| `istiowatcher.enable` | Enable Istio watcher deployment (beta). | `false` |
| `istiowatcher.repository` | Istio watcher image repository. | `otterize` |
| `istiowatcher.image` | Istio watcher image. | `network-mapper-istio-watcher` |
| `istiowatcher.tag` | Istio watcher image tag. | `(pinned to latest version as of this Helm chart version's publish)` |
| `istiowatcher.pullPolicy` | Istio watcher pull policy. | `(none)` |
| `istiowatcher.pullSecrets` | Istio watcher pull secrets. | `(none)` |
| `istiowatcher.resources` | Resources override. | `(none)` |

View the [Helm chart reference](/reference/configuration/otterize-chart) for all other options
20 changes: 10 additions & 10 deletions docs/features/kafka/reference.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ spec:

### Helm Chart options

| Key | Description | Default |
|---------------------------------|-------------------------------------------------------------|--------------------------------|
| `kafkawatcher.enable` | Enable Kafka watcher deployment (beta). | `false` |
| `kafkawatcher.image.repository` | Kafka watcher image repository. | `otterize` |
| `kafkawatcher.image.image` | Kafka watcher image. | `network-mapper-kafka-watcher` |
| `kafkawatcher.image.tag` | Kafka watcher image tag. | `latest` |
| `kafkawatcher.pullPolicy` | Kafka watcher pull policy. | `(none)` |
| `kafkawatcher.pullSecrets` | Kafka watcher pull secrets. | `(none)` |
| `kafkawatcher.resources` | Resources override. | `(none)` |
| `kafkawatcher.kafkaServers` | Kafka servers to watch, specified as `pod.namespace` items. | `(none)` |
| Key | Description | Default |
|-----------------------------|-------------------------------------------------------------|--------------------------------|
| `kafkawatcher.enable` | Enable Kafka watcher deployment (beta). | `false` |
| `kafkawatcher.repository` | Kafka watcher image repository. | `otterize` |
| `kafkawatcher.image` | Kafka watcher image. | `network-mapper-kafka-watcher` |
| `kafkawatcher.tag` | Kafka watcher image tag. | `(pinned to latest version as of this Helm chart version's publish)` |
| `kafkawatcher.pullPolicy` | Kafka watcher pull policy. | `(none)` |
| `kafkawatcher.pullSecrets` | Kafka watcher pull secrets. | `(none)` |
| `kafkawatcher.resources` | Resources override. | `(none)` |
| `kafkawatcher.kafkaServers` | Kafka servers to watch, specified as `pod.namespace` items. | `(none)` |

View the [Helm chart reference](/reference/configuration/otterize-chart) for all other options
4 changes: 1 addition & 3 deletions docs/features/network-mapping-network-policies/tutorials/k8s-network-policies.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ In this tutorial, we will:
## Prerequisites

### Install Otterize on your cluster
To deploy Otterize, head over to [Otterize Cloud](https://app.otterize.com) and associate a Kubernetes cluster on the [Integrations page](https://app.otterize.com/integrations), and follow the instructions.

We will also need the [Otterize CLI](/overview/installation#install-the-otterize-cli).
To deploy Otterize, head over to [Otterize Cloud](https://app.otterize.com) and create a Kubernetes integration on the [Integrations page](https://app.otterize.com/integrations), and follow the instructions. *Make sure to enable enforcement mode for this tutorial*. If you already have a Kubernetes cluster connected, skip this step.

## Tutorial

Expand Down
1 change: 0 additions & 1 deletion docs/features/postgresql/_category_.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
"label": "PostgreSQL",
"position": 5,
"collapsed": true,
"className": "hideme",
"customProps": {
"image": "/img/icons/postgresql-no-word-mark.svg"
}
Expand Down
20 changes: 15 additions & 5 deletions docs/features/postgresql/index.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,6 @@ export const postgres_tutorials = [
Otterize is able to create just-in-time username-and-password pairs for your service, providing them as a Kubernetes Secret that can be mounted to file or mapped to environment variables, as well as `GRANT`ing access to databases and tables, based on `ClientIntents` ([Intents-Based Access Control](/overview/intent-based-access-control)) declarations.
In addition, Otterize can map the access to your PostgreSQL database, showing you which service is accessing which database, table and which operation it's performing. This can be used to automatically generate the `ClientIntents` declarations.

Unlike other access controls in Otterize, PostgreSQL support is exclusively available when using Otterize Cloud.

### Tutorials

To learn how to use the Intents Operator and Credentials Operator to enforce access using PostgreSQL GRANTs, or map access to your PostgreSQL database, try one of these quickstart tutorials.
Expand All @@ -38,8 +36,20 @@ To learn how to use the Intents Operator and Credentials Operator to enforce acc

Otterize Cloud will create a unique PostgreSQL username-password combination for each service's use, exposed via a Kubernetes Secret. The service will use these credentials to connect to the database. `ClientIntents` will define the access required by that service. As the intents are applied, Otterize Cloud will keep the database's list of users and GRANTs up to date so that the service is able to access it.

1. To get started, your cluster must have Otterize Cloud installed.
2. You’ll need to [integrate](https://app.otterize.com/integrations) your database by providing a connection URL and admin-level credentials to manage permissions in your database.
1. To get started, your cluster must have Otterize deployed.
2. You'll need to create a `PostgreSQLServerConfig` in your cluster, providing a connection URL and admin-level credentials for Otterize to manage permissions in your database. Below is an example `PostgreSQLServerConfig` resource.
```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: PostgreSQLServerConfig
metadata:
name: postgres-tutorial-db # database instance name - should match the target in ClientIntents
spec:
address: <PGHOST:PGPORT> # Your Postgres address
credentials:
username: <PGUSER> # Username Otterize will connect with & configure permissions as
password: <PGPASSWORD> # Password for above username
```

3. Each service can request a username-password Secret to be created, by annotating the Pod with `credentials-operator.otterize.com/user-password-secret-name`. Below is an example of that annotation and passing the generated credentials into a container with environmental variables.

```yaml
Expand Down Expand Up @@ -94,7 +104,7 @@ spec:
service:
name: server
calls:
- name: otterize-tutorial-postgres # Same name as our integration
- name: postgres-tutorial-db # Same name as our PostgresSQLServerConfig metadata.name
type: database
databaseResources:
- databaseName: otterize-tutorial
Expand Down
18 changes: 15 additions & 3 deletions docs/features/postgresql/tutorials/postgres-mapping.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,14 +135,26 @@ If your Cloud SQL instance is handling any requests,
you may now open your [Pub/Sub topic's metrics](https://console.cloud.google.com/cloudpubsub/topic/list) page
and observe how audit log records are being directed to it.

### Apply a PostgreSQLServerConfig in your cluster
To enable Otterize operators to access your database, apply a PostgreSQLServerConfig in your cluster:
```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: PostgreSQLServerConfig
metadata:
name: otterize-tutorial-cloudsql # database instance name - should match the target in ClientIntents
spec:
address: <PGHOST:PGPORT> # Your CloudSQL database address
credentials:
username: <PGUSER> # Username Otterize will connect with & configure permissions as
password: <PGPASSWORD> # Password for above username
```

### Create an Otterize database integration and configure visibility log collection
To configure Otterize Cloud to subscribe and start consuming your Cloud SQL instance's audit logs,
create an Otterize database integration and configure it with your GCP project and Pub/Sub topic:
- Navigate to the [Integrations page](https://app.otterize.com/integrations) on Otterize Cloud and click the <b>+ Add Integration</b> button to create a new integration
- Choose the <b>Database</b> integration type
- Name your integration <b>otterize-tutorial-cloudsql</b> (this name will be used later in this tutorial)
- Follow the form to provide your database information & credentials
- Click <b>Test Connection</b> to ensure that Otterize Cloud is able to access your database instance
- Name your integration <b>otterize-tutorial-cloudsql</b>
- Under <b>Visibility settings</b>, choose to collect visibility logs using a <b>GCP Pub/Sub topic</b>
- Enter your GCP Project ID & Topic name
- Click <b>Test Visibility</b> to ensure that Otterize Cloud is able to subscribe to your Pub/Sub topic
Expand Down
87 changes: 35 additions & 52 deletions docs/features/postgresql/tutorials/postgres.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ The server needs appropriate permissions to access the database. You could use o

In this tutorial, we will:
* Deploy an example cluster
* Make our database accessible to Otterize Cloud
* Connect our cluster and database to Otterize Cloud
* Deploy Otterize in our cluster and give it access to our database instance
* Declare a ClientIntents resource for the server, specifying required access
* See that the required access has been granted

Expand All @@ -53,12 +52,11 @@ Then start your Minikube cluster with Calico, in order to enforce network polici
```shell
minikube start --cpus=4 --memory 4096 --disk-size 32g --cni=calico
```

</details>

#### 2. Deploy Otterize
To deploy Otterize, head over to [Otterize Cloud](https://app.otterize.com) and associate a Kubernetes cluster on the [Integrations page](https://app.otterize.com/integrations), and follow the instructions. If you already have a Kubernetes cluster connected, skip this step.

#### 2. ngrok
We will be using it to create a proxy to connect our locally running database to Otterize Cloud, for the tutorial's purposes. Once you have a [ngrok account](https://dashboard.ngrok.com/signup), you’ll want to install it in your terminal using the instructions found here: [ngrok install](https://ngrok.com/download)

# Tutorial

Expand All @@ -80,65 +78,50 @@ kubectl create namespace otterize-tutorial-postgres
kubectl apply -n otterize-tutorial-postgres -f ${ABSOLUTE_URL}/code-examples/postgres/client-server-database.yaml
```

### Make the database accessible to Otterize Cloud

We need to allow Otterize Cloud to access the database server so Otterize Cloud can configure on-demand credentials for our server’s access. This tutorial will expose our database port to our local environment and then proxy it to Otterize Cloud using ngrok. We will need both of these processes up and running during the rest of this tutorial.

In a new terminal window, run the following command to forward our database port from our cluster into your local environment:
```shell
kubectl port-forward svc/database 5432:5432 -n otterize-tutorial-postgres
### Deploy a PostgresServerConfig to allow Otterize DB access
```yaml
apiVersion: k8s.otterize.com/v1alpha3
kind: PostgreSQLServerConfig
metadata:
name: postgres-tutorial-db
spec:
address: database.otterize-tutorial-postgres.svc.cluster.local:5432
credentials:
username: otterize-tutorial
password: jeffdog523
```
The above CRD tells Otterize how to access a database instance named `postgres-tutorial-db`, meaning that when intents
are applied requesting access permissions to `postgres-tutorial-db`, Otterize operators will be able to configure
them.

Now that your database port is accessible to your local environment, we are using ngrok to make that available to Otterize Cloud. For production uses, this can be done through firewall configurations.
In this tutorial, the `database` workload already comes with the predefined username & password, but for future uses a
role will have to be created in the database to grant Otterize access as well as the ability to configure other users.
:::caution
The type PostgreSQLServerConfig should be considered as sensitive and require high cluster privileges to access.
:::

In a new terminal window, run:
Let's apply the above `PostgreSQLServerConfig` so Otterize will know how to access our database instance.
```shell
ngrok tcp 5432
kubectl apply -f pgserverconf.yaml
```

Once ngrok is running, make note of the *Forwarding* host and port. Will need this for our next step.

### Integrate the database to Otterize Cloud

To add the database, we head over to the [Integrations page](https://app.otterize.com/integrations)

1. Click *Add Integration*
2. Select Integration Type: *Database*
3. Provide a name for the integration: *otterize-tutorial-postgres*
4. Leave the database type set to *PostgreSQL*
5. Copy your *Forwarding* host and port from ngrok in the *Address* Field. This will look something like `0.tcp.us-cal-1.ngrok.io:14192`. Be sure to remove the `tcp://` portion of the URL.
6. *Username*: otterize-tutorial, *Password*: jeffdog523
1. Note this is a superuser, which allows Otterize to create unique credentials for each service. For production, it is recommended to create a privileged user for Otterize’s exclusive use. This user should have the necessary permissions to GRANT access to any databases and tables you want it to manage.
7. Hit *Test Connection*, and you should see an “OK” status.
8. Hit the Add button to complete the integration

### Integrate the cluster to Otterize Cloud
Create a Kubernetes integration on the [Integrations page](https://app.otterize.com/integrations), and follow the instructions.

In the second step, after providing an integration name and environment, choose:

1. mTLS and Kafka Support: None
2. Enforcement mode: Enabled.
3. Copy and run the Helm upgrade command.
4. You should see the Connection status change.


### View logs for the server
After the client, server, and database are up and running, we can see that the server does not have the appropriate access to the database by inspecting the logs with the following command.

```shell
kubectl logs -f -n otterize-tutorial-postgres deploy/server
```


Example log:
<Terminal>
pq: password authentication failed for user "svc_9cigb2qemv_otterize_tutorial_postgres_server"
Unable to perform INSERT operation
<br></br>
Unable to perform SELECT operation
</Terminal>

### Define your ClientIntents

ClientIntents are Otterize’s way of defining access through unique relationships, which lead to perfectly scoped access. In this example, we provide our `server` service the ability to insert select records to allow it to access the database.
ClientIntents are Otterize’s way of defining access through unique relationships, which lead to perfectly scoped access. In this example, we provide our `server` workload the ability to insert and select records to allow it to access the database.

Below is our `intents.yaml` file. As you can see, it is scoped to our database named `otterize-tutorial` and our `public.example` table. We also have limited the access to just `SELECT` and `INSERT` operations. We could add more databases, tables, or operations if our service required more access.

Expand All @@ -153,7 +136,7 @@ spec:
service:
name: server
calls:
- name: otterize-tutorial-postgres # Same name as our integration
- name: postgres-tutorial-db # Same name as our PostgresSQLServerConfig metadata.name
type: database
databaseResources:
- databaseName: otterize-tutorial
Expand All @@ -163,24 +146,24 @@ spec:
- INSERT
```

We can now apply our intents. Behind the scenes, Otterize Cloud runs `CREATE USER` and `GRANT` queries on the database, making our `SELECT` and `INSERT` errors disappear.
We can now apply our intents. Behind the scenes,the Otterize credentials-operator created the user for our `server` workload while the intents-operator ran `GRANT` queries on the database, making our `SELECT` and `INSERT` errors disappear.

```shell
kubectl apply -f intents.yaml
```

Example log:
<Terminal>
Successfully INSERTED into our table
Successfully SELECTED, most recent value: 2024-01-22T18:48:43Z
Successfully INSERTED into our table

Successfully SELECTED, most recent value: 2024-04-30T13:20:46Z
</Terminal>

That’s it! If your service’s functionality changes, adding or removing access is as simple as updating your ClientIntents definitions. For fun, try altering the `operations` to just `SELECT` or `INSERT`.

# Teardown
To remove the deployed examples, run:
```shell
kubectl delete clientintents.k8s.otterize.com -n otterize-tutorial-postgres client-intents-for-server && \
kubectl delete namespace otterize-tutorial-postgres
```

End the ngrok and port forwarding processes by closing the terminal windows or Ctrl-C the processes.
```
12 changes: 5 additions & 7 deletions docs/getting-started/README.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@ export const features = [
icon: '/img/icons/kafka-no-word-mark.svg',
url: '/features/kafka/'
},
{
title: 'PostgreSQL',
icon: '/img/icons/postgresql-no-word-mark.svg',
url: '/features/postgresql/'
},
{
title: 'Istio',
icon: '/img/icons/istio-no-word-mark.svg',
Expand All @@ -56,13 +61,6 @@ export const features = [
}
];

[//]: # ({)
[//]: # (title: 'PostgreSQL',)
[//]: # ( icon: '/img/icons/postgresql-no-word-mark.svg',)
[//]: # ( url: '/features/postgresql/')
[//]: # ( },)


export const tutorials_access = [
{
title: 'Create and manage network policies',
Expand Down

0 comments on commit a71fbe8

Please sign in to comment.