Skip to content

Commit

Permalink
Add docs
Browse files Browse the repository at this point in the history
  • Loading branch information
ljupcovangelski committed May 1, 2024
1 parent e97af5e commit 58f5dfb
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 18 deletions.
33 changes: 33 additions & 0 deletions docs/docs/getting-started/installation/helm.md
Expand Up @@ -290,6 +290,39 @@ Run the following command to create the `Airy` platform without the bundled inst
helm install airy airy/airy --timeout 10m --set prerequisites.kafka.enabled=false --values ./airy.yaml
```

#### Confluent

To connect to a Kafka instance in Confluent cloud, settings the `config.kafka.brokers` and `config.kafka.aurhJaas` is enough, prior to deploying the Helm chart.

#### Aiven

Aiven cloud uses a keystore and truststore certificates that need to be loaded on the workloads that are connecting to Kafka. Get the necessary certificates and connection files from Aiven using the `avn` CLI and place them in a separate directory.

```
avn service user-kafka-java-creds {KAFKA_INSTANCE} --username {USERNAME} -d ./aiven/ --password {PASSWORD}
```

Create a Kubernetes ConfigMap that contains the contents of the created directory:

```
kubectl create configmap kafka-config-certs --from-file aiven/
```

Set the connection appropriate parameters in your `airy.yaml` file:

```yaml
config:
kafka:
brokers: "the-aiven-kafka-broker-url"
keyTrustSecret: "the-key-trust-secret"
```

Then install Airy with the following command:

```sh
helm install airy airy/airy --timeout 10m --set prerequisites.kafka.enabled=false --set global.kafkaCertAuth=true --values ./airy.yaml
```

### Kafka partitions per topic

Currently all the default topics in the Airy instance are created with 10 partitions. To create these topics with a different number of partitions, add the following to your `airy.yaml` file before running `helm install` (before the initial creation of the topics):
Expand Down
Expand Up @@ -24,9 +24,6 @@ spec:
volumeMounts:
- name: kafka-create-topics
mountPath: /opt/provisioning
- name: kafka-config
mountPath: /opt/kafka/ca.pem
subPath: KAFKA_SASL_CA
initContainers:
- name: wait
image: "{{ .Values.global.busyboxImage }}"
Expand Down
Expand Up @@ -29,9 +29,6 @@ data:
REPLICAS=${KAFKA_MINIMUM_REPLICAS:-1}
AIRY_CORE_NAMESPACE=${AIRY_CORE_NAMESPACE:-}
AUTH_JAAS=${AUTH_JAAS:-}
KAFKA_SASL_USERNAME=${KAFKA_SASL_USERNAME:-}
KAFKA_SASL_PASSWORD=${KAFKA_SASL_PASSWORD:-}
KAFKA_SASL_CA=${KAFKA_SASL_CA:-}
if [ -n "${AIRY_CORE_NAMESPACE}" ]; then
AIRY_CORE_NAMESPACE="${AIRY_CORE_NAMESPACE}."
Expand All @@ -48,18 +45,6 @@ data:
echo "Using jaas authentication for connecting to Kafka"
fi
if [ -n "${KAFKA_SASL_CA}" ]; then
cat <<EOF > /opt/kafka/jaas.config
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.username=$KAFKA_SASL_USERNAME
sasl.password=$KAFKA_SASL_PASSWORD
sasl.ca.location=/opt/kafka/ca.pem
EOF
CONNECTION_OPTS+=(--command-config /opt/kafka/jaas.config)
echo "Using jaas authentication for connecting to Kafka"
fi
echo "Creating Kafka topics"
Expand Down

0 comments on commit 58f5dfb

Please sign in to comment.