Skip to content

Commit

Permalink
feat: add poll enrich eip module and upgrade knative to v0.4.0
Browse files Browse the repository at this point in the history
- added new module to demonstrate pollEnrich EIP with
- organized redundant instructions into partials
- upgraded instructions to setup Knative v0.4.0

fixes #42, #35
  • Loading branch information
kameshsampath committed Mar 7, 2019
1 parent 0474364 commit 5601991
Show file tree
Hide file tree
Showing 13 changed files with 278 additions and 79 deletions.
2 changes: 1 addition & 1 deletion documentation/modules/ROOT/pages/03scaling.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ We will now send some load to the greeter service. The command below sends 50 c
----
hey -z 10s -c 50 -t 0 \
-host "greeter.knativetutorial.example.com" \
"http://${IP_ADDRESS}"
"http://pass:[${IP_ADDRESS}]"
----

After youÄ've successfully run this small load test, you will notice the number of greeter service pods will have scaled to 5 automatically. You can also view the same via OpenShift dashboard as shown below:
Expand Down
12 changes: 2 additions & 10 deletions documentation/modules/ROOT/pages/05eventing/eventing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,10 @@ You have completed the xref:ROOT:04build/build.adoc[Build] or xref:ROOT:04build/
[#eventing-watch-logs]
== Watching Logs

:kube-ns: knativetutorial
:kube-svc: event-greeter
include::partial$watching-logs.adoc[]

[TIP]
====
* Using stern with the command `stern -n knativetutorial event-greeter`, to filter the logs further add `-c user-container` to the stern command.
[source,bash,subs="+macros,+attributes"]
----
stern -n knativetutorial -c user-container event-greeter
----
====

The logs will have the output like below printing every minute.

[source]
Expand Down
5 changes: 3 additions & 2 deletions documentation/modules/ROOT/pages/_attributes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
:build-repo: 04-build
:eventing-repo: 05-eventing
:experimental:
:knative-version: v0.3.0
:knative-serving-repo: https://github.com/knative/serving/releases/download
:knative-version: v0.4.0
:knative-serving-uri: https://github.com/knative/serving
:knative-serving-repo: {knative-serving-uri}/releases/download
:knative-build-repo: https://github.com/knative/build/releases/download
:knative-eventing-repo: https://github.com/knative/eventing/releases/download
:knative-sources-repo: https://github.com/knative/eventing-sources/releases/download
19 changes: 16 additions & 3 deletions documentation/modules/ROOT/pages/_partials/watching-logs.adoc
Original file line number Diff line number Diff line change
@@ -1,15 +1,28 @@
[kube-ns='knativetutorial']
[kube-svc='']

Since it is a Cron job source, it would have started to emit events every minute. We can watch the logs of the service to see the messages delivered.

The logs could be watched using the command:

[source,bash,subs="+macros,+attributes"]
----
kubectl logs -n knativetutorial -f <pod-name> -c user-container
kubectl logs -n {kube-ns} -f <pod-name> -c user-container
----

.(OR)

[source,bash,subs="+macros,+attributes"]
----
oc logs -n knativetutorial -f <pod-name> -c user-container
----
oc logs -n {kube-ns} -f <pod-name> -c user-container
----

[TIP]
====
* Using stern with the command `stern -n {kube-ns} {kube-svc}`, to filter the logs further add `-c user-container` to the stern command.
[source,bash,subs="+macros,+attributes"]
----
stern -n {kube-ns} -c user-container {kube-svc}
----
====
2 changes: 1 addition & 1 deletion documentation/modules/ROOT/pages/faq.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[#faq-q1]
== How to access the Knative services ?

When looking up the IP address to use for accessing your app, you need to look up the NodePort for the **istio-ingressgateway** well as the IP address used for minishift. You can use the following command to look up the value to use for the {IP_ADDRESS} placeholder used in the sample
When looking up the IP address to use for accessing your app, you need to look up the NodePort for the **istio-ingressgateway** well as the IP address used for minishift. You can use the following command to look up the value to use for the pass:[{IP_ADDRESS}] placeholder used in the sample
[source,bash,linenums]
----
#!/bin/bash
Expand Down
6 changes: 4 additions & 2 deletions documentation/modules/ROOT/pages/setup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ The `work` folder in `$TUTORIAL_HOME` can be used to download the demo applicati
This tutorial was developed and tested with:
- Knative `v0.3.0`
- Knative `v0.4.0`
- minikube `v0.34.1`
- OpenShift `v3.11`
- minishift `v1.30.0+186b034`
Expand Down Expand Up @@ -162,6 +162,7 @@ knative-ingressgateway-75644679c7-5xwx6 1/1 Running 0 6h27m
[source,bash,subs="+macros,+attributes"]
----
kubectl apply --filename {knative-serving-repo}/{knative-version}/serving.yaml
kubectl apply --filename https://raw.githubusercontent.com/knative/serving/{knative-version}/third_party/config/build/clusterrole.yaml
----

[NOTE]
Expand Down Expand Up @@ -190,7 +191,7 @@ webhook-6d9568d-htxlt 1/1 Running 0 6h33m

[source,bash,subs="+macros,+attributes"]
----
kubectl apply --filename {knative-build-repo}/{knative-version}/release.yaml
kubectl apply --filename {knative-build-repo}/{knative-version}/build.yaml
----

[NOTE]
Expand Down Expand Up @@ -218,6 +219,7 @@ build-webhook-6bb747665f-trjxg 1/1 Running 0 6h32m
[source,bash,subs="+macros,+attributes"]
----
kubectl apply --filename {knative-eventing-repo}/{knative-version}/release.yaml
kubectl apply --filename {knative-eventing-repo}/{knative-version}/in-memory-channel.yaml
kubectl apply --filename {knative-sources-repo}/{knative-version}/release.yaml
----

Expand Down
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
14 changes: 13 additions & 1 deletion documentation/modules/camelk/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,16 @@
*** xref:camelk:getting-started.adoc#gs-test-cartoon-messages-mover[Testing Route]
***** xref:camelk:getting-started.adoc#gs-test-data[Upload Test Data]
***** xref:camelk:getting-started.adoc#gs-invoke-service[Invoke service]
*** xref:camelk:getting-started.adoc#gs-cleanup[Cleanup]
*** xref:camelk:getting-started.adoc#gs-cleanup[Cleanup]
** xref:camelk:poll-enrich.adoc[s3 File Downloader]
*** xref:camelk:poll-enrich.adoc#pe-prereq[Prerequisite]
*** xref:camelk:poll-enrich.adoc#pe-overview[Demo Overview]
*** xref:camelk:poll-enrich.adoc#pe-deployment[Deployment]
**** xref:camelk:poll-enrich.adoc#pe-make-s3-data-bucket[Create data s3 bucket]
**** xref:camelk:poll-enrich.adoc#pe-make-s3-top-bucket[Create top s3 bucket]
**** xref:camelk:poll-enrich.adoc#pe-cartoon-messages-downloader[Deploy Camel Route]
*** xref:camelk:poll-enrich.adoc#pe-watch-logs[Watching Logs]
*** xref:camelk:poll-enrich.adoc#pe-test-cartoon-messages-downloader[Testing Route]
***** xref:camelk:poll-enrich.adoc#pe-test-data[Upload Test Data]
***** xref:camelk:poll-enrich.adoc#pe-invoke-service[Invoke service]
*** xref:camelk:poll-enrich.adoc#pe-cleanup[Cleanup]
31 changes: 31 additions & 0 deletions documentation/modules/camelk/pages/_partials/invoke-service.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
[svc-name='']
[file-name='']

#tag::set-svc-url[]
[source,bash,subs="+macros,+attributes"]
----
SVC_URL=`kubectl -n knativetutorial get ksvc {svc-name} -ojsonpath='{.status.domain}'`
----

.(OR)

[source,bash,subs="+macros,+attributes"]
----
SVC_URL=`oc -n knativetutorial get ksvc {svc-name} -ojsonpath='{.status.domain}'`
----
#end::set-svc-url[]

#tag::invoke-svc[]

[source,bash,subs="+macros,+attributes"]
----
curl -v -H pass:["Host:${SVC_URL}"] -H 'fileName: {file-name}' $IP_ADDRESS
----

.(OR)

[source,bash,subs="+macros,+attributes"]
----
http pass:[$IP_ADDRESS "Host:${SVC_URL}"] 'fileName: {file-name}'
----
#end::invoke-svc[]
32 changes: 29 additions & 3 deletions documentation/modules/camelk/pages/_partials/wait-for-pods.adoc
Original file line number Diff line number Diff line change
@@ -1,13 +1,39 @@
[kube-ns='']

#tag::wait-with-kubectl[]

[NOTE]
====
It may take few minutes for pods to come up, you can watch the status of the pods as shown below.
You can use kbd:[CTRL + c ] to terminate the watch
====

[source,bash,subs="+macros,+attributes"]
----
kubectl -n {kube-ns} get pods -w
----
.(OR)
[source,bash,subs="+macros,+attributes"]
----
oc -n {kube-ns} get pods -w
----

#end::wait-with-kubectl[]

#tag::wait-with-timeout[]

Wait for the pods to be running:

[source,bash,subs="+macros,+attributes"]
----
timeout 300 "kubectl get pods -n knativetutorial 2>&1 | grep -v -E '(Running|Completed|STATUS)'"
timeout 300 "kubectl get pods -n {kube-ns} 2>&1 | grep -v -E '(Running|Completed|STATUS)'"
----

.(OR)

[source,bash,subs="+macros,+attributes"]
----
timeout 300 "oc get pods -n knativetutorial 2>&1 | grep -v -E '(Running|Completed|STATUS)'"
----
timeout 300 "oc get pods -n {kube-ns} 2>&1 | grep -v -E '(Running|Completed|STATUS)'"
----
#end::wait-with-timeout[]

70 changes: 16 additions & 54 deletions documentation/modules/camelk/pages/getting-started.adoc
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
= Getting Started
include::_attributes.adoc[]

In this chapter we will use Knative + Camel-k to deploy a simple route that will move __*.xml__ file from one s3 bucket to another.

[#gs-prereq]
== Prerequisite
include::camelk:partial$prereqs.adoc[]

[#gs-overview]
== Enterprise Integration Pattern

[sidebar]
Expand All @@ -15,7 +18,6 @@ The Message Filter has only a single output channel. If the message content matc
Source: https://www.enterpriseintegrationpatterns.com/patterns/messaging/Filter.html
****

[#gs-overview]
== Demo Overview

The demo has an Apache Camel route configured as **knative serving** endpoint, that gets activated once it receives an event i.e. s3 object (xml file) name. Upon activation it starts a Apache Camel route that applies **Message Filter EIP** to process only the `xml` files. The matching xml files will be moved to another bucket called `top(to process)`, while other files are discarded.
Expand Down Expand Up @@ -71,15 +73,13 @@ cd $TUTORIAL_HOME/work/knative-tutorial-camelk/quickstart
[source,bash,subs="+macros,+attributes"]
----
kamel run --dependency='camel-aws'\#<1>
--trait gc.enabled=false \#<2>
--property 's3EndpointUrl=http://minio-server' \
--property 'minioAccessKey=demoaccesskey' \
--property 'minioSecretKey=demosecretkey' \
link:{camelk-demos-github-repo}/quickstart/src/main/java/CartoonMessagesMover.java[src/main/java/CartoonMessagesMover.java]
----

<1> Because of an existing https://github.com/apache/camel-k/issues/527[bug], you need to add the dependency `camel-aws`, this will be removed in future.
<2> **Only for development purpose ** Since we want to have the pods terminated quickly during undeploy, we disable the Garbage Collection.

You can check the deployed integration context:

Expand All @@ -94,18 +94,10 @@ cartoon-messages-mover Building Context ctx-bhvn6lcba0587gdq5jbg
[#gs-watch-logs]
=== Watching Logs

:kube-ns: knativetutorial
:kube-svc: cartoon-messages-mover
include::ROOT:partial$watching-logs.adoc[]

[TIP]
====
* Using stern with the command `stern -n knativetutorialcartoon-messages-mover `, to filter the logs further add `-c user-container` to the stern command.
[source,bash,subs="+macros,+attributes"]
----
stern -n knativetutorial -c user-container cartoon-messages-mover
----
====

[#gs-test-cartoon-messages-mover]
=== Test the Route

Expand All @@ -131,38 +123,17 @@ aws --endpoint-url $S3_URL s3 ls s3://data #<1>
[#gs-invoke-service]
==== Invoke service

Invoke the service by passing a header `fileName` with value `message1.xml`, this `fileName` header value will be used by Camel route to download the file form s3 and return it as the response.
Invoke the service by passing a header `fileName` with value `message1.xml`, this `fileName` header value will be used by Camel route to move the file from s3 bucket `data` to `top`.

[source,bash,subs="+macros,+attributes"]
----
IP_ADDRESS=$(minikube ip):32380
----
include::ROOT:partial$invoke-service.adoc[tag=env]

We need to have knative service `cartoon-message-mover` domain name which needs to be passed as $SVC_URL when invoking the Knative service:

[source,bash,subs="+macros,+attributes"]
----
SVC_URL=`kubectl -n knativetutorial get ksvc cartoon-messages-mover -ojsonpath='{.status.domain}'`
----
:svc-name: cartoon-messages-mover
include::camelk:partial$invoke-service.adoc[tag=set-svc-url]

.(OR)

[source,bash,subs="+macros,+attributes"]
----
SVC_URL=`oc -n knativetutorial get ksvc cartoon-messages-mover -ojsonpath='{.status.domain}'`
----

[source,bash,subs="+macros,+attributes"]
----
curl -H "Host:${SVC_URL}" -H 'fileName: message1.xml' $IP_ADDRESS
----

.(OR)

[source,bash,subs="+macros,+attributes"]
----
http $IP_ADDRESS "Host:${SVC_URL}" 'fileName: message1.xml'
----
:file-name: message1.xml
include::camelk:partial$invoke-service.adoc[tag=invoke-svc]

[NOTE]
====
Expand All @@ -173,27 +144,23 @@ On success which is usually HTTP 200, you can list the out `top` bucket to see t

[source,bash,subs="+macros,+attributes"]
----
# run the command to verify upload
# run the command to verify the file(s) were moved
aws --endpoint-url $S3_URL s3 ls s3://top #<1>
----
<1> You should see the `message1.xml` listed

Lets now try sending the request for `hello.txt`:

[source,bash,subs="+macros,+attributes"]
----
curl -H "Host:${SVC_URL}" -H 'fileName: hello.txt' $IP_ADDRESS
----
:file-name: hello.txt
include::camelk:partial$invoke-service.adoc[tag=invoke-svc]

.(OR)
Since we process only `*.xml` files the `top` bucket will not have the file `hello.txt` moved from `data` bucket, you can verify the same using the command:

[source,bash,subs="+macros,+attributes"]
----
http $IP_ADDRESS "Host:${SVC_URL}" 'fileName: hello.txt'
aws --endpoint-url $S3_URL s3 ls s3://top
----

Since we process only `*.xml` files the `top` bucket will not have the file `hello.txt` moved from `data` bucket.

[NOTE]
====
When the `cartoon-messages-mover` has been scaled to zero; when giving a new request it will try to scale up, it will not be able to serve the request until the pod is up, during such time you might get HTTP 502 Bad Gateway errors, give some more time for the pod to come up before issuing the request again.
Expand All @@ -208,8 +175,3 @@ Right now camel-k does not add Liveliness and Readiness Probes which can help av
----
kamel delete cartoon-messages-mover -n knativetutorial
----

[WARNING]
====
Sometimes the Knative pods takes 3-5 mins to terminate completely, its better to wait for the `0/3 Terminating` for the cartoon-messages-mover pods before moving to next sections of the tutorial or trying new updates.
====

0 comments on commit 5601991

Please sign in to comment.