Mycorrhizal Network

Overview

On the list

• Put it on a map - Thanks for the brainstorming from 🐝

Prerequisites

• Kubernetes 1.18+
• Helm 3.0+ (installing helm)
• Python 3.9
• Poetry (installing poetry)

Part 1 🧠 - Install the Kafka cluster with neo4j

Ensure that the server has at least 8GB memory for Docker

1. Install the Kafka cluster with the neo4j database:

helm repo add tintinrevient https://tintinrevient.github.io/helm-chart
helm repo update
helm install data-door-release tintinrevient/data-door-chart --namespace "data-door" --create-namespace

⚠️ If 0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate, you can untaint the desired node using kubectl taint nodes --all node-role.kubernetes.io/control-plane-. This issue happens if you create a cluster with kubeadm, and the master node (which is node-role.kubernetes.io/control-plane- in this case) is planned to be deployable for pods as well.

💁‍♀️ To install Kubernetes cluster with kubeadm, you can reference this guide.

2. Expose the following four services in the host machine:

kubectl port-forward svc/external-broker -n data-door 9093:9093 --address='0.0.0.0'
kubectl port-forward svc/control-center -n data-door 9021:9021
kubectl port-forward svc/neo4j -n data-door 7687:7687
kubectl port-forward svc/neo4j-web -n data-door 7474:7474

3. Access the control center via http://localhost:9021/ and create this traffic neo4j connector and this hops neo4j connector in ksqlDB, which will be used to sink traffic and hops topics into neo4j database.

4. When finished, the environment can be nuked by the following command:

$ helm uninstall data-door-release --namespace "data-door"

Part 2 🐙 - Install the Kafka producer with scapy

1. Install all the dependencies:

poetry install

2. Run the packet sniffer to capture and dissect IP datagram as source IP and destination IP, and send to traffic topic in the Kafka broker:

poetry run network monitor_ip --broker="[local IP address of your exposed external Kafka broker]:9093"

3. Run traceroute to capture the hops between source IP and destination IP, and send to hops topic in the Kafka broker:

poetry run network trace_route --broker="[local IP address of your exposed external Kafka broker]:9093"

Result

The inter-connected hops can be viewed in neovis via this html:

References

• https://en.wikipedia.org/wiki/Mycorrhizal_network