Skip to content

rongfengliang/memsql-kafka-pipeline-docker-compose

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 

Repository files navigation

memsql with kafka pipeline demo

how to run

  • mkdir .env file && get license from memsql website && copy license content to .env
LICENSE_KEY=${youlicensecontent}
  • start deps
docker-compose up -d
  • create kafka topic

note: must inside kafka container

/opt/kafka/bin/kafka-topics.sh --topic test --zookeeper zk:2181 --create --partitions 8 --replication-factor 1
  • create insert memsql database && table && pipeline

note: you can use mysql client do below ops

database:
CREATE DATABASE quickstart_kafka;
USE quickstart_kafka;
table:
CREATE TABLE messages (id text);

pipeline:
CREATE PIPELINE `quickstart_kafka` AS LOAD DATA KAFKA 'kafka/test' INTO TABLE `messages`;
  • start pipeline
START PIPELINE quickstart_kafka;
  • send message

note: must inside kafka container

/opt/kafka/bin/kafka-console-producer.sh --topic test --broker-list 127.0.0.1:9092
  • select insert message
SELECT * FROM quickstart_kafka.messages;

Releases

No releases published

Packages

No packages published