-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to use the example (counter) #239
Comments
Hi, thanks for trying it out. Yes, you're gonna need a config which contains the shard mapping for your cluster. We are migrating from using an admin tool to manage clusters to a Helix based solution which is implemented in the cluster_management directory. The helix based solution greatly reduced our operation burden. Would you mind telling us more about your use case? Not sure how urgent is your use case. We are gonna open source a production system based on rocksplicator soon, which is integrated with both admin tool and helix based solutions. |
Great news @newpoo, looking forward to your release! We are looking for a reasonably fast, horizontally scalable, replicated key-value store and rocksDB+rocksplicator looks like a good solution. The fact that you use this setup at Pinterest in production suggests that this would work for us too. So for now, I would just like to try rocksplicator and see how it works. Now I am trying to set up a demo project using rocksdb::DB* db;
rocksdb::Options options;
options.create_if_missing = true;
rocksdb::Status s = rocksdb::DB::Open(options, "/tmp/masterdb", &db);
assert(s.ok());
RocksDBReplicator* my_replicator = RocksDBReplicator::instance();
my_replicator->addDB("my_testdb", std::shared_ptr<rocksdb::DB>(db), DBRole::MASTER);
std::this_thread::sleep_for(100s); And two slaves, one is writing the db, the other reading it: rocksdb::DB* db;
rocksdb::Options options;
options.create_if_missing = true;
rocksdb::Status s = rocksdb::DB::Open(options, "/tmp/slavedb1", &db);
assert(s.ok());
RocksDBReplicator* my_replicator = RocksDBReplicator::instance();
my_replicator->addDB("my_testdb", std::shared_ptr<rocksdb::DB>(db), DBRole::SLAVE, folly::SocketAddress("127.0.0.1", 9091));
while (true)
{
// write something ...
std::this_thread::sleep_for(10s);
} // ...
rocksdb::Status s = rocksdb::DB::Open(options, "/tmp/slavedb2", &db);
RocksDBReplicator* my_replicator = RocksDBReplicator::instance();
my_replicator->addDB("my_testdb", std::shared_ptr<rocksdb::DB>(db), DBRole::SLAVE, folly::SocketAddress("127.0.0.1", 9091));
while (true)
{
// read the same key...
std::this_thread::sleep_for(2s);
} The idea is that the new values written by slave1 would be visible on slave2 via replication through the master. However, when I run the slaves, they are unable to connect to the master node. Could you please give me some pointers about this? Is this the right way to use rocksplicator or am I badly misunderstanding things? Thanks! |
For Master-Salve setup, writes are only allowed to go to Master, and Master will replicate these updates to all Slaves. In a production setting, you may use the interface defined in rocksdb_admin directory directly. the counter service has some example codes. |
Thanks for the timely reply @newpoo! Alright, so I'll go back to the counter example for now. So I started a I tried to send a request to |
Hi, I wanted to thank you for open sourcing this great library. |
Is there any documentation (step by step) to start "COUNTER" service? |
@newpoo Hi, thanks for open-sourcing this great lib!
BUT. counter print error
What am I missing? Maybe a few rocksdb should be started? |
hi , I have the same questions, did you already have the answer? |
Hi, thanks for open-sourcing this great lib!
I am having problems running the example counter application. I've successfully built everything, but when I run the example I get the following error:
I guess I need to pass
shard_config_path
. What is the syntax of this file?What happens when I run the example application? I understand it starts a server. I guess the python cluster management scripts communicate with this server? So is this a single Master node? How do I set up Slaves and how do I connect these components?
What is the advantage of using these admin tools / cluster management modules, as opposed to instantiating
RocksDBReplicator
directly from my code?Thanks!
The text was updated successfully, but these errors were encountered: