Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Post-commit hook not triggered on Riak 3.0.9 #1098

Open
pingenglj opened this issue Dec 27, 2021 · 6 comments
Open

Post-commit hook not triggered on Riak 3.0.9 #1098

pingenglj opened this issue Dec 27, 2021 · 6 comments

Comments

@pingenglj
Copy link

Hi, Ive installed Riak 3.0.9, and am trying to add a postcommit hook:

cat > post_commit.erl << EOF
-module(post_commit).
-export([post/1]).

post(RiakObject) ->
  ssl:start(),
  application:start(inets),
  httpc:request(get,
      {"http://10.0.0.11:8000/", []}, [], []).    <----- Running a local API here
EOF
/usr/lib/riak/erts-10.7/bin/erlc post_commit.erl # must compile with riak erlc
sudo mv ./post_commit.beam /tmp/riak/
riak admin bucket-type create watch '{"props": {"postcommit": ["post_commit"]}}'
riak admin bucket-type activate watch

I have the following in /etc/riak/advanced.config, and did a riak chkconfig and riak restart after I made this change:

...
  {riak_kv, 
    [{add_paths, ["/tmp/riak/"]}]
  },
...

When I manually run the Erlang hook I see a response on my API running at port 8000: post_commit:post("").
However, when I add some data into a bucket, I don't see anything:

curl -X PUT http://10.0.0.11:8098/types/watch/buckets/some-bucket/keys/test.json -H "Content-type: application/json" --data-binary @test.json

I cannot see any indication in the logs that the postcommit is triggered/executed (or if there is some error), the bucket props show it being there though, and bucket is active:

curl -X GET http://10.0.0.11:8098/types/watch/buckets/some-bucket/props
{"props":{"name":"some-bucket","active":true,"allow_mult":true,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"claimant":"riak@10.0.0.11","dvv_enabled":true,"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"node_confirms":0,"notfound_ok":true,"old_vclock":86400,"postcommit":["post_commit"],"pr":0,"precommit":[],"pw":0,"r":"quorum","rw":"quorum","small_vclock":50,"sync_on_write":"backend","w":"quorum","young_vclock":20}}

Am I missing anything?

@pingenglj
Copy link
Author

pingenglj commented Dec 27, 2021

If I look at /var/log/riak/run_erl.log I do see some indication of disabling hooks, but Im unsure if it is relevant for enabling postcommit hooks:

==> /var/log/riak/run_erl.log <==
run_erl [17359] Mon Dec 27 14:58:08 2021
Args before exec of shell:
run_erl [17359] Mon Dec 27 14:58:08 2021
argv[0] = sh
run_erl [17359] Mon Dec 27 14:58:08 2021
argv[1] = -c
run_erl [17359] Mon Dec 27 14:58:08 2021
argv[2] = exec "/usr/lib/riak/bin/riak" "console" '-pa' '/usr/lib/riak/lib/patches'  --relx-disable-hooks

I have a number of riak processes, both the run_erl and riak binaries seem to have the --relx-disable-hooks flag, would that interfere with my postcommit hook?:

riak     17063  0.0  0.0   8276    84 ?        S    14:57   0:00 /usr/lib/riak/erts-10.7/bin/epmd -daemon
riak     17356  0.0  0.2  15084  2064 ?        S    14:58   0:00 /usr/lib/riak/erts-10.7/bin/run_erl -daemon /tmp/erl_pipes/riak@10.0.0.11/ /var/log/riak exec "/usr/lib/riak/bin/riak" "console" '-pa' '/usr/lib/riak/lib/patches'  --relx-disable-hooks
riak     17359  0.8  7.1 2971304 71820 pts/0   Ssl+ 14:58   0:10 /usr/lib/riak/bin/riak -scl false -sfwi 500 -P 256000 -e 256000 -Q 262144 -A 64 -K true -W w -Bi -zdbbl 32768 -- -root /usr/lib/riak -progname usr/lib/riak/bin/riak -- -home /var/lib/riak -- -boot /usr/lib/riak/releases/3.0.9/riak -mode interactive -boot_var ERTS_LIB_DIR /usr/lib/riak/lib -config /var/lib/riak/sys.config -setcookie riak -name riak@10.0.0.11 -smp enable true -- -- console -pa /usr/lib/riak/lib/patches --relx-disable-hooks --
riak     17495  0.0  0.1   4528  1600 ?        Ss   14:58   0:00 erl_child_setup 65536
riak     17523  0.0  0.0   4636   828 ?        Ss   14:58   0:00 sh -s disksup
riak     17524  0.0  0.0   4388   808 ?        Ss   14:58   0:00 /usr/lib/riak/lib/os_mon-2.5.1/priv/bin/memsup
riak     17525  0.0  0.0   4520   808 ?        Ss   14:58   0:00 /usr/lib/riak/lib/os_mon-2.5.1/priv/bin/cpu_sup

@pingenglj
Copy link
Author

Also no luck with:

riak admin bucket-type update watch '{"props": {"postcommit": [{"mod": "post_commit", "fun": "post"}]}}'

@martinsumner
Copy link
Contributor

I think the correct format is to define both "mod" and "fun" in the postcommit bucket property as you have done above.

Are you able to call the Mod:Fun/1 from riak remote_console?
Have you tried adding a lager log line to your hook to see if it is called?
In riak admin stats is the postcommit_fail stat incrementing as you PUT objects?

@pingenglj
Copy link
Author

@martinsumner Thanks for the quick response. Curiously, after doing a clean install it now seems to work. Perhaps a file permission issue, I was messing with compiling and moving the .erl/.beam and config files, while the Vagrant install script Im using is executed as root.

@baoxiangwei
Copy link

This Riak version is 3.2.0, using Post-commit hook .

This is my erlang code:
`-module(post_commit).
-export([post/1]).

post(Object) ->
Path="/home/firefly/riak_new/riak_3.20/riak/rel/riak/test.sh",

KeyName = riak_object:key(Object),
io:format("Key name: ~s~n", [KeyName]),

BucketName = riak_object:bucket(Object), 
io:format("Bucket name: ~p~n", [BucketName]),

Value = riak_object:get_value(Object),
io:format("Object value: ~p~n", [Value]),

Command= io_lib:format("sh ~s ~s ~s ~n", [Path, [KeyName], [BucketName]]),
io:format("Command value: ~s~n", [Command]),

Output = os:cmd(Command),
io:format("~s~n", [Output]). `

when I add some data into a bucket, curl -XPUT http://localhost:8098/types/my_bucket_type/buckets/some-bucket/keys/babbbvv -H "Content-Type: application/json" -d '{"name": "John Smith", "age": 42, "email": "john.smith@example.caa1111odd9911m"}'

I can only get the key name "babbbvv " , but I can not get the bucket name "some-bucket".

@martinsumner
Copy link
Contributor

Can you clarify what you mean by "can not get the bucket name"? Do you mean in the io:format output you get something else (perhaps undefined) where you expected to get BucketName?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants