The computaion of neg_reward is wrong

[zzxn]

This code uses batch-averaged (sample_rouge - baseline rouge), but it don't make sense in math and this item should be sample-wise because what we really want to maximize is this:

[saiprabhakar]

Check #7 . The negative sign is included in the LogP so the author has reversed it in the reward.