Add graph pass and backward option

[xinyual]

Description

This is the branch to add the zero sharding (especially partitioning gradient). The changes can be divided into three part:

1. in C++, I add an operation called reduce operation. It will do nothing in forward but reduce gradient on backward.
2. Then open an API called backward option to delete the output of certain graph.

Current problem

1. The reduce operation will cause a deadlock(won't happen in NaiveEngine)
2. The decrease of memory consumption hasn't been verified.

[mxnet-bot]

Hey @xinyual , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

• To trigger all jobs: @mxnet-bot run ci [all]
• To trigger specific jobs: @mxnet-bot run ci [job1, job2]

---

CI supported jobs: [unix-gpu, centos-cpu, website, unix-cpu, windows-gpu, windows-cpu, centos-gpu, miscellaneous, clang, sanity, edge]

---

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

[leezu]

Thank you @xinyual! Could you please make sure the CI tests (ci/jenkins/mxnet-validation/sanity) pass, then we can merge this code into the zero_sharding feature branch for future devolpment

[xinyual]

Thank you @xinyual! Could you please make sure the CI tests (ci/jenkins/mxnet-validation/sanity) pass, then we can merge this code into the zero_sharding feature branch for future devolpment

Hi leo, I don't know where is the problem since the error information only shows the wrong command. Is this related to third party ondnn? It seems I don't update it to latest version.

[leezu]

Thank you @xinyual. Please rebase on top of the apache/incubator-mxnet master branch and force push to your xinyual/incubator-mxnet add_graph_pass_and_backward_option branch