Added random_forest_regressor

[arthiondaena]

addresses issue #3598

I implemented random_forest_regressor class.

[mlpack-bot]

Thanks for opening your first pull request in this repository! Someone will review it when they have a chance. In the mean time, please be sure that you've handled the following things, to make the review process quicker and easier:

• All code should follow the style guide
• Documentation added for any new functionality
• Tests added for any new functionality
• Tests that are added follow the testing guide
• Headers and license information added to the top of any new code files
• HISTORY.md updated if the changes are big or user-facing
• All CI checks should be passing

Thank you again for your contributions! 👍

[shaojunjie0912]

Compared with the example of decision tree regression, it is found that there is a problem with the prediction value of random forest regression, and the specific code is as follows.

#include <armadillo>
#include <mlpack/methods/random_forest.hpp>

int main() {
    arma::mat dataset(10, 1000, arma::fill::randu);            // 1000 points
    arma::rowvec responses = arma::randn<arma::rowvec>(1000);  // responses

    responses.print();  // print responses

    arma::mat testDataset(10, 500, arma::fill::randu);  // 500 test points
    mlpack::RandomForestRegressor rf;
    rf.Train(dataset, responses);  // train
    arma::rowvec predictions;
    rf.Predict(testDataset, predictions);  // predict

    predictions.print();  // print predictions
}

The predicted values for decision tree regression are as follows.

The predicted values for random forest regression are as follows.

[arthiondaena]

@shaojunjie0912 Thank you for informing, I forgot to change the type of prediction variable. The size_t is trying to store double, the result was the large integer value you were checking.

If there are anymore errors inform me, I will try my best to fix them.

[arthiondaena]

Hey @zoq, I completed everything that a random forest regressor needs, if there is any need for modification or addition, ping me.

Now I want to add an adaboost regressor which is another part of that issue, these are the things I will be implementing:

• Prune function for decision tree regressor
• Loss functions(linear, square law, exponential)
• Atlast the implementation
  What are your thoughts about it?

[zoq]

Looking at the code, do you think it would be possible to add another Predict function to the existing RandomForest implementation?

[zoq]

Can you add some more details here? You can look at https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/linear_regression/linear_regression.hpp#L25 as a reference.

[zoq]

Would be great to have a description for the none trivial template parameters.

[zoq]

Suggested change

	//Calculate r2 score for random forest regressor
	double rfrR2Score = 1 - arma::sum(square(Ytest-rfrPred))/arma::sum(square(Ytest-TestMean));
	arma::Row<double> dtrPred;
	dtr.Predict(Xtest, dtrPred);
	//Calculate r2 score for decision tree regressor
	// Calculate r2 score for random forest regressor.
	double rfrR2Score = 1 - arma::sum(square(Ytest-rfrPred))/arma::sum(square(Ytest-TestMean));
	arma::Row<double> dtrPred;
	dtr.Predict(Xtest, dtrPred);
	// Calculate r2 score for decision tree regressor.

Minor style adjustments, to follow https://github.com/mlpack/mlpack/wiki/DesignGuidelines.

[zoq]

Looks like the indentation is off here, see https://github.com/mlpack/mlpack/wiki/DesignGuidelines#line-length-and-wrapping for more details.

[zoq]

Suggested change

	double dtrR2Score = 1 - arma::sum(square(Ytest-dtrPred))/arma::sum(square(Ytest-TestMean));
	double dtrR2Score = 1 - arma::sum(square(Ytest-dtrPred)) / arma::sum(square(Ytest-TestMean));

Use space between operations, also all names should follow the naming scheme https://github.com/mlpack/mlpack/wiki/DesignGuidelines#naming-conventions.

[arthiondaena]

Looking at the code, do you think it would be possible to add another Predict function to the existing RandomForest implementation?

What should be the functionality for this new predict function that is different from other?
And do you mean random forest or random forest regressor, as random forest already has 5-6 predict functions

[shaojunjie0912]

Hi~

Pls when can this branch be merged into the master branch?

@arthiondaena

[arthiondaena]

Hi~

Pls when can this branch be merged into the master branch?

@arthiondaena

Should probably ask the reviewer.

[arthiondaena]

Hey @zoq, when can this be merged?

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍