Support for XGBClassifier

[LeLoMAu]

Hello guys, first of all great work about current package!

I'm using it within my company and I'm finding it very useful.

The main limitation that I see right now is the fact that is not applicable to the most common XGBoost implementation (i.e. xgboost one; XGBClassifier), but only to the one of scikit-learn.

Do you see value on extending it? If so I can submit a PR to you

I've read your paper and quickly looked at your code, and I guess I should work on creating the proper adapter (https://github.com/linkedin/TE2Rules/blob/main/te2rules/adapter.py)

Let me know your feedback!

[groshanlal]

Thank you for reaching out and sharing your positive feedback on TE2Rules! We're happy to hear that TE2Rules has been useful for you and your company. We're eager to hear more about your specific use case and how the explainability provided by TE2Rules has benefited your product.

You're absolutely right; our current support is limited to scikit-learn. Extending the package to include the original xgboost model is a great idea. Please feel free to initiate a pull request to create an adapter for xgboost models. We are excited to receive your contribution, which will enhance the accessibility of TE2Rules to a broader range of models.

If you have any questions or need further clarification as you work on the adapter, please feel free to reach out. We're here to support you throughout the process. We look forward to reviewing your pull request.

Thanks!

[groshanlal]

Just wanted to drop a quick note with some code that might come in handy for you. While working on our paper, we had to switch a scikit-learn GradientBoostingClassifier to an xgboost model for some baseline runs. You can check out the code to convert skl2xgb here: https://github.com/linkedin/TE2Rules/blob/main/demo/converter_utils.py. It breaks down how the internal variables of scikit-based models and xgboost models relate. It could be useful as you put together your own adapter for the xgboost model. Take a look and feel free to use bits of it in your work!

[LeLoMAu]

Thanks @groshanlal! I will take a closer look this weekend then, and let you know if I have further doubts / questions, thank you!

About my current use case:

I work for a fintech (Revolut) and we use ML models for cross-selling products.
Since the implementation of a complete ML model is expensive in terms of resources, especially for those cases where the benefits of the model are not very clear, the implementation of simple rules (extracted from the model itself) is much simpler and more suitable to start with.

Combined with the fact that logics are much easier to interpret, explain and debug.

For all of this I'm benefiting a lot from your work!

[groshanlal]

Sounds good @LeLoMAu! Your use case of using explainability in fintech is very interesting. Thanks again for your contribution.

[wenqinglim]

Hi both, just wondering if any work has been done on extending this to xgb?

I've tried te2rules on sklearn's random forest and gradient boosting classifiers and the rules generated look great! However, there are a couple of benefits that xgb provides over the sklearn models, and an xgb adaptor would be super helpful.

[groshanlal]

Hi @LeLoMAu, do you have any updates on building an adapter for the XGBoost classifier? If you need help, we can work together on this. Extending TE2Rules to work directly with XGBoost models would be a really useful feature to have!

[LeLoMAu]

Hey @groshanlal @wenqinglim just submitted PR

Btw @wenqinglim, can I ask you in which context are you applying te2rules?

Just for curiosity

[wenqinglim]

Hi! Thank you both for your replies, and I'm looking forward to the PR merge :)

@LeLoMAu I'm using it in fraud detection, where stakeholders require clear, human-readable decisions. We've started off with extracting decision paths from a single decision tree, but te2rules opens up the possibility of using ensemble tree-based models instead.

[groshanlal]

Thanks @LeLoMAu for your work. I have merged your PR with the main branch and released a new version to PyPI. I'm happy to announce that the latest version of TE2Rules: v1.0.0 has support for XGBoost classifiers.

Thanks @wenqinglim for providing details on your use case. You can now use TE2Rules with XGBoost classifiers for your use case. I have also updated the README with some additional details on how to use TE2Rules for local and global level explainability. All of these will now be applicable for both Scikit and XGBoost models. Let us know if you run into any issues while using TE2Rules on the xgboost models.