[Roadmap] XGBoost 1.0.0 Roadmap #4680

CodingCat · 2019-07-18T15:49:27Z

@dmlc/xgboost-committer please add your items here by editing this post. Let's ensure that

each item has to be associated with a ticket
major design/refactoring are associated with a RFC before committing the code
blocking issue must be marked as blocking
breaking change must be marked as breaking

for other contributors who have no permission to edit the post, please comment here about what you think should be in 1.0.0

I have created three new types labels, 1.0.0, Blocking, Breaking

Improve installation experience on Mac OSX (Better XGBoost installation on Mac OSX? #4477)
Remove old GPU objectives.
Remove gpu_exact updater (deprecated) Deprecate gpu_exact, bump required cuda version in docs #4527
Remove multi threaded multi gpu support (deprecated) [RFC] Remove support for single process multi-GPU #4531
External memory for gpu and associated dmatrix refactoring [RFC] External memory support for GPU #4357 [RFC] Possible DMatrix refactor #4354
Spark Checkpoint Performance Improvement ([jvm-packages] Checkpointing performance issue in XGBoost4J-Spark #3946)
[BLOCKING] the sync mechanism in hist method in master branch is broken due to the inconsistent shape of tree in different workers ([HOTFIX] distributed training with hist method #4716, [BLOCKING] Per-node sync slows down distributed training with 'hist' #4679)
Per-node sync slows down distributed training with 'hist' ([BLOCKING] Per-node sync slows down distributed training with 'hist' #4679)
Regression tests including binary IO compatibility, output stability, performance regressions.

thesuperzapper · 2019-07-19T03:39:05Z

Not a committer, but can we please target PySpark API for 1.0?
Issue: #3370
Current PR: #4656

CodingCat · 2019-07-19T03:40:23Z

for other contributors who have no permission to edit the post, please comment here about what you think should be in 1.0.0

thesuperzapper · 2019-07-19T03:40:27Z

Also, should we target moving exclusively to the Scala based Rabit tracker (for Spark) in 1.0?

trams · 2019-07-20T00:30:07Z

I am also not a committer but me and the company I work in is very interested in fixing the performance issue with checkpointing (or at least mitigate it) #3946

trivialfis · 2019-07-20T13:19:29Z

@trams @thesuperzapper I think this is an overview for everyone to have a feeling for what's coming next. It would be difficult to list everything coming since XGBoost is a community driven project. Just open a PR when it's ready.

Not a committer, but can we please target PySpark API for 1.0?

@thesuperzapper Let's track the progress. I certainly hope that I can start testing it. :-)

thesuperzapper · 2019-07-21T01:57:30Z

There is also the secondary consideration, that we might not be ready for 1.0, and the API guarantees that come with that, for example, we could instead do 0.10.0 next?

trivialfis · 2019-07-21T03:24:44Z

@thesuperzapper 1.0 is not gonna be a final version. It's just we are trying to do semantic versioning.

RAMitchell · 2019-07-23T02:56:51Z

Added some gpu related items.

chenqin · 2019-08-08T17:36:11Z

would like to get native xgb fix included.
#4753

trivialfis · 2019-08-12T15:59:00Z

JSON is removed from the list. See #4683 (comment)

thesuperzapper · 2019-08-16T02:49:07Z

I raised an issue for my above suggestion: #4781 (To remove the python Rabit tracker)

Daniel8hen · 2019-08-18T14:36:28Z

FeatureImportance in the Spark version will be great as well (i.e. easily have the feature Importance)
#988

trivialfis · 2019-08-21T18:49:00Z

Added regression test.

hcho3 · 2019-08-21T19:30:14Z

@chenqin I'd like to hear from you about regression tests, since you have experience with managing ML in production. Any suggestions?

chenqin · 2019-08-22T19:02:00Z

@chenqin I'd like to hear from you about regression tests, since you have experience with managing ML in production. Any suggestions?

I think we should cover regression test on various of workloads and benchmark against prediction accuracy and stability (equal or better) than previous version within approximate same time. Two candidates on top of my head are

https://archive.ics.uci.edu/ml/datasets/HIGGS

sparse Dmatrix
https://www.kaggle.com/c/ClaimPredictionChallenge

We can try various of tree methods and configurations to ensure good coverage

tree_method, configurations / dataset / standalone or cluster

declaimer:
I think it worth clarify a bit.

Release regression is not something we already done in the company I worked.
The data sets I proposed is arbitrary which may not used as benchmark to claim one framework better than another. (this is most concerning when I saw biased benchmarks from time to time)
In fact, the essence of tune and uncover proper features/settings have always been more important. Unfortunately we may not cover this in regression tests.

May be more organized plan is to build a automation tool where user can take and benchmark various settings against their private data-set and model in their own data center.

thesuperzapper · 2019-09-17T01:06:21Z

We should add fixing #4779 as a requirement to ship 1.0

codingforfun · 2019-09-26T14:22:49Z

I add #4899 as a cleanup step.

hcho3 · 2019-10-05T06:17:44Z

@dmlc/xgboost-committer Since we have quite a few tasks left for 1.0, maybe we should make an interim release 0.91?

thesuperzapper · 2019-10-05T15:21:08Z

@hcho3 Or perhaps 0.10.0

trivialfis · 2019-10-05T16:32:32Z

@thesuperzapper That will confuse version system. I don't mind a 0.91 release, but still I want to see proper procedures for regression tests.

thesuperzapper · 2019-10-05T17:08:41Z

@trivialfis If master has API changes, shouldn't we bump a major version, which I guess would look like 0.100.0

hcho3 · 2019-10-05T17:18:58Z

@thesuperzapper The 1.0.0 version is the first version we would adopt semantic versioning scheme, so no, semantic versioning won't apply to the interim release. It's a bit tricky, since we have quite a lot to do until 1.0.0 is released.

hcho3 · 2019-10-08T18:41:41Z

@CodingCat How about 0.100 or 0.95? "Preview" sounds like the 1.0.0 release is just around the corner, but we have quite a few major features (PySpark) on the line.

douglasren · 2019-10-09T21:07:01Z

Does it support weight xgboost ?

CodingCat · 2019-10-09T21:30:36Z

I am not worrying about the impression of 1.0.0 to users Spark 3.0 preview is releasing in this month, but formal release is next April (around spark summit) maybe

…

On Tue, Oct 8, 2019 at 11:41 AM Philip Hyunsu Cho ***@***.***> wrote: @CodingCat <https://github.com/CodingCat> How about 0.100 or 0.95? "Preview" sounds like the 1.0.0 release is just around the corner, but we have quite a few major features (PySpark) on the line. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4680?email_source=notifications&email_token=AAFFQ6AOGIWIB6W6TW3R5W3QNTH6TA5CNFSM4IE5CQGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAVF7MA#issuecomment-539647920>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFFQ6HF52HBR7ZNSKLIY3TQNTH6TANCNFSM4IE5CQGA> .

thesuperzapper · 2019-10-11T05:05:32Z

@CodingCat at least from the point of view of xgboost4j-spark, that 1.0.0 preview won't be useful for most people, as almost no one is running Spark on 2.12. Additionally, you can't easily get a compiled binary as https://spark.apache.org/downloads.html dosen't distribute compiled versions of Spark for 2.12 with the Hadoop binaries included.

CodingCat · 2019-10-11T05:07:38Z

Then we should release nothing?

…

On Thu, Oct 10, 2019 at 10:05 PM Mathew Wicks ***@***.***> wrote: @CodingCat <https://github.com/CodingCat> at least from the point of view of xgboost4j-spark, that 1.0.0 preview won't be useful for most people, as almost no one is running Spark on 2.12. Additionally, you can't easily get a compiled binary as https://spark.apache.org/downloads.html dosen't distribute compiled versions of Spark for 2.12 with the Hadoop binaries included. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4680?email_source=notifications&email_token=AAFFQ6AN3FJQ7ZE7EOTXLW3QOACSFA5CNFSM4IE5CQGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA6ZM2Q#issuecomment-540907114>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFFQ6EJRRMTNY7R7JVALTDQOACSFANCNFSM4IE5CQGA> .

hcho3 · 2019-10-11T05:09:45Z

@CodingCat @thesuperzapper I thought #4574 would allow for compiling XGBoost with both Scala 2.11 and 2.12? In that case, we should compile XGBoost with 2.11 and upload JAR to Maven.

trivialfis · 2019-10-11T12:05:45Z

Removed:

Release Gpu memory after training Allow releasing GPU memory #4668

I don't think we can get to there right now.

jkbradley · 2019-10-11T17:45:13Z

@thesuperzapper It will be come easier to develop against the Apache Spark master (3.0) branch and Scala 2.12 after Spark releases a 3.0 preview (targeted pretty soon this fall). I'd expect a much bigger shift to Scala 2.12 in the Spark community after the final 3.0 release (targeted early 2020), but you're right that there isn't a ton of 2.12 usage now. I created #4926 to solicit discussion around the upcoming Spark release.

trams · 2019-10-11T18:27:42Z

@CodingCat @thesuperzapper I thought #4574 would allow for compiling XGBoost with both Scala 2.11 and 2.12? In that case, we should compile XGBoost with 2.11 and upload JAR to Maven.

#4574 does not allow to cross compile.
What it allows is for someone to check out the code, manually override scala version and recompile

So someone may compile a jar with 2.11 and upload to Maven
I had a pull request with migration to SBT which would allow to cross compile
I also know the trick how to support a cross compilation in Maven (we used it in our company). I can share if you are interested

trivialfis · 2019-10-16T17:09:03Z

@hcho3 Is it possible to use CPack for easing the installation for OSX? Please ignore this comment if it's not possible.

douglasren · 2019-10-22T09:11:19Z

Does it support Multi objective learning?

trivialfis · 2019-10-22T09:29:20Z

@douglasren Sadly no. Could you start a new issue so we can discuss it? The term "multi objective" can vary depending on contexts, like one objective function for multiple outputs, multiple objectives with one output or multiple objectives with multiple outputs?

EricSpeidel · 2019-11-29T09:36:04Z

I would like to cast my vote towards an interim release as well.

hcho3 · 2019-12-23T07:42:57Z

#5146 fixes #4477.

trivialfis · 2019-12-23T10:27:18Z

Removed:

PySpark API support ([jvm-packages] PySpark Support Checklist #3370) ([jvm-packages] initial pyspark api (WIP) #4656) .

TylerADavis · 2020-01-08T00:35:09Z

An interim release would be great as the macOS installation is still a pain right now

dubeyrahul · 2020-01-16T08:13:53Z

Can we get documented support for learning to rank (pairwise) with XGBoost4J-Spark? Currently, there is no concrete solution to how to specify training data. There's some confusion around partitioning by groupID and training data needing to follow same partition strategy, but it's quite vague.
An example or clear documentation would be really helpful!

lucagiovagnoli · 2020-01-24T01:22:06Z

I'd like to cast my vote to an interim release as well. We're looking forward to the next version mostly for the missing value fix by @cpfarrell (see #4805).

Is there a time estimate related to the next release (major or interim)?

PS: @thesuperzapper we're using 2.11 and 2.12 and an interim release would be extremely helpful for us

trivialfis · 2020-01-30T08:34:38Z

@hcho3 Can we make create a release branch and have a week or so for testing?

hcho3 · 2020-01-30T14:47:55Z

Yes!

terrytangyuan · 2020-01-30T14:51:06Z

@hcho3 In addition to a branch, we can also make an official release candidate on GitHub Releases so that the community can have more confidence to test it as well.

lucagiovagnoli · 2020-01-30T20:16:06Z

This sounds awesome! Really looking forward to the next release. Let me know if we can help. We're definitely going to test it out at Yelp.

hcho3 · 2020-01-31T07:15:16Z

I will cut a new branch release_1.0.0 after #5248 is merged. Thanks everyone for your patience.

hcho3 · 2020-01-31T11:45:06Z

Release candidate is now available for Python: #5253. You can try it today by running

pip3 install xgboost==1.0.0rc1

hcho3 · 2020-02-20T05:52:11Z

1.0.0 is now out:

pip3 install xgboost==1.0.0

CodingCat added the type: roadmap label Jul 18, 2019

CodingCat pinned this issue Jul 18, 2019

hcho3 mentioned this issue Jul 22, 2019

Optimizations for CPU #4529

Merged

This was referenced Aug 6, 2019

remove gpu_exact tree method #4742

Merged

[BREAKING] prevent multi-gpu usage #4749

Merged

rongou mentioned this issue Aug 21, 2019

make HostDeviceVector single gpu only #4773

Merged

jakirkham mentioned this issue Aug 22, 2019

Next Release? #4799

Closed

hcho3 mentioned this issue Sep 5, 2019

[RFC] Callback interface for logging internal information, to aid debugging #4837

Closed

hcho3 mentioned this issue Oct 25, 2019

Impossible to reproduce model results #4989

Closed

hcho3 mentioned this issue Feb 19, 2020

[RFC] XGBoost 1.0.0 Release Candidate #5253

Closed

12 tasks

hcho3 closed this as completed Feb 20, 2020

hcho3 unpinned this issue Feb 21, 2020

lock bot locked as resolved and limited conversation to collaborators May 20, 2020

[Roadmap] XGBoost 1.0.0 Roadmap #4680

[Roadmap] XGBoost 1.0.0 Roadmap #4680

Comments

CodingCat commented Jul 18, 2019 • edited by trivialfis

thesuperzapper commented Jul 19, 2019

CodingCat commented Jul 19, 2019

thesuperzapper commented Jul 19, 2019

trams commented Jul 20, 2019

trivialfis commented Jul 20, 2019

thesuperzapper commented Jul 21, 2019

trivialfis commented Jul 21, 2019

RAMitchell commented Jul 23, 2019

chenqin commented Aug 8, 2019

trivialfis commented Aug 12, 2019

thesuperzapper commented Aug 16, 2019

Daniel8hen commented Aug 18, 2019

trivialfis commented Aug 21, 2019

hcho3 commented Aug 21, 2019

chenqin commented Aug 22, 2019 • edited

thesuperzapper commented Sep 17, 2019

codingforfun commented Sep 26, 2019

hcho3 commented Oct 5, 2019 • edited

thesuperzapper commented Oct 5, 2019 • edited

trivialfis commented Oct 5, 2019

thesuperzapper commented Oct 5, 2019

hcho3 commented Oct 5, 2019 • edited

hcho3 commented Oct 8, 2019

douglasren commented Oct 9, 2019

CodingCat commented Oct 9, 2019 via email

thesuperzapper commented Oct 11, 2019

CodingCat commented Oct 11, 2019 via email

hcho3 commented Oct 11, 2019 • edited

trivialfis commented Oct 11, 2019

jkbradley commented Oct 11, 2019

trams commented Oct 11, 2019

trivialfis commented Oct 16, 2019 • edited

douglasren commented Oct 22, 2019

trivialfis commented Oct 22, 2019 • edited

EricSpeidel commented Nov 29, 2019

hcho3 commented Dec 23, 2019

trivialfis commented Dec 23, 2019

TylerADavis commented Jan 8, 2020

dubeyrahul commented Jan 16, 2020

lucagiovagnoli commented Jan 24, 2020

trivialfis commented Jan 30, 2020

hcho3 commented Jan 30, 2020

terrytangyuan commented Jan 30, 2020

lucagiovagnoli commented Jan 30, 2020

hcho3 commented Jan 31, 2020

hcho3 commented Jan 31, 2020 • edited

hcho3 commented Feb 20, 2020

CodingCat commented Jul 18, 2019 •

edited by trivialfis

chenqin commented Aug 22, 2019 •

edited

hcho3 commented Oct 5, 2019 •

edited

thesuperzapper commented Oct 5, 2019 •

edited

hcho3 commented Oct 5, 2019 •

edited

hcho3 commented Oct 11, 2019 •

edited

trivialfis commented Oct 16, 2019 •

edited

trivialfis commented Oct 22, 2019 •

edited

hcho3 commented Jan 31, 2020 •

edited