Skip to content

[WIP] Monotonic features in tree-based models #7266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

[WIP] Monotonic features in tree-based models #7266

wants to merge 4 commits into from

Conversation

pat-oreilly
Copy link
Contributor

@pat-oreilly pat-oreilly commented Aug 27, 2016

Reference Issue

#6656

What does this implement/fix?

Support for monotonically increasing or decreasing features in decision trees, random forests and gradient boosted trees.

Any other comments?

Tasks

  • DecisionTreeRegressor
  • DecisionTreeClassifier
  • ExtraTreeClassifier
  • ExtraTreeRegressor
  • RandomForestClassifier
  • RandomForestRegressor
  • ExtraTreesClassifier
  • ExtraTreesRegressor
  • RandomTreesEmbedding
  • GradientBoostingClassifier
  • GradientBoostingRegressor
  • Test runtime when no monotonicity specified and consider a separate loop
  • Refactor monotone check into a function
  • Finalize public API and argument error handling
  • Decide if splitter="random"should honour monotonic constraints and verify behaviour
  • Check if the divisor criterion.weighted_n_<left/right> can ever be 0.0 and handle it. Answer: it's OK.
  • Finalize docstrings
  • Add tests
  • Satisfy coding guidelines

@nelson-liu
Copy link
Contributor

fwiw, criterion.weighted_n_<left/right>can never be 0. Technically it's set to 0 in some functions but attains a positive nonzero value by the end of the function (so nonzero for all intents and purposes)

@pat-oreilly
Copy link
Contributor Author

Thanks @nelson-liu!

@jnothman
Copy link
Member

Just a note that we're focusing on a release now. If it's quiet around
here, please ping when 0.18 is out.

On 29 August 2016 at 05:46, PatrickOReilly notifications@github.com wrote:

Thanks @nelson-liu https://github.com/nelson-liu!


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#7266 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEz68He_PmSo3rR7VFM2tMrjuqy35Evks5qkeWUgaJpZM4JuuVb
.

@amueller
Copy link
Member

amueller commented Sep 6, 2016

tests are unhappy ;)

@pat-oreilly
Copy link
Contributor Author

Thanks @amueller, I think the tests are better now but the Travis build fails with sklearn.tree._splitter.Splitter has the wrong size, try recompiling. Expected 184, got 176. For me, running python setup.py clean --all before the build resolves this. Is this expected behaviour?

I'm running some tests to see if the runtime is affected when monotonic_constraint == 0. In the meantime, do you have any thoughts on the public API? I've gone with DecisionTreeRegressor(..., increasing=[0,3], decreasing=[2,5]). Currently the arguments must be python lists, perhaps they should rather be 'array-like'?

@amueller
Copy link
Member

the test error is because of travis caching. I'll try to fix it. I think you should accept array-likes, i.e. anything that can be converted to an array. You should validate them with the check_array function.

@amueller
Copy link
Member

hm for some reason I don't see the cache for this PR on travis :-/

@dfagnan
Copy link

dfagnan commented Mar 14, 2017

any update on this @amueller? seems like monotonicity pull-requests have been close for over a year but not quite getting merged

@JasonSanchez
Copy link

This would be a great feature to merge.

@jnothman
Copy link
Member

jnothman commented Jan 9, 2019

@PatrickOReilly are you still working on this?

@aldanor
Copy link

aldanor commented Feb 28, 2019

Would anyone care to pick this up?

@samronsin
Copy link
Contributor

I'm planning on having a closer look at this.

@NicolasHug
Copy link
Member

@samronsin Just in case, depending on how important it is to support this for random forests, you might want to try implementing this for the new GBDTs instead (#12807)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.