Skip to content

[MRG] MNT Initialize histograms in parallel and don't call np.zero in Hist-GBDT #18341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 4, 2020
Merged

[MRG] MNT Initialize histograms in parallel and don't call np.zero in Hist-GBDT #18341

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
15eb966
parallel init
NicolasHug Sep 4, 2020
50a78cf
whatsnew
NicolasHug Sep 4, 2020
e7665d6
docs
NicolasHug Sep 4, 2020
e915206
Add comment
NicolasHug Sep 4, 2020
ebef2f5
comment + added Egor
NicolasHug Sep 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions doc/whats_new/v0.24.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,13 @@ Changelog
usage in `fit`. :pr:`18334` by `Olivier Grisel`_ `Nicolas Hug`_, `Thomas
Fan`_ and `Andreas Müller`_.

- |Efficiency| Histogram initialization is now done in parallel in
:class:`ensemble.HistGradientBoostingRegressor` and
:class:`ensemble.HistGradientBoostingClassifier` which results in speed
improvement for problems that build a lot of nodes on multicore machines.
:pr:`18341` by `Olivier Grisel`_, `Nicolas Hug`_, `Thomas Fan`_, and
:user:`Egor Smirnov <SmirnovEgorRu>`.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crediting @SmirnovEgorRu here since you opened a similar PR a while ago #14380


- |API|: The parameter ``n_classes_`` is now deprecated in
:class:`ensemble.GradientBoostingRegressor` and returns `1`.
:pr:`17702` by :user:`Simona Maggio <simonamaggio>`.
Expand Down
11 changes: 9 additions & 2 deletions sklearn/ensemble/_hist_gradient_boosting/histogram.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,8 @@ cdef class HistogramBuilder:
G_H_DTYPE_C [::1] gradients = self.gradients
G_H_DTYPE_C [::1] ordered_hessians = self.ordered_hessians
G_H_DTYPE_C [::1] hessians = self.hessians
hist_struct [:, ::1] histograms = np.zeros(
# Histograms will be initialized to zero later within a prange
hist_struct [:, ::1] histograms = np.empty(
shape=(self.n_features, self.n_bins),
dtype=HISTOGRAM_DTYPE
)
Expand Down Expand Up @@ -177,6 +178,12 @@ cdef class HistogramBuilder:
self.ordered_hessians[:n_samples]
unsigned char hessians_are_constant = \
self.hessians_are_constant
unsigned int bin_idx = 0

for bin_idx in range(self.n_bins):
histograms[feature_idx, bin_idx].sum_gradients = 0.
histograms[feature_idx, bin_idx].sum_hessians = 0.
histograms[feature_idx, bin_idx].count = 0

if root_node:
if hessians_are_constant:
Expand Down Expand Up @@ -227,7 +234,7 @@ cdef class HistogramBuilder:
cdef:
int feature_idx
int n_features = self.n_features
hist_struct [:, ::1] histograms = np.zeros(
hist_struct [:, ::1] histograms = np.empty(
shape=(self.n_features, self.n_bins),
dtype=HISTOGRAM_DTYPE
)
Expand Down