Reference Issues/PRs

Fixes #15077. See also #10856.

What does this implement/fix? Explain your changes.

This implements general naive Bayes (GeneralNB) in addition to the existing naive Bayes implementations like GaussianNB and BernoulliNB.

This implementation allows multiple assumptions on the features, namely the Bernoulli, Gaussian, Multinomial, and Categorical distributions. In the API, the user will be able to specify these distributions and their respective features.

I have divided this description into 3 sections as below:

1. Design and usage
2. Under the hood
3. Runtime checks

1. Design and usage

The design of the API is similar to that of ColumnTransformer and Pipeline. To specify that columns 0-2 and 3-4 are to be modelled with Gaussian and categorical naive Bayes respectively, indicate these in the GeneralNB constructor and fit accordingly:

>>> clf = GeneralNB(models=[
...     ("gaussian", GaussianNB(), [0, 1, 2]),
...     ("categorical", CategoricalNB(), [3, 4])
... ])
>>> clf.fit(X, y)

It also accepts a list of strings of column names if the data to be fitted are pandas DataFrames:

>>> clf = GeneralNB([
...     ("gaussian", GaussianNB(), ["a", "b", "c"]),
...     ("categorical", CategoricalNB(), ["d", "e"])
... ])

Lastly, similar to ColumnTransformer, it also accepts callables like make_column_selector to specify DataFrame columns:

>>> from sklearn.compose import make_column_selector
>>> clf = GeneralNB([
...     ("gaussian", GaussianNB(), make_column_selector(pattern=r"[abc]")),
...     ("categorical", CategoricalNB(), make_column_selector(pattern=r"[de]"))
... ])

The attributes of the fitted estimators can be accessed using the self.named_models_ attribute. For example, to access the theta_ parameter of the bernoulli model,

clf.named_models_.bernoulli.theta_

2. Under the hood

For the GeneralNB.predict() function, we sum the _joint_log_likelihood() for each naive Bayes estimator, then subtract (n-1) log P(c) from this sum. Here is a pseudocode:

jlls = [model._joint_log_likelihood() for model in models]
jlls = jlls - log_prior
jll = jlls.sum(axis=0) + log_prior

3. Runtime checks

Check self.models:

• Duplicate specification of column is not tolerated.
• No. of cols specified must match the no. of cols in X.

Checks on parameter consistency across naive Bayes estimators are performed to ensure that specific parameters across the estimators stay the same. Otherwise, the calculation of the joint log likelihood will be wrong. Such parameters are:

• class_prior* or priors^
• fit_prior*
• class_log_prior* or class_prior^

*used in BernoulliNB, MultinomialNB, ComplementNB, CategoricalNB
^used in GaussianNB

Data checks:
Methods like _check_X_y() and _check_X() check if the data type used during fitting is the same during prediction (i.e. NumPy array and pandas DataFrame).

4. Others

Partial fitting is not supported.
Currently, it is not okay to leave some columns out.

Progress:

• API design
• Main code
• Docs
• Tests

Any other comments?

PR submitted.