Do you think it'll be useful to show a more informative warning for this context?

Yes, we can.

The current message (from Numpy, I believe) is:

"invalid value encountered in power"

I was thinking of something like this:

"Some values in the inverse-transformed data are NaN (column 0), which may be due to extreme skewness in the data. Consider addressing outliers before applying the transformation."

The code could be like this:

for i, lmbda in enumerate(self.lambdas_):
    with np.errstate(invalid="ignore"):
        X[:, i] = inv_fun(X[:, i], lmbda)
        if np.isnan(X[:, i]).any():
            # raise UserWarning if return NaNs
            warnings.warn(
                f"""Some values in the inverse-transformed data are NaN (column {i}), which
                may be due to extreme skewness in the data. Consider addressing outliers 
                before applying the transformation.""",
                UserWarning
            )```

Could we only check the nan values when the exception is raised to avoid a scan on the data? A try/except might be better then.

Something like this?

        for i, lmbda in enumerate(self.lambdas_):
            with np.errstate(invalid='warn'):
                with warnings.catch_warnings(record=True) as captured_warnings:
                    X[:, i] = inv_fun(X[:, i], lmbda)
            if captured_warnings and np.isnan(X[:, i]).any():
                    warnings.warn(
                        f"""Some values in column {i} of the inverse-transformed data are NaN. This may be due to 
                        extreme skewness or outliers in the data for this column. Consider addressing these issues, 
                        such as removing or imputing outliers, before applying the transformation.""",
                        UserWarning
                    )            

I believe the catch_warnings would fit better than the try-except because we want the results to be returned. I have also included the column number to help users identify where the issue is.

for i, lmda in enumerate(self.lambda_):
    with np.errstate(invalid="raise"):
        try:
            X[:, i] = inv_fun(X[:, i], lmbda)
        except Exception:
            if np.isnan(X[:, i]).any():
                warning.warn(...)

I think that we could still specialize the type of the Exception depending on what is raised. Also, is there any other "invalid" exception that would be raised or in other words, do we really need the check for the nan or can we just always raise the warning?

@glemaitre, I'm avoiding try-except because the code

X[:, i] = inv_fun(X[:, i], lmbda)

would have to execute twice, such as in a finally block. On the other hand, capturing and addressing the warnings ensures that the inv_fun function runs only once.

Running np.isnan(X[:, i]) creates a NumPy array of shape (n_sample, ), which adds memory overhead.

I know it's more brittle, but can we check that the NumPy warning message contains "invalid value encountered in power" and then raise our own warning?

I created a warning called TransformationFailedWarning and used it for a more straightforward check, as you suggested.

Sorry for being unclear. I mostly wanted to convert np.isnan(X[:, i]).any() to "invalid value encountered in power" in warning_message and leave everything else the same.

Can you revert the new warning object?

Done.

@maf-rnmourao the rst syntax is not quite right, would you be kind enough to open a PR with the following change 🙏:

By :user:`Roberto Mourao <maf-rnmourao>`

From the dev website:

Hi @lesteve ,

Here is the PR: #32093

Best Regards,