Enhance piecewise to allow functions to have arbitrary number of input arguments #5187

pbrod · 2014-10-14T21:52:16Z

Made piecewise function simpler and more general. The new piecewise function allows functions of the type f(x0, x1,.. ,xn) i.e. to have arbitrary number of input arguments that are evaluated conditionally.

The old piecewise function did allow condlist to be a single bool list/array or a single int array
(That feature was not documented in the docstring of the function, but there were tests testing this functionality.) Question is if this feature should be deprecated. For now this feature is kept in the new version.

See also: http://www.mail-archive.com/numpy-discussion@scipy.org/msg46304.html

Examples

Examples that work both in previous and current version

>>> x = np.linspace(-2,2,5)
>>> np.piecewise(x, [x<1], [1]) 
array([ 1.,  1.,  1.,  0.,  0.])

>>> np.piecewise((x,), [x < 0, x >= 0], [lambda x: -x, lambda x: x])
array([ 2.,  1.,  0.,  1.,  2.])

can be written as

>>> np.piecewise(x,[x < 0, x >= 0], [lambda x: -x, lambda x: x])
array([ 2.,  1.,  0.,  1.,  2.])

or

>>> np.piecewise(x, [x < 0,], [lambda x: -x, lambda x: x])
array([ 2.,  1.,  0.,  1.,  2.])

Example that still works but should possibly be deprecated:

>>> np.piecewise(x, x < 0, [lambda x: -x, lambda x: x])
array([ 2.,  1.,  0.,  1.,  2.])

Examples demonstrating the new functionality

>>> X,Y = np.meshgrid(x,x)
>>> np.piecewise((X,Y), [X*Y<0,], [lambda x,y: -x*y, lambda x,y: x*y])
array([[ 4.,  2., -0.,  2.,  4.],
       [ 2.,  1., -0.,  1.,  2.],
       [-0., -0.,  0.,  0.,  0.],
       [ 2.,  1.,  0.,  1.,  2.],
       [ 4.,  2.,  0.,  2.,  4.]])

>>> np.piecewise((X,Y),[X*Y<-0.5, X*Y>0.5], [lambda x,y: -x*y, lambda x,y: x*y, np.nan])
array([[  4.,   2.,  nan,   2.,   4.],
       [  2.,   1.,  nan,   1.,   2.],
       [ nan,  nan,  nan,  nan,  nan],
       [  2.,   1.,  nan,   1.,   2.],
       [  4.,   2.,  nan,   2.,   4.]])

charris · 2015-01-23T15:32:10Z

We can't break backward compatibility here. Either a new function is needed, or the enhanced function needs to be fixed to work with old arguments.

pbrod · 2015-01-27T06:46:45Z

According to the docstring of the old function:

 condlist : list of bool arrays
        Each boolean array corresponds to a function in `funclist`.  Wherever
    `    condlist[i]` is True, `funclist[i](x)` is used as the output value.

Thus the example

x = np.linspace(-2,2,5)
np.piecewise(x, x < 0, [lambda x: -x, lambda x: x])

violates the condition that condlist should be a list. Here condlist is a ndarray. The old function allowed this calling syntax, but it is clearly an undocumented feature of the old function. The question is if that is a wanted behavior?

For now I have allowed this behavior and removed any backward incompatibilties as far as I know. The only thing I have added is a deprecation warning if the condlist is not a list of booleans or boolean arrays.

homu · 2015-06-20T01:49:28Z

☔ The latest upstream changes (presumably #5985) made this pull request unmergeable. Please resolve the merge conflicts.

pbrod · 2015-07-13T11:06:43Z

This version of the 'piecewice' function resolves the problems mentioned in ticket #5729 and #5737 as well as #4792.

Added regression tests for #5729 and #5737.

homu · 2015-07-26T05:17:27Z

☔ The latest upstream changes (presumably #6115) made this pull request unmergeable. Please resolve the merge conflicts.

homu · 2016-02-01T02:04:18Z

☔ The latest upstream changes (presumably #7145) made this pull request unmergeable. Please resolve the merge conflicts.

hunterakins · 2018-10-31T04:16:26Z

Seems this didn't merge...would be a useful enhancement. Any update on this?

eric-wieser · 2018-10-31T04:22:36Z

numpy/lib/function_base.py

@@ -838,7 +838,19 @@ class ndarray is returned.
    return a


-def piecewise(x, condlist, funclist, *args, **kw):
+def valarray(shape, value=0.0, typecode=None):


This function is just a less reliable np.full - and it looks like it's not used anyway

Can you explain how you can substitute np.piecewise with np.full?
Or, to avoid the xy problem: How would one write a conditional function that works cleanly with both 1-d numpy arrays as well as the 2-d array outputs from meshgrid?

I'm commenting on valarray, not piecewise

eric-wieser · 2018-10-31T04:28:04Z

@hunterakins: The problem is that piecewise((x1, x2. x3), ...) is already valid syntax, and is the same as np.piecewise(np.array([x1, x2, x3])). We can't change that behavior without at least a deprecation cycle.

The best way forward would be to introduce a FutureWarning when x is a tuple, and then maybe in 1.18 the new behavior can be introduced

hunterakins · 2018-10-31T04:31:48Z

That's alright. I still don't understand how that has to do with np.full or defining conditional functions that work on meshgrid 2D arrays.

…

On Tue, Oct 30, 2018 at 9:28 PM Eric Wieser ***@***.***> wrote: @hunterakins <https://github.com/hunterakins>: The problem is that piecewise((x1, x2. x3), ...) is already valid syntax, and is the same as np.piecewise(np.array([x1, x2, x3])). We can't change that behavior without at least a deprecation cycle. The best way forward would be to introduce a FutureWarning when x is a tuple, and then maybe in 1.18 the new behavior can be introduced — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5187 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVO9O4tTEUhSK3AlF2eVis26twdq0aLBks5uqSbcgaJpZM4Cus3K> .

anirudh2290 · 2020-05-21T18:17:25Z

@pbrod This has been open for a while. Do you plan to come back to this ?

pbrod · 2020-05-21T23:04:39Z

@anirudh2290 Well, I am happy to complete this, but I need guidance on how to proceed.

anirudh2290 · 2020-06-01T21:36:00Z

@pbrod According to me, looks like this enhancement would be a good to have.
As for the further direction, as @eric-wieser suggests we should first deprecate the tuple input which is just converted to a single ndarray and add a FutureWarning for the same. We can add this new behavior after 2 releases.

Thus the example

x = np.linspace(-2,2,5)
np.piecewise(x, x < 0, [lambda x: -x, lambda x: x])
violates the condition that condlist should be a list. Here condlist is a ndarray. The old function
allowed this calling syntax, but it is clearly an undocumented feature of the old function. The > question is if that is a wanted behavior?

Since single condition is promoted to a list of one condition, I won't mind keeping it as it is and documenting this behavior.

cc @eric-wieser for a second look.

eric-wieser · 2020-06-01T22:17:09Z

Can we come up with a new spelling that doesn't overlap the old one?

As a reminder, we currently have:

np.piecewise(
    arr_like,
    [cond1, cond2, cond3],
    [func1, func2, func3, func_default],
    *func_args, **func_kwargs)

One option might be pulling some magic with __getitem__:

np.piecewise[2](
    arr_like1,
    arr_like2,
    [cond1, cond2, cond3],
    [func1, func2, func3, func_default],
    *func_args, **func_kwargs)

or by currying (noting that 2-argument invocations are currently not allowed):

np.piecewise(
    [cond1, cond2, cond3],
    [func1, func2, func3, func_default])(arr_like1, arr_like2)

Will edit this post if I think of more...

eric-wieser

Looks like there are more behavior changes here than I first spotted

eric-wieser · 2020-06-01T22:53:17Z

numpy/lib/function_base.py


        The length of `condlist` must correspond to that of `funclist`.
        If one extra function is given, i.e. if
        ``len(funclist) - len(condlist) == 1``, then that extra function
        is the default value, used wherever all conditions are false.
-    funclist : list of callables, f(x,*args,**kw), or scalars
+    funclist : list of callables, f(*(xi + args), **kw), or scalars


Suggested change

funclist : list of callables, f(*(xi + args), **kw), or scalars

funclist : list of callables, f(*xi, *args, **kw), or scalars

is also legal and probably clearer

xi may be slightly misleading since these funcs run on indexed array as such which will be 1d and may not be all elements of xi.

eric-wieser · 2020-06-01T22:54:15Z

numpy/lib/function_base.py

+    # corresponding condlist as a boolean mask. This is done in reverse
+    # order since the first choice should take precedence.


xref #16475, this is a behavior change too

eric-wieser · 2020-06-01T22:56:18Z

numpy/lib/function_base.py

+    if deprecated_ints:
+        msg = "piecewise condlists containing integer ndarrays is deprecated " \
+            "and will be removed in the future. Use `.astype(bool)` to " \
+            "convert to bools."
+        warnings.warn(msg, DeprecationWarning)


This change is probably sensible to extract to its own PR. Note this is missing stacklevel=2.

eric-wieser · 2020-06-01T22:57:58Z

numpy/lib/tests/test_function_base.py

        assert_(y.ndim == 0)
        assert_(y == 0)

        x = 5
        y = piecewise(x, [[True], [False]], [1, 0])
-        assert_(y.ndim == 0)
+        assert_(y.ndim == 1)


This is a breaking change as well sadly.

rgommers added 01 - Enhancement component: numpy.lib labels Mar 8, 2015

pbrod force-pushed the master branch from 3384d44 to 8644801 Compare July 13, 2015 10:32

pbrod closed this Sep 30, 2015

pbrod force-pushed the master branch from 5e059a7 to ddc5388 Compare September 30, 2015 13:27

pbrod reopened this Oct 2, 2015

eric-wieser reviewed Oct 31, 2018

View reviewed changes

seberg added the 55 - Needs work label Apr 26, 2019

anirudh2290 added the 61 - Stale label May 21, 2020

eric-wieser reviewed Jun 1, 2020

View reviewed changes

simfinite mentioned this pull request Aug 4, 2020

Better Piecewise printer for NumPy sympy/sympy#15014

Open

Base automatically changed from master to main March 4, 2021 02:03

pbrod closed this May 10, 2021

pbrod force-pushed the master branch from 6189877 to 7548dc0 Compare May 10, 2021 02:48

pbrod mentioned this pull request May 10, 2021

ENH: Update piecewise to handle multivariate functions #18979

Open

	funclist : list of callables, f((xi + args), *kw), or scalars
	funclist : list of callables, f(xi, args, **kw), or scalars

		# corresponding condlist as a boolean mask. This is done in reverse
		# order since the first choice should take precedence.

Uh oh!

Enhance piecewise to allow functions to have arbitrary number of input arguments #5187

Enhance piecewise to allow functions to have arbitrary number of input arguments #5187

Uh oh!

Conversation

pbrod commented Oct 14, 2014

Examples

Examples that work both in previous and current version

Example that still works but should possibly be deprecated:

Examples demonstrating the new functionality

Uh oh!

charris commented Jan 23, 2015

Uh oh!

pbrod commented Jan 27, 2015

Uh oh!

homu commented Jun 20, 2015

Uh oh!

pbrod commented Jul 13, 2015

Uh oh!

homu commented Jul 26, 2015

Uh oh!

homu commented Feb 1, 2016

Uh oh!

hunterakins commented Oct 31, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eric-wieser commented Oct 31, 2018

Uh oh!

hunterakins commented Oct 31, 2018 via email

Uh oh!

anirudh2290 commented May 21, 2020

Uh oh!

pbrod commented May 21, 2020

Uh oh!

anirudh2290 commented Jun 1, 2020

Uh oh!

eric-wieser commented Jun 1, 2020

Uh oh!

eric-wieser left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!