axes.bar fails when x is int-indexed pandas.Series #15162

infrub · 2019-08-31T10:18:03Z

Bug report

Bug summary

The following codes fail in matplotlib 3.1.1, while work properly in 3.0.3

case 1

Code for reproduction

import pandas as pd
from matplotlib import pyplot as plt

df = pd.DataFrame({"x":[1,2,3],"width":[.2,.4,.6]},index=[1,2,3])

plt.figure()
plt.bar(df.x, 1, width=df.width)
plt.show()

Actual outcome

Traceback (most recent call last):
  File "/Desktop/fail_example.py", line 7, in <module>
    plt.bar(df.x, 1, width=df.width)
  File "/.pyenv/versions/3.7.3/lib/python3.7/site-packages/matplotlib/pyplot.py", line 2440, in bar
    **({"data": data} if data is not None else {}), **kwargs)
  File "/.pyenv/versions/3.7.3/lib/python3.7/site-packages/matplotlib/__init__.py", line 1601, in inner
    return func(ax, *map(sanitize_sequence, args), **kwargs)
  File "/.pyenv/versions/3.7.3/lib/python3.7/site-packages/matplotlib/axes/_axes.py", line 2430, in bar
    label='_nolegend_',
  File "/.pyenv/versions/3.7.3/lib/python3.7/site-packages/matplotlib/patches.py", line 707, in __init__
    Patch.__init__(self, **kwargs)
  File "/.pyenv/versions/3.7.3/lib/python3.7/site-packages/matplotlib/patches.py", line 89, in __init__
    self.set_linewidth(linewidth)
  File "/.pyenv/versions/3.7.3/lib/python3.7/site-packages/matplotlib/patches.py", line 368, in set_linewidth
    self._linewidth = float(w)

Expected outcome
Outcome in 3.0.3:

case 2

Code for reproduction

import pandas as pd
from matplotlib import pyplot as plt

df = pd.DataFrame({"x":[1,3,10]},index=[1000,2000,3000])

plt.figure()
plt.bar(df.x, 1, width=.2)
plt.show()

Actual outcome

Expected outcome
Outcome in 3.0.3:

Matplotlib version

Operating system: MacOSX
Matplotlib version: 3.1.1 (installed from pip)
Matplotlib backend (print(matplotlib.get_backend())): MacOSX
Python version: 3.7.3
Jupyter version (if applicable):
Other libraries: pandas 0.24.2

Cause of the bug

_axes.py, line 2363 in bar:
            x0 = x
            x = np.asarray(self.convert_xunits(x))
            width = self._convert_dx(width, x0, x, self.convert_xunits)
_axes.py, line 2166 in _convert_dx: 
            try:
                x0 = x0[0]
            except (TypeError, IndexError, KeyError):
                x0 = x0

I guess x0 is expected to be a scalar after this line, However when x0 is an int-indexed (and 0 is not in its indices) pandas.Series, it is continued to be pandas.Series and causes chained disorders to end in the error above.
To evidence, they worked properly when rewritten as (ugly!):

            try:
                x0 = x0[0]
            except (TypeError, IndexError, KeyError):
                try:
                    x0 = x0.iat[0]
                except:
                    x0 = x0

The text was updated successfully, but these errors were encountered:

ImportanceOfBeingErnest · 2019-08-31T11:59:10Z

This came in with #12903.

Worth noting that plt.bar(df.x, 1, width=.1) also produces wrong output (though there is no error in that case).

jklymak · 2019-08-31T15:25:49Z

Sorry for the bug. I guess we could just put the old messy code back in, but it'd be nice if someone who understood pandas data types and inheritance figured out the proper solution. Obviously if you remove the index argument this works, or changing the index to zero-indexing. Is iat the way to get the first element in the series regardless of how its been indexed? Its inelegant of us to have special pandas magic all over our code, so it would be nice if something turned x into a proper numpy array earlier.

ImportanceOfBeingErnest · 2019-08-31T15:44:01Z

Apparently .iat is preferred over .iloc.

jklymak · 2019-08-31T16:06:44Z

Yeah but why doesn’t df.x[0] return the first element?

ImportanceOfBeingErnest · 2019-08-31T16:12:45Z

For the same reason {1 : True}[0] raises an error I suppose. The confusing bit is just that df.index[0] actually works, because an index behaves differently than a series.

jklymak · 2019-08-31T17:05:12Z

OK, well, this comes back to the units parsing. We can't just np.asarray because the way some packages implement units, this kills the units information, and we have no good way to get the units out to save for later. But, if we don't do this, then pandas doesn't work if someone manually sets the indices because x[0] no longer works. Yes, we could keep special casing every possible way data gets passed to functions, or we could come up with a few rules that we always apply to everything that comes in.

The obvious stopgap is to just put in the x.iat[0], but again, I really think that its bad engineering to have triple-nested try/except blocks in the code as we try and guess what kind of data we have been passed. So I'll keep this open to hopefully engender discussion.

ImportanceOfBeingErnest · 2019-08-31T20:18:26Z

Is there any alternative to finding the type of the first element? The culprit seems to be the unit package that subclasses numpy.ndarray, right? So unless that package has some other means to know what the array contains, one would have no choice but to stick to the currently solution. And in that case, special-casing pandas series is the only way out. Instead of try..except one could also use hasattr(x, "iat"), or "series" in str(type(x)).split["."] or so.

jklymak · 2019-08-31T21:45:59Z

I think I have a solution that squares all the constraints, but avoids using a library-specific method like iat. I’ll post later tonight.

tacaswell · 2019-09-01T16:20:16Z

Yeah but why doesn’t df.x[0] return the first element?

It is better to think of data frames as "dicts of Series with a bunch of helper methods" and Series as "ordered dicts of values with an optional index based lookup and helper methods".

I have lobbied Jeff Reback to get the behavior of [] change to be positional (aka integer) by default, but a) that would cause an insane amount of code breakage b) there is a big constituency of pandas users who have non-trivial indexes and benefit from s.loc[k] / s.at[k] (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.loc.html the "label" based one) being fast-pathed rather and s.iloc[k] / s.iat[k]. The extra tricky thing is if your labels happen to be integers it looks like they behave the same. Everyone has questionable early design decision that they can not back out of 🤷‍♀️ .

infrub changed the title ~~Axes._convert_dx(dx,x0,x,convert) fails when x0 is an int-indexed pandas.Series~~ bar(Axes._convert_dx(dx,x0,x,convert) fails when x0 is an int-indexed pandas.Series Aug 31, 2019

infrub changed the title ~~bar(Axes._convert_dx(dx,x0,x,convert) fails when x0 is an int-indexed pandas.Series~~ axes.bar fails when x and width are int-indexed pandas.Series Aug 31, 2019

infrub changed the title ~~axes.bar fails when x and width are int-indexed pandas.Series~~ axes.bar fails when x is int-indexed pandas.Series Aug 31, 2019

ImportanceOfBeingErnest added the third-party integration label Aug 31, 2019

ImportanceOfBeingErnest added this to the v3.1.2 milestone Aug 31, 2019

jklymak added the topic: units and array ducktypes label Aug 31, 2019

jklymak mentioned this issue Aug 31, 2019

FIX: indexed pandas bar #15166

Merged

6 tasks

dstansby closed this as completed in #15166 Sep 2, 2019

jklymak mentioned this issue Sep 16, 2019

Axes.violinplot has small issue in using pandas.DataFrame without index 0. #15272

Closed

This was referenced Sep 26, 2019

pyplot plot raises IndexError when x or y are pandas Index objects #15342

Closed

Bar width expands between subsequent bars #15351

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

axes.bar fails when x is int-indexed pandas.Series #15162

axes.bar fails when x is int-indexed pandas.Series #15162

infrub commented Aug 31, 2019 •

edited

Loading

ImportanceOfBeingErnest commented Aug 31, 2019 •

edited

Loading

jklymak commented Aug 31, 2019 •

edited

Loading

ImportanceOfBeingErnest commented Aug 31, 2019

jklymak commented Aug 31, 2019

ImportanceOfBeingErnest commented Aug 31, 2019

jklymak commented Aug 31, 2019

ImportanceOfBeingErnest commented Aug 31, 2019 •

edited

Loading

jklymak commented Aug 31, 2019

tacaswell commented Sep 1, 2019

axes.bar fails when x is int-indexed pandas.Series #15162

axes.bar fails when x is int-indexed pandas.Series #15162

Comments

infrub commented Aug 31, 2019 • edited Loading

Bug report

Bug summary

case 1

case 2

ImportanceOfBeingErnest commented Aug 31, 2019 • edited Loading

jklymak commented Aug 31, 2019 • edited Loading

ImportanceOfBeingErnest commented Aug 31, 2019

jklymak commented Aug 31, 2019

ImportanceOfBeingErnest commented Aug 31, 2019

jklymak commented Aug 31, 2019

ImportanceOfBeingErnest commented Aug 31, 2019 • edited Loading

jklymak commented Aug 31, 2019

tacaswell commented Sep 1, 2019

infrub commented Aug 31, 2019 •

edited

Loading

ImportanceOfBeingErnest commented Aug 31, 2019 •

edited

Loading

jklymak commented Aug 31, 2019 •

edited

Loading

ImportanceOfBeingErnest commented Aug 31, 2019 •

edited

Loading