Cannot plot bar graph with dates: "TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')" #13142

derekhuizhang · 2019-01-10T05:18:29Z

Bug report

Bug summary

Unable to plot bar graph. Plotting a line graph works fine.

Code for reproduction

from boto3.dynamodb.conditions import Key, Attr
import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
import matplotlib.dates as mdate
from datetime import datetime

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame(table.query(KeyConditionExpression=Key('partition_key').eq(name))['Items'])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
plt.bar(df['date'], df['activity'].values)
plt.show();

Actual outcome

TypeError                                 Traceback (most recent call last)
<ipython-input-13-f28db9680a8e> in <module>()
     26 
     27 plt.figure(figsize=(20,10))
---> 28 plt.bar(df['date'], df['activity'].values)

c:\python27\lib\site-packages\matplotlib\pyplot.pyc in bar(*args, **kwargs)
   2773                       mplDeprecation)
   2774     try:
-> 2775         ret = ax.bar(*args, **kwargs)
   2776     finally:
   2777         ax._hold = washold

c:\python27\lib\site-packages\matplotlib\__init__.pyc in inner(ax, *args, **kwargs)
   1865                         "the Matplotlib list!)" % (label_namer, func.__name__),
   1866                         RuntimeWarning, stacklevel=2)
-> 1867             return func(ax, *args, **kwargs)
   1868 
   1869         inner.__doc__ = _add_data_doc(inner.__doc__,

c:\python27\lib\site-packages\matplotlib\axes\_axes.pyc in bar(self, *args, **kwargs)
   2262         if align == 'center':
   2263             if orientation == 'vertical':
-> 2264                 left = x - width / 2
   2265                 bottom = y
   2266             elif orientation == 'horizontal':

TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')

Expected outcome

Expected: a plot of the (date, activity) in bar graph form. Changing plt.bar to plt.plot, as shown below, works perfectly fine:

from boto3.dynamodb.conditions import Key, Attr
import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
import matplotlib.dates as mdate
from datetime import datetime

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame(table.query(KeyConditionExpression=Key('partition_key').eq(name))['Items'])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
**plt.plot(df['date'], df['activity'].values)**
plt.show();

Matplotlib version

Operating system: Windows 10
Matplotlib version: 2.2.3
Matplotlib backend (print(matplotlib.get_backend())): module://ipykernel.pylab.backend_inline
Python version: 2.7

The text was updated successfully, but these errors were encountered:

jklymak · 2019-01-10T06:21:18Z

Can you make this self contained? Also, I suspect this has been fixed. But see #12862 for potentially similar issue.

derekhuizhang · 2019-01-10T06:43:44Z

Try this:

import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pdf import PdfPages
import matplotlib.dates as mdate
from datetime import datetime

import matplotlib

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame([{
    'timestamp': 1500854400000,
    'activity': 5},
{
    'timestamp': 1500940800000,
    'activity': 6
}])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
plt.bar(df['date'], df['activity'])
plt.xlabel('date')
plt.ylabel('activity')
plt.legend()
plt.show();

I don't think it has been fixed. I'm using the latest version (2.2.3) and it's the same issue, but it doesn't work for me.

jklymak · 2019-01-10T15:04:20Z

I’ll look at it, but 3.0.2 is the latest version.

jklymak · 2019-01-10T17:08:08Z

The default width for a bar is a float width=0.8, which as pointed out in the error message can't be added to a numpy.datetime64 in a sensible manner. The fix is to specify the width in the appropriate units:

plt.bar(df['date'], df['activity'], width=np.timedelta64(12, 'h'), )

I think there is a question here about whether Matplotlib should do something about this - i.e. if the data on the xaxis is not a float, what width should bar use by default? But then the question is what value do you ascribe to the width?

I think a second question is if matplotlib could supply a better error if the xaxis units are not numbers and width is not specified. My tendency would be to do this rather than try to guess what the user wants.

timhoffm · 2019-01-10T17:37:36Z

+1 for the better error message. Probably something like

try:
    left = x - width / 2
except TypeError as e:
    raise ValueError('x and width have incompatible types') from e

jklymak · 2019-01-10T17:55:53Z

Yeah, I looked at doing that quickly. My only hesitation is if that is particularly more helpful than:

c:\python27\lib\site-packages\matplotlib\axes\_axes.pyc in bar(self, *args, **kwargs)
   2262         if align == 'center':
   2263             if orientation == 'vertical':
-> 2264                 left = x - width / 2
   2265                 bottom = y
   2266             elif orientation == 'horizontal':

TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')

timhoffm · 2019-01-10T18:20:01Z

Well, maybe if we do

Parameters x and width have incompatible types

That explicitly tells the user to go and change the parameters. Whereas in the original traceback, he first has to realize that x - width is the problem and then has to know that width comes from a parameter (which he might not be aware of if he's using the default).

jklymak · 2019-01-10T18:36:57Z

@timhoffm Thats a good point, and agree with that as a good reason for a better error message

derekhuizhang · 2019-01-10T21:30:31Z

import pandas as pd
import json
import decimal 
import matplotlib.pyplot as plt
from matplotlib.backends.backend_pdf import PdfPages
import matplotlib.dates as mdate
from datetime import datetime

import matplotlib

def timestamp_to_datetime(raw):
    return datetime.fromtimestamp(raw/1000)

df = pd.DataFrame([{
    'timestamp': 1500854400000,
    'activity': 5},
{
    'timestamp': 1500940800000,
    'activity': 6
}])
df['date'] = df['timestamp'].apply(timestamp_to_datetime) 

plt.figure(figsize=(20,10))
plt.bar(df['date'], df['activity'], width=np.timedelta64(24, 'h'))
plt.xlabel('date')
plt.ylabel('activity')
plt.legend()
plt.show();

This works. Thanks so much!

jklymak added topic: date handling API: argument checking labels Jan 10, 2019

jklymak mentioned this issue Jan 11, 2019

DOC: better argcheck bar #13162

Merged

6 tasks

jklymak closed this as completed in #13162 Jan 17, 2019

QuLogic added this to the v3.1 milestone Jan 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot plot bar graph with dates: "TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')" #13142

Cannot plot bar graph with dates: "TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')" #13142

derekhuizhang commented Jan 10, 2019 •

edited

Loading

jklymak commented Jan 10, 2019 •

edited

Loading

derekhuizhang commented Jan 10, 2019 •

edited

Loading

jklymak commented Jan 10, 2019

jklymak commented Jan 10, 2019

timhoffm commented Jan 10, 2019

jklymak commented Jan 10, 2019

timhoffm commented Jan 10, 2019 •

edited

Loading

jklymak commented Jan 10, 2019

derekhuizhang commented Jan 10, 2019

Cannot plot bar graph with dates: "TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')" #13142

Cannot plot bar graph with dates: "TypeError: ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('float64')" #13142

Comments

derekhuizhang commented Jan 10, 2019 • edited Loading

Bug report

jklymak commented Jan 10, 2019 • edited Loading

derekhuizhang commented Jan 10, 2019 • edited Loading

jklymak commented Jan 10, 2019

jklymak commented Jan 10, 2019

timhoffm commented Jan 10, 2019

jklymak commented Jan 10, 2019

timhoffm commented Jan 10, 2019 • edited Loading

jklymak commented Jan 10, 2019

derekhuizhang commented Jan 10, 2019

derekhuizhang commented Jan 10, 2019 •

edited

Loading

jklymak commented Jan 10, 2019 •

edited

Loading

derekhuizhang commented Jan 10, 2019 •

edited

Loading

timhoffm commented Jan 10, 2019 •

edited

Loading