bug: inconsistent types accepted in DateLocator subclasses #3897

wholmgren · 2014-12-06T21:56:07Z

I've usually used np arrays, mostly out of habit, to construct my HourLocator and MinuteLocator objects, and today I was surprised to find that this did not work for MonthLocator objects. I also tested DayLocator and WeekdayLocator and found that they did not work with np arrays. The mpl docs suggest that the arguments can be "an int or sequence" -- not sure if this would include np arrays or not.

Not a big deal, but it did confuse me for a few minutes.

In [47]:

mpl.dates.DayLocator(np.arange(2,13,2))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-47-b23ea962d29f> in <module>()
----> 1 mpl.dates.DayLocator(np.arange(2,13,2))

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/matplotlib/dates.py in __init__(self, bymonthday, interval, tz)
   1097             bymonthday = list(xrange(1, 32))
   1098         o = rrulewrapper(DAILY, bymonthday=bymonthday,
-> 1099                          interval=interval, **self.hms0d)
   1100         RRuleLocator.__init__(self, o, tz)
   1101 

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/matplotlib/dates.py in __init__(self, freq, **kwargs)
    593         self._construct = kwargs.copy()
    594         self._construct["freq"] = freq
--> 595         self._rrule = rrule(**self._construct)
    596 
    597     def set(self, **kwargs):

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/dateutil/rrule.py in __init__(self, freq, dtstart, interval, wkst, count, until, bysetpos, bymonth, bymonthday, byyearday, byeaster, byweekno, byweekday, byhour, byminute, bysecond, cache)
    272                     raise ValueError("bysetpos must be between 1 and 366, "
    273                                      "or between -366 and -1")
--> 274         if not (byweekno or byyearday or bymonthday or
    275                 byweekday is not None or byeaster is not None):
    276             if freq == YEARLY:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

In [48]:

mpl.dates.HourLocator(np.arange(2,13,2))
Out[48]:
<matplotlib.dates.HourLocator at 0x7f05a2149470>
In [49]:

mpl.dates.MinuteLocator(np.arange(2,13,2))
Out[49]:
<matplotlib.dates.MinuteLocator at 0x7f05a1e941d0>
In [50]:

mpl.dates.MonthLocator(np.arange(2,13,2))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-50-971e82a6be3a> in <module>()
----> 1 mpl.dates.MonthLocator(np.arange(2,13,2))

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/matplotlib/dates.py in __init__(self, bymonth, bymonthday, interval, tz)
   1056             bymonth = list(xrange(1, 13))
   1057         o = rrulewrapper(MONTHLY, bymonth=bymonth, bymonthday=bymonthday,
-> 1058                          interval=interval, **self.hms0d)
   1059         RRuleLocator.__init__(self, o, tz)
   1060 

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/matplotlib/dates.py in __init__(self, freq, **kwargs)
    593         self._construct = kwargs.copy()
    594         self._construct["freq"] = freq
--> 595         self._rrule = rrule(**self._construct)
    596 
    597     def set(self, **kwargs):

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/dateutil/rrule.py in __init__(self, freq, dtstart, interval, wkst, count, until, bysetpos, bymonth, bymonthday, byyearday, byeaster, byweekno, byweekday, byhour, byminute, bysecond, cache)
    283                 byweekday = dtstart.weekday()
    284         # bymonth
--> 285         if not bymonth:
    286             self._bymonth = None
    287         elif isinstance(bymonth, integer_types):

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

In [51]:

mpl.dates.SecondLocator(np.arange(2,13,2))
Out[51]:
<matplotlib.dates.SecondLocator at 0x7f05a1f25d68>
In [52]:

mpl.dates.WeekdayLocator(np.arange(2,13,2))
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-52-94aba8a8506e> in <module>()
----> 1 mpl.dates.WeekdayLocator(np.arange(2,13,2))

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/matplotlib/dates.py in __init__(self, byweekday, interval, tz)
   1079         o = rrulewrapper(DAILY, byweekday=byweekday,
-> 1080                          interval=interval, **self.hms0d)
   1081         RRuleLocator.__init__(self, o, tz)
   1082 

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/matplotlib/dates.py in __init__(self, freq, **kwargs)
    593         self._construct = kwargs.copy()
    594         self._construct["freq"] = freq
--> 595         self._rrule = rrule(**self._construct)
    596 
    597     def set(self, **kwargs):

/home/will/.virtualenvs/dev3/lib/python3.4/site-packages/dateutil/rrule.py in __init__(self, freq, dtstart, interval, wkst, count, until, bysetpos, bymonth, bymonthday, byyearday, byeaster, byweekno, byweekday, byhour, byminute, bysecond, cache)
    347                 if isinstance(wday, integer_types):
    348                     self._byweekday.append(wday)
--> 349                 elif not wday.n or freq > MONTHLY:
    350                     self._byweekday.append(wday.weekday)
    351                 else:

AttributeError: 'numpy.int64' object has no attribute 'n'

In [53]:

mpl.dates.MonthLocator(np.arange(2,13,2).tolist())
Out[53]:
<matplotlib.dates.MonthLocator at 0x7f05a20a0ba8>

tacaswell · 2014-12-07T19:43:23Z

it should include arrays. Tagging as 1.5 as this should be dealt with as part of an over-all overhaul of the date handling system.

@cimarronm You have been working on the rrule code recently, is there a quick fix for this?

pganssle · 2014-12-27T05:14:06Z

As far as I can tell, there are two different things causing this problem. For DayLocator, MonthLocator and YearLocator, the problem the fact that numpy arrays don't have well-defined truth values.For WeekdayLocator, it's due to what I think is a bug in dateutils where they use type(wday) is int rather than isinstance(wday, int).

For DayLocator and MonthLocator, this is a consequence of the way that dateutils checks for the existence of certain parameters see L371 of rrule.py. They check the truth value of byweekno, bymonthday and bymonthday rather than checking if any of them is not None.

For YearLocator, the problem is actually a bit different - the error is raised in ticker.py:L1151, in the constructor for Base. In this case, it seems that YearLocator isn't designed to be specified with a sequence, and that it should throw an error when a sequence is passed to it, but because of the way that assert is designed, non-empty sequences actually satisfy the condition, which causes an error only when you try to render it.

I'm going to see if I can fix what I see as bugs in the dateutil code, but assuming that those are not in fact bugs (or that we want to support older versions of dateutil), for DayLocator, WeekdayLocator and MonthLocator, does anyone see any problem with just converting the numpy arrays into lists before constructing the rrules? For WeekdayLocator, we'd also need to cast them to int as well.

I think for now Base should be updated to detect (and reject) all sequences, and if date handling is to be overhauled, then YearLocator should be brought into line with the other constructors to allow sequences as inputs.

pganssle · 2014-12-27T05:25:30Z

Looks like the bug giving problems with WeekdayLocator is fixed in the version of dateutil that is on GitHub, so that's something.

pganssle · 2014-12-28T06:51:17Z

My PRs over at dateutil were accepted, and I tested this against what is now the master branch over there (as of this commit), and it resolves this issue, so eventually these issues will go away when matplotlib is built with I guess the next release of dateutil. If we want to support older versions of dateutil, like I said we can explicitly convert numpy arrays to lists of ints before passing them to rrule. If that's the way people want to go, I'd be happy to prepare a PR that does it, since I've been fiddling around with dates.py anyway lately.

WeatherGod · 2014-12-30T21:55:44Z

I think that makes sense to do (explicit conversion of numpy array to a
list of ints). I don't want to raise the dependency version for datetutils
when there is a simple enough workaround for us. Go ahead and submit a PR
for that (if you haven't already, as I am still wading through emails).

On Sun, Dec 28, 2014 at 1:51 AM, Paul G. notifications@github.com wrote:

My PRs over at dateutil were accepted, and I tested this against what is
now the master branch over there (as of this commit
dateutil/dateutil@ada102f),
and it resolves this issue, so eventually these issues will go away when
matplotlib is built with I guess the next release of dateutil. If we want
to support older versions of dateutil, like I said we can explicitly
convert numpy arrays to lists of ints before passing them to rrule. If
that's the way people want to go, I'd be happy to prepare a PR that does
it, since I've been fiddling around with dates.py anyway lately.

—
Reply to this email directly or view it on GitHub
#3897 (comment)
.

pganssle · 2014-12-30T23:06:11Z

@WeatherGod Since I already made a lot of changes related to date handling, I can make the change in PR #3947.

That said, I'm a bit ambivalent about doing this. It seems a bit strange to be doing dateutils' validation for it because of a bug in the current version. If I could think of any potential consequences other than the almost certainly negligible performance hit, I'd probably advocate just adding a note in the documentation that these specific locators don't accept numpy arrays when matplotlib was built with dateutils <= 2.3. Since we don't seem to be making any library calls ourselves using numpy arrays, this gives the user the choice of whether to switch the data type of their sequence or upgrade dateutils.

pganssle · 2014-12-31T00:54:27Z

OK, I've implemented it in the latest revision to #3947.

pganssle · 2016-03-17T16:07:31Z

By the way, this was fixed ages ago, both in matplotlib and dateutil. Here's a MWE, run with a deliberately old version of dateutil:

import matplotlib
import matplotlib.dates as m_dates
import numpy as np
import dateutil

print("Matplotlib version: {}".format(matplotlib.__version__))
print("Numpy version: {}".format(np.__version__))
print("Dateutil version: {}".format(dateutil.__version__))

print("DayLocator: Used to fail")
m_dates.DayLocator(np.arange(2, 13, 2))

print("HourLocator: Used to succeed")
m_dates.HourLocator(np.arange(2, 13, 2))

print("MonthLocator: Used to fail")
m_dates.MinuteLocator(np.arange(2, 13, 2))

print("SecondLocator: Used to succeed")
m_dates.SecondLocator(np.arange(2,13,2))

print("WeekdayLocator: Used to fail")
m_dates.WeekdayLocator(np.arange(2,13,2))

print("MonthLocator as list: Used to succeed")
m_dates.MonthLocator(np.arange(2,13,2).tolist())

print("All good!")

Results:

$ python test_issue3897.py 
Matplotlib version: 1.5.1
Numpy version: 1.10.4
Dateutil version: 2.2
DayLocator: Used to fail
HourLocator: Used to succeed
MonthLocator: Used to fail
SecondLocator: Used to succeed
WeekdayLocator: Used to fail
MonthLocator as list: Used to succeed
All good!

Does this need tests before the issue can be closed?

tacaswell · 2016-03-17T16:11:59Z

Nope, this is good enough! Thanks for going through old issues.

tacaswell added API: consistency topic: date handling labels Dec 7, 2014

tacaswell added this to the v1.5.x milestone Dec 7, 2014

tacaswell closed this as completed Mar 17, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bug: inconsistent types accepted in DateLocator subclasses #3897

bug: inconsistent types accepted in DateLocator subclasses #3897

wholmgren commented Dec 6, 2014

tacaswell commented Dec 7, 2014

pganssle commented Dec 27, 2014

pganssle commented Dec 27, 2014

pganssle commented Dec 28, 2014

WeatherGod commented Dec 30, 2014

pganssle commented Dec 30, 2014

pganssle commented Dec 31, 2014

pganssle commented Mar 17, 2016

tacaswell commented Mar 17, 2016

bug: inconsistent types accepted in DateLocator subclasses #3897

bug: inconsistent types accepted in DateLocator subclasses #3897

Comments

wholmgren commented Dec 6, 2014

tacaswell commented Dec 7, 2014

pganssle commented Dec 27, 2014

pganssle commented Dec 27, 2014

pganssle commented Dec 28, 2014

WeatherGod commented Dec 30, 2014

pganssle commented Dec 30, 2014

pganssle commented Dec 31, 2014

pganssle commented Mar 17, 2016

tacaswell commented Mar 17, 2016