Skip to content

FIX: bar mixed units, allow ValueError as well #13187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 21, 2019

Conversation

jklymak
Copy link
Member

@jklymak jklymak commented Jan 15, 2019

PR Summary

Closes #13186

Follow up to #12903 where we try and do math first for bar (x+dx) before units conversion. Pandas throws a ValueError for a previously working case. This adds that to the allowable excepts...

PR Checklist

  • Has Pytest style unit tests
  • Code is Flake 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

@jklymak
Copy link
Member Author

jklymak commented Jan 15, 2019

QT tests are failing - not this PR

NelleV
NelleV previously requested changes Jan 16, 2019
Copy link
Member

@NelleV NelleV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test that the error is not being raised when using pandas?

@jklymak
Copy link
Member Author

jklymak commented Jan 16, 2019

100% agree that pandas datetimes + bar should be tested.

OTOH, I'm not 100% convinced its matplotlib that should be responsible for those tests if the pandas datetime unit converters are registered, which they are in this case. See #11664 for where I propose we explictly unregister pandas datetime unit conversion so then we can test our own converters (which work with basic pandas objects without the need for their converters). This was prompted by import pandas no longer registering their converters by default. They backed down on that change, but may re-impliment it at some point.

Not to say pandas' own converters shouldn't work - but it seems that they should be responsible for reporting breakages to us (like this one). However, I'm open to the idea we should fully test pandas if that is how it should be done.

For this particular case, I'm pretty ignorant about how the user got to the point where bar worked at all (#13186), so I'm hesitant to add to the tests. But if someone knoweledgeable of pandas wanted to add or propose a test, I'm more than happy for that to go in...

@NelleV
Copy link
Member

NelleV commented Jan 16, 2019

Have we reported the breakage to pandas? If that is the case, we may want to simply add a test and a comment that this should be fixed in pandas version XXX, and remove the work around and and the test when times come.
Does that seem reasonable ?

@jklymak
Copy link
Member Author

jklymak commented Jan 17, 2019

Hmm, looking at it again, I guess pandas thinks its a ValueError because self.freq is None.

In any case, not convinced this needs a test from our end, but the change should go in to make the except clause here as liberal as possible. In fact I'd argue its should be a bare except - if adding convert(x+dx) fails for any reason give us the old behaviour of just calling convert(dx)

@jklymak jklymak force-pushed the fix-pandas-datetime branch from fd5647b to 293c529 Compare January 17, 2019 17:57
@jklymak
Copy link
Member Author

jklymak commented Jan 17, 2019

... ah apparently except Exception is the way to do this cleanly without a dangerous bare exception. Hopefully that is deemed OK by folks...

@@ -2057,7 +2057,7 @@ def _convert_dx(dx, x0, xconv, convert):
dx = [convert(x0 + ddx) - x for ddx in dx]
if delist:
dx = dx[0]
except (TypeError, AttributeError) as e:
except Exception:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please leave a comment justifying (briefly) the nonspecific catch-everything behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't that catch everything?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle I'm ok with except Exception:, but for targetted code. That's a pretty big try block to be catching with Exception--a lot of unexpected behavior could happen in those lines. Is it possible to just wrap the try around dx = [convert(x0 + ddx) - x for ddx in dx]?

FIX: make exception in bar for convert(x+dx) more liberal
@jklymak jklymak force-pushed the fix-pandas-datetime branch from 293c529 to e25f9f7 Compare January 17, 2019 19:36
@NelleV
Copy link
Member

NelleV commented Jan 17, 2019

The current solution catches all exceptions, which we should avoid. It means will accidentally catch errors and exception we don't want to catch. I don't think the proposed solution is reasonable (sorry!)

@NelleV
Copy link
Member

NelleV commented Jan 17, 2019

(I need to run, but I'll review this more carefully in the next day or so)

@jklymak
Copy link
Member Author

jklymak commented Jan 17, 2019

@NelleV I'm happy to go back to explicit exception types, but except Exceptions is the pep-8 way to make exception as liberal as possible:

A bare except: clause will catch SystemExit and KeyboardInterrupt exceptions, making it harder to interrupt a program with Control-C, and can disguise other problems. If you want to catch all exceptions that signal program errors, use except Exception: (bare except is equivalent to except BaseException:).

so I thought this would be OK.

https://www.python.org/dev/peps/pep-0008/

@NelleV
Copy link
Member

NelleV commented Jan 18, 2019

@jklymak But making catching exception as liberal as possible is not good practice. It means the code "works", in cases where it should probably fail. It actually happened to me several time with Matplotlib that the plot would show and be wrong due to broad error catching such as this one.

I'll dismiss my review, as I am about to board a plane and will be offline for the next 24h, but I think it is better to be as narrow and explicit as possible when catching errors.

@NelleV NelleV dismissed their stale review January 18, 2019 03:17

Not available in the near future to review this PR.

@jklymak
Copy link
Member Author

jklymak commented Jan 19, 2019

Tightened exception

@dopplershift I can't make the try-except block shorter because the error is sometimes in the procedure that gets the first element.

Just to be clear - the idea is "try some stuff to get this to work. If that fails just do what we did before this PR". So having this try/except block makes things a bit better at no cost (other than code complexity).

OTOH I don't feel passionately about this solution. If someone has a better solution for bar that would be great.

@dopplershift
Copy link
Contributor

@jklymak That's fine--but then for a block that big, I agree with @NelleV that we should be as specific as possible in the exceptions caught. IMHO except Exception: is only ok when there's one specific line that you're catching errors from.

@jklymak
Copy link
Member Author

jklymak commented Jan 21, 2019

@dopplershift sure, but is the new version OK? It has three explicit errors.

Copy link
Contributor

@dopplershift dopplershift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jklymak Sorry, new version is fine. Was holding off on review due to failing CI and lack of time to dig in.

@dopplershift dopplershift merged commit a9b4fd5 into matplotlib:master Jan 21, 2019
@NelleV
Copy link
Member

NelleV commented Jan 21, 2019

That PR was missing a test. I'll create a PR to fix this.

@NelleV NelleV mentioned this pull request Jan 21, 2019
@dopplershift
Copy link
Contributor

Oops. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ax.bar throws when x axis is pandas datetime
6 participants