Skip to content

time series plotting gap causes strange jump #16266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Fealthas opened this issue Jan 19, 2020 · 5 comments
Closed

time series plotting gap causes strange jump #16266

Fealthas opened this issue Jan 19, 2020 · 5 comments
Labels
status: closed as inactive Issues closed by the "Stale" Github Action. Please comment on any you think should still be open. status: inactive Marked by the “Stale” Github Action third-party integration topic: date handling

Comments

@Fealthas
Copy link

Fealthas commented Jan 19, 2020 •
edited
Loading

Bug report

Gap in time series data causes MPL to jump to a very small datetime that doesnt exist in the dataset.
image

If I don't manually set the axis, it tries to autofit.

ax1.set_xlim( [ min(datana.date.dt.to_pydatetime()) , max(datana.date.dt.to_pydatetime()) ])

Autofitting looks like this:
image

Note that the minimum x datetime in the dataset is in 2019! Not sure where the 1680ish time comes from. I've manually checked what actually gets passed into mpl, a numpy object array of datetime objects, does not have a time value below 2019.

Abbreviated code:

import datetime
import time
import matplotlib
matplotlib.use("agg")
import matplotlib.pyplot as plt
import pandas as pd

fig1, ax1 = plt.subplots( figsize=(15,5) )

    data = pd.read_csv("temps.dat",header=None, names=['date','temp'])
    data.temp = data.temp/1000
    data.date = pd.to_datetime(data.date)
    data.date = data.date.dt.ceil(freq='s') #strip microseconds
    data['roll'] = data['temp'].rolling(window=722 ).mean()
    data['roll2'] = data['temp'].rolling(window=14220  ).mean()
    datana = data.dropna(subset=['roll'])
    datana2 = data.dropna(subset=['roll2'])

    ax1.cla()
 
    ax1.plot(datana2.date.dt.to_pydatetime()  ,datana2.roll2, lw=3, alpha=.8)
    ax1.plot(datana.date.dt.to_pydatetime()  ,datana.roll, lw=.7)

    ax1.set_xlim( [ min(datana.date.dt.to_pydatetime()) , max(datana.date.dt.to_pydatetime()) ])
fig1.autofmt_xdate()
fig1.savefig("fig1.png",format='png' ,bbox_inches='tight', dpi=500)

temps.zip
Matplotlib version

  • Operating system: debian (raspbian)

  • Matplotlib version: 3.1.2 , in a venv, installed by pip

  • Matplotlib backend (print(matplotlib.get_backend())): agg

  • Python version: 3.7

  • Other libraries: pandas, Pillow
@jklymak
Copy link
Member

jklymak commented Jan 19, 2020

You will get faster turnaround on this if you can provide something more minimal with a data set you can type out rather than providing it as a zip file. In producing that minimal example you may discover the specific problem and narrow down if the problem is in matplotlib or your data. Even better would be to not require pandas. Thanks!

@anntzer
Copy link
Contributor

anntzer commented Jan 20, 2020 •
edited
Loading

There are some NaT rows in your dataset. I guess old pandas (which I don't have around) would convert these to some nonsensical value when using toordinal() (which is what we use internally in dates._to_ordinalf); pandas 1.0(pre) now raises a ValueError here. I guess we could detect NaT and convert it to nan here... (pandas-dev/pandas#10372?)

Edit: Actually I think the change is due to pandas-dev/pandas#18720.

@Fealthas
Copy link
Author

Fixing the offending NaT fixed the issue.

I guess the real bug here is that neither pandas nor mpl threw an error for an invalid value and just attempted to finish what it started no matter what.

@github-actions
Copy link

github-actions bot commented Jul 7, 2023

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

@github-actions github-actions bot added the status: inactive Marked by the “Stale” Github Action label Jul 7, 2023
@github-actions github-actions bot added the status: closed as inactive Issues closed by the "Stale" Github Action. Please comment on any you think should still be open. label Aug 7, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 7, 2023
@tacaswell
Copy link
Member

I am comfortable leaving this closed as it was fixed upstream by pandas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status: closed as inactive Issues closed by the "Stale" Github Action. Please comment on any you think should still be open. status: inactive Marked by the “Stale” Github Action third-party integration topic: date handling
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tacaswell @anntzer @jklymak @Fealthas