Skip to content

Datetime documentation draft #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
288 changes: 288 additions & 0 deletions doc/source/reference/arrays.datetime.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,288 @@
.. currentmodule:: numpy

.. _arrays.datetime:

************************
Datetimes and Timedeltas
************************

.. versionadded:: 1.7.0

Starting in NumPy 1.7, there are core array data types which natively
support datetime functionality. The data type is called "datetime64",
so named because "datetime" is already taken by the datetime library
included in Python.

Basic Datetimes
===============

The most basic way to create datetimes is from strings in
ISO 8601 date or datetime format. The unit for internal storage
is automatically selected from the form of the string, and can
be either a :ref:`date unit <arrays.dtypes.dateunits>` or a
:ref:`time unit <arrays.dtypes.timeunits>`. The date units are years ('Y'),
months ('M'), weeks ('W'), and days ('D'), while the time units are
hours ('h'), minutes ('m'), seconds ('s'), milliseconds ('ms'), and
more SI-prefix seconds-based units.

.. admonition:: Example

A simple ISO date:

>>> np.datetime64('2005-02-25')
numpy.datetime64('2005-02-25')

Using months for the unit:

>>> np.datetime64('2005-02')
numpy.datetime64('2005-02')

Specifying just the month, but forcing a 'days' unit:

>>> np.datetime64('2005-02', 'D')
numpy.datetime64('2005-02-01')

Using UTC "Zulu" time:

>>> np.datetime64('2005-02-25T03:30Z')
numpy.datetime64('2005-02-24T21:30-0600')

ISO 8601 specifies to use the local time zone
if none is explicitly given:

>>> np.datetime64('2005-02-25T03:30')
numpy.datetime64('2005-02-25T03:30-0600')

When creating an array of datetimes from a string, it is still possible
to automatically select the unit from the inputs, by using the
datetime type with generic units.

.. admonition:: Example

>>> np.array(['2007-07-13', '2006-01-13', '2010-08-13'], dtype='datetime64')
array(['2007-07-13', '2006-01-13', '2010-08-13'], dtype='datetime64[D]')

>>> np.array(['2001-01-01T12:00', '2002-02-03T13:56:03.172'], dtype='datetime64')
array(['2001-01-01T12:00:00.000-0600', '2002-02-03T13:56:03.172-0600'], dtype='datetime64[ms]')


The datetime type works with many common NumPy functions, for
example :meth:`arange` can be used to generate ranges of dates.

.. admonition:: Example

All the dates for one month:

>>> np.arange('2005-02', '2005-03', dtype='datetime64[D]')
array(['2005-02-01', '2005-02-02', '2005-02-03', '2005-02-04',
'2005-02-05', '2005-02-06', '2005-02-07', '2005-02-08',
'2005-02-09', '2005-02-10', '2005-02-11', '2005-02-12',
'2005-02-13', '2005-02-14', '2005-02-15', '2005-02-16',
'2005-02-17', '2005-02-18', '2005-02-19', '2005-02-20',
'2005-02-21', '2005-02-22', '2005-02-23', '2005-02-24',
'2005-02-25', '2005-02-26', '2005-02-27', '2005-02-28'],
dtype='datetime64[D]')

The datetime object represents a single moment in time. If two
datetimes have different units, they may still be representing
the same moment of time, and converting from a bigger unit like
months to a smaller unit like days is considered a 'safe' cast
because the moment of time is still being represented exactly.

.. admonition:: Example

>>> np.datetime64('2005') == np.datetime64('2005-01-01')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that the year in this and similar example corresponds to the earliest time compatible with both sides of the comparison.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really 'compatible with', more 'equal by definition'.

True

>>> np.datetime64('2010-03-14T15Z') == np.datetime64('2010-03-14T15:00:00.00Z')
True

An important exception to this rule is between datetimes with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'date units' and 'time units' should probably be introduced up near the beginning and the distinction explained.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about just splitting up the table of units in the section below into two tables, one for each type?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would make one table rather small. Maybe star the lower case types, or explain why they are lower case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the change to the table, I think your idea of two tables would look better.

:ref:`date units <arrays.dtypes.dateunits>` and datetimes with
:ref:`time units <arrays.dtypes.timeunits>`. This is because this kind
of conversion generally requires a choice of timezone and
particular time of day on the given date.

.. admonition:: Example

>>> np.datetime64('2003-12-25', 's')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Cannot parse "2003-12-25" as unit 's' using casting rule 'same_kind'

>>> np.datetime64('2003-12-25') == np.datetime64('2003-12-25T00Z')
False


Datetime and Timedelta Arithmetic
=================================

NumPy allows the subtraction of two Datetime values, an operation which
produces a number with a time unit. Because NumPy doesn't have a physical
quantities system in its core, the timedelta64 data type was created
to complement datetime64.

Datetimes and Timedeltas work together to provide ways for
simple datetime calculations.

.. admonition:: Example

>>> np.datetime64('2009-01-01') - np.datetime64('2008-01-01')
numpy.timedelta64(366,'D')

>>> np.datetime64('2009') + np.timedelta64(20, 'D')
numpy.datetime64('2009-01-21')

>>> np.datetime64('2011-06-15T00:00') + np.timedelta64(12, 'h')
numpy.datetime64('2011-06-15T12:00-0500')

>>> np.timedelta64(1,'W') / np.timedelta64(1,'D')
7.0

There are two Timedelta units, years and months, which are treated
specially, because how much time they represent changes depending
on when they are used. While a timedelta day unit is equivalent to
24 hours, there is no way to convert a month unit into days, because
different months have different numbers of days.

.. admonition:: Example

>>> a = np.timedelta64(1, 'Y')

>>> np.timedelta64(a, 'M')
numpy.timedelta64(12,'M')

>>> np.timedelta64(a, 'D')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Cannot cast NumPy timedelta64 scalar from metadata [Y] to [D] according to the rule 'same_kind'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line break needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, I guess the 'doctest' thing isn't taken very seriously anyway...


Datetime Units
==============

The Datetime and Timedelta data types support a large number of time
units, as well as generic units which can be coerced into any of the
other units based on input data.

Datetimes are always stored based on POSIX time (though having a TAI
mode which allows for accounting of leap-seconds is proposed), with
a epoch of 1970-01-01T00:00Z. This means the supported dates are
always a symmetric interval around 1970.

Here are the date units:

.. _arrays.dtypes.dateunits:

======== ================ ======================= ==========================
Date unit Time span Time span (years)
------------------------- ----------------------- --------------------------
Code Meaning Relative Time Absolute Time
======== ================ ======================= ==========================
Y year +- 9.2e18 years [9.2e18 BC, 9.2e18 AD]
M month +- 7.6e17 years [7.6e17 BC, 7.6e17 AD]
W week +- 1.7e17 years [1.7e17 BC, 1.7e17 AD]
D day +- 2.5e16 years [2.5e16 BC, 2.5e16 AD]
======== ================ ======================= ==========================

And here are the time units:

.. _arrays.dtypes.timeunits:

======== ================ ======================= ==========================
Time unit Time span Time span (years)
------------------------- ----------------------- --------------------------
Code Meaning Relative Time Absolute Time
======== ================ ======================= ==========================
h hour +- 1.0e15 years [1.0e15 BC, 1.0e15 AD]
m minute +- 1.7e13 years [1.7e13 BC, 1.7e13 AD]
s second +- 2.9e12 years [ 2.9e9 BC, 2.9e9 AD]
ms millisecond +- 2.9e9 years [ 2.9e6 BC, 2.9e6 AD]
us microsecond +- 2.9e6 years [290301 BC, 294241 AD]
ns nanosecond +- 292 years [ 1678 AD, 2262 AD]
ps picosecond +- 106 days [ 1969 AD, 1970 AD]
fs femtosecond +- 2.6 hours [ 1969 AD, 1970 AD]
as attosecond +- 9.2 seconds [ 1969 AD, 1970 AD]
======== ================ ======================= ==========================
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if you add 1as to "2011-6-30' you get an overflow error. This unit doesn't seem that useful except for elapsed time from some separately specified epoch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silent overflow, fully compatible with the silent overflow in all of NumPy's integer types. ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, silent overflow seems a bit dangerous for business. Actually, I did get an error:

OverflowError: Integer overflow getting a common metadata divisor for NumPy datetime metadata [D] and [as]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then this

In [18]: datetime64('2011-06-30T00Z', 'as')
Out[18]: numpy.datetime64('1969-12-31T16:59:51.731375747389980672-0700')

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, I don't think business stuff will accidentally use attoseconds, though.

I've added it to my TODO list though


Business Day Functionality
==========================

To allow the datetime to be used in contexts where accounting for weekends
and holidays is important, NumPy includes a set of functions for
working with business days.

The function :meth:`busday_offset` allows you to apply offsets
specified in business days to datetimes with a unit of 'day'. By default,
a business date is defined to be any date which falls on Monday through
Friday, but this can be customized with a weekmask and a list of holidays.

.. admonition:: Example

>>> np.busday_offset('2011-06-23', 1)
numpy.datetime64('2011-06-24')

>>> np.busday_offset('2011-06-23', 2)
numpy.datetime64('2011-06-27')

When an input date falls on the weekend or a holiday,
:meth:`busday_offset` first applies a rule to roll the
date to a valid business day, then applies the offset. The
default rule is 'raise', which simply raises an exception.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the example, the default rule looks like 'forward'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which example? I see an example raising an exception.

Maybe an offset of 0 instead of 2 in that example would help.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where's the exception? I assume that the exception is only raised if the start date isn't a business day.

The rules most typically used are 'forward' and 'backward'.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to think of it in terms of what the function does:

  1. Apply the roll rule
  2. Apply the offset to the date that has been rolled

I tried to explain it in the paragraph, but maybe you have ideas to make it more clear.

Negative offsets work as one would expect.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I saw that further down. That's why I deleted my comment.


.. admonition:: Example

>>> np.busday_offset('2011-06-25', 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: Non-business day date in busday_offset
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the exception


>>> np.busday_offset('2011-06-25', 0, roll='forward')
numpy.datetime64('2011-06-27')

>>> np.busday_offset('2011-06-25', 2, roll='forward')
numpy.datetime64('2011-06-29')

>>> np.busday_offset('2011-06-25', 0, roll='backward')
numpy.datetime64('2011-06-24')

>>> np.busday_offset('2011-06-25', 2, roll='backward')
numpy.datetime64('2011-06-28')

In some cases, an appropriate use of the roll and the offset
is necessary to get a desired answer.

.. admonition:: Example

The first business day on or after a date:

>>> np.busday_offset('2011-03-20', 0, roll='forward')
numpy.datetime64('2011-03-21','D')
>>> np.busday_offset('2011-03-22', 0, roll='forward')
numpy.datetime64('2011-03-22','D')

The first business day strictly after a date:

>>> np.busday_offset('2011-03-20', 1, roll='backward')
numpy.datetime64('2011-03-21','D')
>>> np.busday_offset('2011-03-22', 1, roll='backward')
numpy.datetime64('2011-03-23','D')

The function is also useful for computing some kinds of days
like holidays. In Canada and the U.S., Mother's day is on
the second Sunday in May, which can be computed with a special
weekmask.

.. admonition:: Example

>>> np.busday_offset('2012-05', 1, roll='forward', weekmask='Sun')
numpy.datetime64('2012-05-13','D')

When performance is important for manipulating many business date
with one particular choice of weekmask and holidays, there is
an object :class:`busdaycalendar` which stores the data necessary
in an optimized form.

The other two functions for business days are :meth:`is_busday`
and :meth:`busday_count`, which are more straightforward and
not explained here.
1 change: 1 addition & 0 deletions doc/source/reference/arrays.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,4 @@ of also more complicated arrangements of data.
arrays.classes
maskedarray
arrays.interface
arrays.datetime
11 changes: 1 addition & 10 deletions numpy/core/numerictypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,10 @@
int_, uint,
longlong, ulonglong,


single, csingle,
float_, complex_,
longfloat, clongfloat,


datetime_
timedelta_, (this inherits from from signedinteger, as it is
a signed integer with an associated time unit)


As part of the type-hierarchy: xx -- is bit-width

generic
Expand Down Expand Up @@ -405,9 +398,7 @@ def _set_up_aliases():
('longcomplex', 'clongdouble'),
('bool_', 'bool'),
('unicode_', 'unicode'),
('object_', 'object'),
('timedelta_', 'timedelta'),
('datetime_', 'datetime')]
('object_', 'object')]
if sys.version_info[0] >= 3:
type_pairs.extend([('bytes_', 'string'),
('str_', 'unicode'),
Expand Down
2 changes: 1 addition & 1 deletion numpy/core/tests/test_regression.py
Original file line number Diff line number Diff line change
Expand Up @@ -1306,7 +1306,7 @@ def test_fromstring_crash(self):
def test_ticket_1539(self):
dtypes = [x for x in np.typeDict.values()
if (issubclass(x, np.number)
and not issubclass(x, np.timedelta_))]
and not issubclass(x, np.timedelta64))]
a = np.array([], dtypes[0])
failures = []
for x in dtypes:
Expand Down