I have been using Python 2.7.13 on a windows machine to write my code. But I am now trying to run my code on a Unix cluster using Python 2.7.12-goolf-2015a. When I run my code on the cluster with 2.7.12, I encounter an error in the pandas function to_datetime which does not occur using 2.7.13.
The error occurs when I try to convert a date entries in a dataframe to datetime.
df = pd.DataFrame({'year': [2015, 2016],
'month': [2, 3],
'day': [4, 5]})
pd.to_datetime(df)
Here is an example ot the error I get with the to_datetime command
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/easybuild/software/Python/2.7.12-goolf-
2015a/lib/python2.7/site-packages/pandas-0.16.1-py2.7-linux-
x86_64.egg/pandas/tseries/tools.py", line 343, in to_datetime
return _convert_listlike(arg, box, format)
File "/usr/local/easybuild/software/Python/2.7.12-goolf-
2015a/lib/python2.7/site-packages/pandas-0.16.1-py2.7-linux-
x86_64.egg/pandas/tseries/tools.py", line 266, in _convert_listlike
if com.is_datetime64_ns_dtype(arg):
File "/usr/local/easybuild/software/Python/2.7.12-goolf-
2015a/lib/python2.7/site-packages/pandas-0.16.1-py2.7-linux-
x86_64.egg/pandas/core/common.py", line 2513, in is_datetime64_ns_dtype
tipo = _get_dtype(arr_or_dtype)
File "/usr/local/easybuild/software/Python/2.7.12-goolf-
2015a/lib/python2.7/site-packages/pandas-0.16.1-py2.7-linux-
x86_64.egg/pandas/core/common.py", line 2458, in _get_dtype
return arr_or_dtype.dtype
File "/usr/local/easybuild/software/Python/2.7.12-goolf-
2015a/lib/python2.7/site-packages/pandas-0.16.1-py2.7-linux-
x86_64.egg/pandas/core/generic.py", line 2083, in __getattr__
(type(self).__name__, name))
AttributeError: 'DataFrame' object has no attribute 'dtype'
The to_datetime works when I load Python/3.5.2-intel-2016.u3, but that causes other problems. I am inexperienced with Unix operating systems and as I don’t have admin rights to software on the cluster, I am pretty tied to using 2.7.12
2 Answers
This works for me on the current version of pandas:
pd.to_datetime(df)
0 2015-02-04
1 2016-03-05
dtype: datetime64[ns]
Pandas to_datetime has special support to parse a datetime column from a supplied DataFrame input iff it contains specifically "year", "month" and "day" columns (order doesn't matter, and case sensitivity doesn't either).
It's likely you're working with a much older version of pandas in which case I'd recommend an upgrade!
However...
If upgrading isn't an option, you can always join the columns and call to_datetime on the Series.
Try joining the columns and then applying pd.to_datetime.
pd.to_datetime(df[['year', 'month', 'day']].astype(str).apply('-'.join, 1))
0 2015-02-04
1 2016-03-05
dtype: datetime64[ns]
DataFrame.dtypes is an attribute to list data types, for series it's a dtype.
reference:
