pd.DataFrame.from_csv Fehler im Zusammenhang mit infer_datetime_format

Question

Ich versuche, eine CSV-Datei in einen Pandas-Datenrahmen zu lesen. Es enthält eine Mischung aus Zahlen, Strings und Daten. Ich habe versucht:

trades_df = pd.DataFrame.from_csv('trades.csv') 

und bekam den Fehler:

--------------------------------------------------------------------------- 
ParserError        Traceback (most recent call last) 
<ipython-input-10-af86bdcdc851> in <module>() 
     1 # read a database from CSV and load it into a pandas dataframe 
----> 2 trades_df = pd.DataFrame.from_csv('trades.csv', infer_datetime_format=True) 
     3 trades_df.head() 

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/core/frame.py in from_csv(cls, path, header, sep, index_col, parse_dates, encoding, tupleize_cols, infer_datetime_format) 
    1249       parse_dates=parse_dates, index_col=index_col, 
    1250       encoding=encoding, tupleize_cols=tupleize_cols, 
-> 1251       infer_datetime_format=infer_datetime_format) 
    1252 
    1253  def to_sparse(self, fill_value=None, kind='block'): 

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision) 
    653      skip_blank_lines=skip_blank_lines) 
    654 
--> 655   return _read(filepath_or_buffer, kwds) 
    656 
    657  parser_f.__name__ = name 

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds) 
    409 
    410  try: 
--> 411   data = parser.read(nrows) 
    412  finally: 
    413   parser.close() 

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows) 
    1003     raise ValueError('skipfooter not supported for iteration') 
    1004 
-> 1005   ret = self._engine.read(nrows) 
    1006 
    1007   if self.options.get('as_recarray'): 

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows) 
    1746  def read(self, nrows=None): 
    1747   try: 
-> 1748    data = self._reader.read(nrows) 
    1749   except StopIteration: 
    1750    if self._first_chunk: 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)() 

pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28765)() 

ParserError: Error tokenizing data. C error: Expected 49 fields in line 1110195, saw 65 

I Pandas denken, ist wegen der Datumsformat in der CSV-Datei nicht glücklich. Ich habe versucht, verschiedene Codierung, infer_datetime_format = True, dtype = str, dtype = Objekt. ? Kein Glück :(Irgendwelche Ideen

Answer 1

Okay, ich am Ende herauszufinden, bis diese meine zwei Ansätze waren:..

1. erstellen alles als ein String dann Typ später ändern

   mystuff = pandas.read_csv(trades.csv, dtype=str) 
   

   Dann ich habe die Art mit:.

   mystuff['col_a'] = mystuff[['col_b']].apply(pd.to_numeric) 
   mystuff['col_c'] = mystuff['col_c'].apply(pd.to_datetime, format='%Y%m%d', errors='coerce')                     
   

2. zuordnen, welche Art von Daten in jeder Spalte von Anfang an ist

   pandas.read_csv(path,dtype={'col_a':str,'col_b':int, 'col_c':datetime...})