2017-03-28 2 views
0

Ich fusioniere 6 xlsx-Dateien, jeweils ein paar MB. Sie haben im Allgemeinen die gleichen Daten, aber einige haben einige zusätzliche Variablen (die ich beibehalten möchte), und einige haben auch eindeutige Namen für die gleichen Variablen. Unten ist der Python-Code, den ich geschrieben habe, um die Aufgabe zu erledigen, aber ich bekomme einen MemoryError (ich sah im Task-Manager, dass er zu der Zeit ~ 2 GB RAM benötigte). Es macht keinen Sinn, dass dieses Programm viel mehr als ein paar hundert MB RAM für mich in Anspruch nimmt, wenn man die Dateigrößen berücksichtigt. Ich hoffe also, dass jemand darauf hinweisen kann, wo mein Fehler liegt.Pandas MemoryError mit concat auf relativ kleinen Excel-Dateien

import pandas as pd 
import os.path 

fpath = 'K:\\DATA\\python\\Mergingtest\\tess\\' 

#Specify file locations and sheet names 
data_headers = pd.read_excel(os.path.join(fpath, 'TESS_Headers_2006-2015.xlsx'), sheetname='Sheet1', header=1) 
data_07 = pd.read_excel(os.path.join(fpath, 'TESS 2007.xlsx'), sheetname='Errors', header=0) 
data_08 = pd.read_excel(os.path.join(fpath, 'TESS 2008.xlsx'), sheetname='Errors', header=0) 
data_09 = pd.read_excel(os.path.join(fpath, 'TESS 2009.xlsx'), sheetname='Errors', header=0) 
data_10 = pd.read_excel(os.path.join(fpath, 'TESS 2010.xlsx'), sheetname='Sheet1', header=0) 
data_11 = pd.read_excel(os.path.join(fpath, 'TESS 2011.xlsx'), sheetname='Sheet1', header=0) 
data_12_15 = pd.read_excel(os.path.join(fpath, 'COPY_TESS 2012-2015 Master File.xlsx'), sheetname='Sheet1', header=0) 

#Standardizing naming for equivalent variables from each file 
data_headers.columns = ["fldErrorID", "fdSiteCaseID", "fdSiteID", "fdYear", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC", "fdReviewedWithPersonInvolved", "fdBloodSupplierManufacurerNotifi", "cptr_delay2", "clin_lab_err", "sitegraded", "cptr_delay", "gen_code1", "disc_cnsq1", "cnsqnc_gen", "capacity1", "capacity", "severity1", "SECTION A - DISCOVERY INFORMATION: Describe the event", "SECTION A - DISCOVERY INFORMATION: Describe how the event was discovered", "SECTION A - DISCOVERY INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Did the actual event harm the patient?", "SECTION B - OCCURRENCE INFORMATION: If event caused harm to patient, describe", "SECTION B - OCCURRENCE INFORMATION: Was the near-miss event discovered because of a planned/unplanned detection procedure/process?", "SECTION B - OCCURRENCE INFORMATION: Discovery/Consequence code: Specify", "SECTION B - OCCURRENCE INFORMATION: Actual Severity", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 4", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 8", "SECTION B - OCCURRENCE INFORMATION: What was the outcome of the event to the patient up to 30 days after it was discovered?", "SECTION B - OCCURRENCE INFORMATION: If patient died, did the consequence of the event play a role in death?", "Date Created", "Case State", "Site ID grading", "dvcq"] 
data_07.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_08.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "sitegraded", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "gen_code1", "event2", "fdDiscoveryConsequenceEvent", "cnsqnc_gen", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "Site ID grading", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_09.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "sitegraded", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "gen_code1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_10.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "sitegraded", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "cptr_delay", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "gen_code1", "dvcq", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "capacity1", "capacity", "severity1", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdNoAction", "fdClosed", "fdDiscoveryTime", "fdReviewedWithPersonInvolved", "fdTimeEventOccurred", "cptr_delay2", "clin_lab_err"] 
data_11.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_12_15.columns = ["fldErrorID", "fdSiteCaseID", "fdSiteID", "fdYear", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC", "fdReviewedWithPersonInvolved", "fdBloodSupplierManufacurerNotifi", "cptr_delay2", "clin_lab_err", "sitegraded", "cptr_delay", "gen_code1", "disc_cnsq1", "cnsqnc_gen", "capacity1", "capacity", "severity1", "SECTION A - DISCOVERY INFORMATION: Describe the event", "SECTION A - DISCOVERY INFORMATION: Describe how the event was discovered", "SECTION A - DISCOVERY INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Did the actual event harm the patient?", "SECTION B - OCCURRENCE INFORMATION: If event caused harm to patient, describe", "SECTION B - OCCURRENCE INFORMATION: Was the near-miss event discovered because of a planned/unplanned detection procedure/process?", "SECTION B - OCCURRENCE INFORMATION: Discovery/Consequence code: Specify", "SECTION B - OCCURRENCE INFORMATION: Actual Severity", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 4", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 8", "SECTION B - OCCURRENCE INFORMATION: What was the outcome of the event to the patient up to 30 days after it was discovered?", "SECTION B - OCCURRENCE INFORMATION: If patient died, did the consequence of the event play a role in death?", "Date Created", "Case State"] 

concat = (pd.concat([data_headers,data_07,data_08,data_09,data_10,data_11,data_12_15])) 
#specify column order to prevent default alphabetical-ordering 
concat = concat[["fldErrorID", "fdSiteCaseID", "fdSiteID", "fdYear", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC", "fdReviewedWithPersonInvolved", "fdBloodSupplierManufacurerNotifi", "cptr_delay2", "clin_lab_err", "sitegraded", "cptr_delay", "gen_code1", "disc_cnsq1", "cnsqnc_gen", "capacity1", "capacity", "severity1", "SECTION A - DISCOVERY INFORMATION: Describe the event", "SECTION A - DISCOVERY INFORMATION: Describe how the event was discovered", "SECTION A - DISCOVERY INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Did the actual event harm the patient?", "SECTION B - OCCURRENCE INFORMATION: If event caused harm to patient, describe", "SECTION B - OCCURRENCE INFORMATION: Was the near-miss event discovered because of a planned/unplanned detection procedure/process?", "SECTION B - OCCURRENCE INFORMATION: Discovery/Consequence code: Specify", "SECTION B - OCCURRENCE INFORMATION: Actual Severity", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 4", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 8", "SECTION B - OCCURRENCE INFORMATION: What was the outcome of the event to the patient up to 30 days after it was discovered?", "SECTION B - OCCURRENCE INFORMATION: If patient died, did the consequence of the event play a role in death?", "Date Created", "Case State", "Site ID grading", "dvcq"]] 

concat.to_excel(os.path.join(fpath, 'Merge_Output.xlsx')) 

Antwort

0

versuchen stattdessen

data_07 = pd.read_excel(os.path.join(fpath, 'TESS 2007.xlsx'), sheetname='Errors') 
data_08 = pd.read_excel(os.path.join(fpath, 'TESS 2008.xlsx'), sheetname='Errors') 
#keep headers for all xls files 

concat =pd.concat([data_07,data_08], axis = 0) 
#remove data headers and add axis = 0 

#get rid of individual files 
del data_07 
del data_08 
+0

So dies gelungen, aber es stieg noch auf mehrere hundert MB RAM-Nutzung. Ist das normal? Die Dateien sind nur ein paar MB, aber enthalten ~ 10k Zeilen. – melonfacedoom

+0

sollten Sie anschließend den einzelnen Datenrahmen löschen. Siehe bearbeiteten Code –

+0

, wenn Sie damit einverstanden sind. Bitte akzeptieren und/oder upvote –