Herunterladen von Dateien aus s3 Eimer

Question

Kann jemand mir den Python-Code, um alle Dateien aus Amazon S3 Eimer mit der Boto-Bibliothek in Python herunterladen?

Answer 1

Erstellen Sie ein lokales Verzeichnis, um den Inhalt herunterzuladen.

DOWNLOAD_LOCATION_PATH = '/home/Desktop/s3' 

BUCKET_NAME = "xxxxxxxxxxxxxxxxx" 
AWS_ACCESS_KEY_ID= os.getenv("AWS_KEY_ID") # set your AWS_KEY_ID on your environment path 
AWS_ACCESS_SECRET_KEY = os.getenv("AWS_ACCESS_KEY") # set your AWS_ACCESS_KEY on your environment path 
conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_ACCESS_SECRET_KEY) 
bucket = conn.get_bucket(BUCKET_NAME) 

erhalten Sie die Liste aller Dateien und Ordner in S3

bucket_list = bucket.list() 

Schleife durch jede Datei/Ordner, den Inhalt und im Download-Ordner

for l in bucket_list: 
     key_string = str(l.key) 
     s3_path = DOWNLOAD_LOCATION_PATH + key_string 
     try: 
      print ("Current File is ", s3_path) 
      l.get_contents_to_filename(s3_path) 
     except (OSError,S3ResponseError) as e: 
      pass 
      # check if the file has been downloaded locally 
      if not os.path.exists(s3_path): 
       try: 
        os.makedirs(s3_path) 
       except OSError as exc: 
        # let guard againts race conditions 
        import errno 
        if exc.errno != errno.EEXIST: 
         raise