Das Speichern von GZIP-JSON führt zum Zurücksetzen der Verbindung

Question

Ich habe eine gzipped JSON file, die ich versuche, herunterzuladen, aber wenn ich versuche, den Inhalt der Antwort mit der Bibliothek Anfragen zu lesen, wird die Verbindung zurückgesetzt. Hier

data = requests.request("GET", i, stream=True) 
with gzip.open(i.rsplit("/")[-1], "wb") as fh: 
    for chunk in data.iter_content(chunk_size=1024): 
    fh.write(chunk) 

ist das, was ich bekomme, wenn ich versuche, den Inhalt zu lesen:

Traceback (most recent call last): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 228, in _error_catcher 
    yield 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 501, in read_chunked 
    chunk = self._handle_chunk(amt) 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 461, in _handle_chunk 
    value = self._fp._safe_read(amt) 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\http\client.py", line 592, in _safe_read 
    chunk = self.fp.read(min(amt, MAXAMOUNT)) 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\socket.py", line 575, in readinto 
    return self._sock.recv_into(b) 
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\models.py", line 664, in generate 
    for chunk in self.raw.stream(chunk_size, decode_content=True): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 349, in stream 
    for line in self.read_chunked(amt, decode_content=decode_content): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 526, in read_chunked 
    self._original_response.close() 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\contextlib.py", line 77, in __exit__ 
    self.gen.throw(type, value, traceback) 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 246, in _error_catcher 
    raise ProtocolError('Connection broken: %r' % e, e) 
requests.packages.urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "H:/Programming/Python/warehouse/main.py", line 55, in <module> 
    compile_auctions(slugs) 
    File "H:/Programming/Python/warehouse/main.py", line 44, in compile_auctions 
    for chunk in data.iter_content(chunk_size=1024): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\models.py", line 667, in generate 
    raise ChunkedEncodingError(e) 
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) 

ich die JSON in meinem Browser lesen kann und in Postman, so dass ich bin mir nicht sicher, was ich tue falsch. Ich verwende Python 3.5.1 unter Windows 10.

Ist dies der richtige Weg, um GZipped JSON zu speichern?

Answer 1

Anforderungen dekomprimieren bereits die Daten.

Sie brauchen nicht gzip zu verwenden:

import requests 

req = requests.get("http://auction-api-us.worldofwarcraft.com/auction-data/4923213e5eb3ec3b7e03d22b632bda36/auctions.json", stream=True) 

with open("out.json", "wb") as f: 
    for chunk in req.iter_content(chunk_size=4096): 
     f.write(chunk) 

Answer 2

Also das Problem ist nicht wirklich mit dem API Ich verwende, dann ist es entweder um einen Fehler mit den Anfragen Bibliothek für Windows oder ein Problem mit Python 3.5 .1 unter Windows 10.

Als ich diesen genauen Code unter Ubuntu 16.04 unter Verwendung von Python 3.5.1 ausführte, funktionierte es gut.