Scrapy/OpenSSL Schaben HTTPS-Sites: Attribute: ‚Modul‘ Objekt hat kein Attribut ‚SSL_CTX_set_session_id_context‘

Question

Ich habe versucht, eine HTTPS-Website zu crawlen bekam aber den folgenden Fehler:

2016-05-14 20:11:21 [scrapy] ERROR: Error downloading <GET https://www.anHTTPSsite.com/> 
Traceback (most recent call last): 
    File "/usr/local/lib/python3.4/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks 
    result = result.throwExceptionIntoGenerator(g) 
    File "/usr/local/lib/python3.4/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator 
    return g.throw(self.type, self.value, self.tb) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/core/downloader/middleware.py", line 43, in process_request 
    defer.returnValue((yield download_func(request=request,spider=spider))) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/utils/defer.py", line 45, in mustbe_deferred 
    result = f(*args, **kw) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/core/downloader/handlers/__init__.py", line 65, in download_request 
    return handler.download_request(request, spider) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/core/downloader/handlers/http11.py", line 60, in download_request 
    return agent.download_request(request) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/core/downloader/handlers/http11.py", line 281, in download_request 
    method, to_bytes(url, encoding='ascii'), headers, bodyproducer) 
    File "/usr/local/lib/python3.4/dist-packages/twisted/web/client.py", line 1594, in request 
    endpoint = self._getEndpoint(parsedURI) 
    File "/usr/local/lib/python3.4/dist-packages/twisted/web/client.py", line 1578, in _getEndpoint 
    return self._endpointFactory.endpointForURI(uri) 
    File "/usr/local/lib/python3.4/dist-packages/twisted/web/client.py", line 1454, in endpointForURI 
    uri.port) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/core/downloader/contextfactory.py", line 57, in creatorForNetloc 
    return ScrapyClientTLSOptions(hostname.decode("ascii"), self.getContext()) 
    File "/usr/local/lib/python3.4/dist-packages/scrapy/core/downloader/contextfactory.py", line 54, in getContext 
    return self.getCertificateOptions().getContext() 
    File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1618, in getContext 
    self._context = self._makeContext() 
    File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1656, in _makeContext 
    ctx.set_session_id(sessionName) 
    File "/usr/local/lib/python3.4/dist-packages/OpenSSL/SSL.py", line 719, in set_session_id 
    _lib.SSL_CTX_set_session_id_context(
AttributeError: 'module' object has no attribute 'SSL_CTX_set_session_id_context' 

Allerdings, wenn ich versuche zu kriechen seine HTTP-Version (einfach das Protokoll in der URL von https zu http ändern), ist der Fehler verschwunden.

EDIT: scrapy version -v Ausgabe:

Scrapy : 1.1.0 
lxml  : 3.4.4.0 
libxml2 : 2.9.2 
Twisted : 16.1.1 
Python : 3.4.3+ (default, Oct 14 2015, 16:03:50) - [GCC 5.2.1 20151010] 
pyOpenSSL : 16.0.0 (OpenSSL 1.0.2d 9 Jul 2015) 
Platform : Linux-4.2.0-36-generic-x86_64-with-Ubuntu-15.10-wily 

Jede Hilfe apreciated wird.

Answer 1

Ich hatte das gleiche Problem. Es scheint mit den letzten Änderungen in pyOpenSSL in Verbindung zu stehen. Downgrade auf 0.15.1 löste es für mich. Bevor ich dies tue, empfehle ich, das Changelog von pyOpenSSL zu betrachten, um zu sehen, ob Sie das Gefühl haben, dass sich eine Herabstufung lohnt.

Auf OSX, dies sollte funktionieren:

sudo pip install pyOpenSSL==0.15.1