httplib.BadStatusLine: '' mit Selenium und PhantomJS

Question

Ich habe diesen Fehler beim Laden einer URL und bekomme einen seltsamen Fehler Ich habe keine Ahnung, wie zu beheben. Meine Situation verlangt, dass ich PhantomJS verwenden muss, da ich nicht glaube, dass ich die Firefox-Treiber auf AWS Lambda verwenden kann, und in meinem Scraping stoße ich auf einen Knopf, auf den der Chromedriver nicht klicken kann.

Wenn ich PhantomJS für Chrome oder Firefox auswähle, wird die URL korrekt aufgelöst.

Verwendung von Selen == 3.4.1 und PhantomJS 2.1.1

user_agent = (
     "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0") 

dcap = dict(DesiredCapabilities.PHANTOMJS) 
dcap["phantomjs.page.settings.userAgent"] = user_agent 
dcap["phantomjs.page.settings.javascriptEnabled"] = True 

browser = webdriver.PhantomJS(service_log_path=os.path.devnull, service_args=[ 
    '--ignore-ssl-errors=true'], desired_capabilities=dcap) 

browser.set_window_size(1120, 550) 
browser.get('https://drizly.com/session/new') 



File "main.py", line 257, in <module> 
    lambda_handler(None, None) 
    File "main.py", line 103, in lambda_handler 
    browser.get('https://drizly.com/session/new') 
    File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 264, in get 
    self.execute(Command.GET, {'url': url}) 
    File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in execute 
    response = self.command_executor.execute(driver_command, params) 
    File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 464, in execute 
    return self._request(command_info[0], url, body=data) 
    File "/Users/aymon/Envs/drizly/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 526, in _request 
    resp = opener.open(request, timeout=self._timeout) 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open 
    response = self._open(req, data) 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open 
    '_open', req) 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain 
    result = func(*args) 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1227, in http_open 
    return self.do_open(httplib.HTTPConnection, req) 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1200, in do_open 
    r = h.getresponse(buffering=True) 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse 
    response.begin() 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin 
    version, status, reason = self._read_status() 
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 417, in _read_status 
    raise BadStatusLine(line) 
httplib.BadStatusLine: '' 

Answer 1

BadStatusLine ist eine Unterklasse von HTTPException, die, wenn a server responds with a HTTP status code that we don’t understand angehoben wird. Sie können es fangen wollen, wie

#... 
browser.set_window_size(1120, 550) 
try: 
    browser.get('https://drizly.com/session/new') 
except httplib.BadStatusLine as bsl: 
    print('[!!!] {message}'.format(bsl.message)) 
#... 

Hinweis folgt, dass eine gute Praxis ist, dass ein Fehler should never pass silently. Daher diese Verwendung von print.