Scrapy: SQLAlchemy in scrapy Verwendung hat "Nameerror: name 'Verbindung' nicht definiert ist"

Question

SQLAlchemy in scrapy Verwendung hat NameError, die Fehlermeldung wie folgt:

Traceback (most recent call last): 
    File "e:\weibo_spider\venv\lib\site-packages\twisted\internet\defer.py", line 1386, in _inlineCallbacks 
    result = g.send(result) 
    File "e:\weibo_spider\venv\lib\site-packages\scrapy\crawler.py", line 79, in crawl 
    yield self.engine.open_spider(self.spider, start_requests) 
NameError: name 'connection' is not defined 

Und hier ist meine ScrapyPipeline Klasse:

from sqlalchemy import create_engine 
from sqlalchemy.orm import sessionmaker 

from .models import MyModel # my sqlalchemy model 


class WeiboSpiderPipeline(object): 

    def open_spider(self, spider): 
     # using pymysql as the conncetor 
     engine = create_engine('mysql+pymysql://root@localhost/wbspider_data') 
     Session = sessionmaker(bind=engine) 
     self.conn = engine.connect() 
     self.session = Session(bind=connection) 

    def close_spider(self, spider): 
     self.conn.close() 

    def process_item(self, item, spider): 
     return item 

ich das Modell in der Befehlszeile zu testen, kann es funktionieren, aber es die NameError auftritt, nachdem ich den scrapy crawl myspidername Befehl ausführen.

Hilfe!

Answer 1

Verwenden Sie diesen Ansatz.

from sqlalchemy.orm import sessionmaker 

Session = sessionmaker(bind=engine) 
session = Session() 

# it will send the connection back to the pool of connections in the engine, but it will not close the connection instead it will make the connection idle 
session.close() 
# to close the engine having pool of connections will close all the idle connections in the pool 
engine.dispose()() 


# for directing close the connection on session.close, use this code 
# Disabling pooling using NullPool: 

from sqlalchemy.pool import NullPool 
engine = create_engine(
     'postgresql+psycopg2://scott:tiger@localhost/test', 
     poolclass=NullPool) 
session.close()