scrapy Werte von mehreren Standorten erhalten

Question

Ich versuche, einen Wert von einer Funktion übergeben.

Ich schaute die Dokumente und verstand es einfach nicht. ref:

def parse_page1(self, response): 
    item = MyItem() 
    item['main_url'] = response.url 
    request = scrapy.Request("http://www.example.com/some_page.html", 
          callback=self.parse_page2) 
    request.meta['item'] = item 
    yield request 

def parse_page2(self, response): 
    item = response.meta['item'] 
    item['other_url'] = response.url 
    yield item 

hier ist ein psudo Code von dem, was ich achive wollen:

import scrapy 

class GotoSpider(scrapy.Spider): 
    name = 'goto' 
    allowed_domains = ['first.com', 'second.com] 
    start_urls = ['http://first.com/'] 

def parse(self, response): 
    name = response.xpath(...) 
    price = scrapy.Request(second.com, callback = self.parse_check) 
    yield(name, price) 


def parse_check(self, response): 
    price = response.xpath(...) 
    return price 

Answer 1

Dies ist, wie Sie einen beliebigen Wert passieren kann, Link etc. zu anderen Methoden:

import scrapy 

class GotoSpider(scrapy.Spider): 
    name = 'goto' 
    allowed_domains = ['first.com', 'second.com'] 
    start_urls = ['http://first.com/'] 

    def parse(self, response): 
     name = response.xpath(...) 
     link = response.xpath(...) # link for second.com where you may find the price 
     request = scrapy.Request(url=link, callback = self.parse_check) 
     request.meta['name'] = name 
     yield request 

    def parse_check(self, response): 
     name = response.meta['name'] 
     price = response.xpath(...) 
     yield {"name":name,"price":price} #Assuming that in your "items.py" the fields are declared as name, price