2017-07-05 5 views
0

Ich bin neu in Scrapy i Datum von den Seiten des Bereichs zu kratzen versuchen (1,70000000) der Code I verwendet wird, istScrapy START_URL Fehler

import scrapy, json, re 
from blackberry.items import BlackberryItem 
class BlackSpider(scrapy.Spider): 
    name = 'datas' 
    start_urls = [ 
       'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' %page for page in xrange(1, 10000000), 
       'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%y for y in xrange(10000000, 20000000), 
       'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%a for a in xrange(20000000, 30000000), 
       'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%b for b in xrange(40000000, 50000000), 
       'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%c for c in xrange(50000000, 60000000), 
       'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%d for d in xrange(60000000, 70000000) 
       ] 

Aber ich habe diesen Fehler:

"y is not defined" 

Antwort

0

Eine der möglichen Lösungen ist wie folgt.

import scrapy 
import json 
import re 
from blackberry.items import BlackberryItem 
class BlackSpider(scrapy.Spider): 
    name = 'datas' 
    start_urls = ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(10000000, 20000000)] 
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(20000000, 30000000)] 
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(30000000, 40000000)] 
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(40000000, 50000000)] 
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(50000000, 60000000)] 
+0

Es wird Speicherfehler angezeigt – emon

+0

Es ist nicht genügend Speicher vorhanden. Sie müssen mit einem kleineren Bereich beginnen. –

Verwandte Themen