im using python 2.7 (anaconda) on windows 7 x64. installed scrapy via pip. version installed 1.0.1. trying run basic example in tutorial i.e.:
import scrapy class stackoverflowspider(scrapy.spider): name = 'stackoverflow' start_urls = ['http://stackoverflow.com/questions?sort=votes'] def parse(self, response): href in response.css('.question-summary h3 a::attr(href)'): full_url = response.urljoin(href.extract()) yield scrapy.request(full_url, callback=self.parse_question) def parse_question(self, response): yield { 'title': response.css('h1 a::text').extract()[0], 'votes': response.css('.question .vote-count-post::text').extract()[0], 'body': response.css('.question .post-text').extract()[0], 'tags': response.css('.question .post-tag::text').extract(), 'link': response.url, } and @ command line:
scrapy runspider stackoverflow_spider.py -o top-stackoverflow-questions.json however python crashes before doing anything. windows reports:
python.exe has stopped working
the output in command prompt is:
c:\anaconda\lib\site-packages\scrapy\commands\deploy.py:16: scrapydeprecationwarning: module `scrapy.command` deprecated, use`scrapy.commands` instead scrapy.command import scrapycommand 2015-07-17 16:39:28 [scrapy] info: scrapy 1.0.1 started (bot: scrapybot) 2015-07-17 16:39:28 [scrapy] info: optional features available: ssl, http11, boto 2015-07-17 16:39:28 [scrapy] info: overridden settings: {'feed_format': 'json','feed_uri': 'top-stackoverflow-questions.json'} does know how scrapy working? have installed incorrectly?
following @amol answer, have uninstalled scrapy using pip , removed folders python/anconda package directory. reinstalled using pip. same problem persists first line of output before crashes has gone. output is:
2015-07-17 16:39:28 [scrapy] info: scrapy 1.0.1 started (bot: scrapybot) 2015-07-17 16:39:28 [scrapy] info: optional features available: ssl, http11, boto 2015-07-17 16:39:28 [scrapy] info: overridden settings: {'feed_format': 'json','feed_uri': 'top-stackoverflow-questions.json'}
scrapy\commands\deploy.py file should not present version 1.0.1. file may have code dependent on deprecated / moved code causing python.exe crash.
a quick @ files in virtual environments present on machines tells me:
scrapy 0.24.4: file present
scrapy 1.0.1: file has been removed
this file being present in scrapy 1.0.1 installation suggests installation might not have happened correctly. that, file(s) previous version still lying around.
you might want remove scrapy, delete scrapy directory , reinstall.
Comments
Post a Comment