python - Get a result from callback scrapy request immediately -


i not sure question correct, quite new in scrapy. therefore, comments related topic useful.

my problem have website structure( json document ):

{ 2: {         1: 'http://example.com/1.jpg',         2: 'http://example.com/2.jpg' }  ...// bunch of other information }  

and links related, need direct link images. trying solve task as:

urldata = scrapy.request(url='http://myserver/1.jpg', callback=self.parse_link)

where callback function is:

 def parse_link( self, response ):     hxs = scrapy.selector.htmlxpathselector(response)     data = lacbaclink()     link =  hxs.select("(//img)[1]/@src").extract()     data["link"] = link     return data 

i consider work if able result callback request function in scrapy immediately.

i have tried write code presented documentation:

 def parse_page1(self, response):     return scrapy.request("http://www.example.com/some_page.html",                           callback=self.parse_page2)  def parse_page2(self, response):     # log http://www.example.com/some_page.html     self.logger.info("visited %s", response.url) 

it not work me. if know how solve task using code, please, give me realisation better understanding.

this isn't working code -- read comments added.

def parse_page1(self, response):     # here collect information need first page , put in item     r = scrapy.request("http://www.example.com/some_page.html",                           callback=self.parse_page2)     r.meta['item'] = item     yield r  def parse_page2(self, response):     item = response.meta['item']     # add need response item     yield item # if need parse more pages yield request instead. 

Comments