javascript - Scraping a webpage with python to get onclick values -


first of have say: patient me because not familiar argument going illustrate you.

i'd download intraday historical values of equities on frankfurt boerse website. let me take equity example: http://www.boerse-frankfurt.de/en/equities/adidas+ag+de000a1ewww0/price+turnover+history/tick+data#page=1

as can see there 2 options: trades on frankfurt , trades on xetra. i'd love download latters. tried scrape data knowledge of python poor.

how can 'select' desired onclick option?

thanks in advance replies. regards

ps: information, noted following fact inspecting xetra element: changes value when move on next page , if come value again different. here example: first time on page 1 got

 onclick="d39081344_fkt_set_par('6');d39081344_fkt_set_active(this);" class="brs_d39081344_li current last" 

, moved on page 2 , got

 onclick="d51109535_fkt_set_par('6');d51109535_fkt_set_active(this);" class="brs_d51109535_li current last"
, coming page 1 got
a onclick="d96086211_fkt_set_par('6');d96086211_fkt_set_active(this);" class="brs_d96086211_li current last"

the trick @ calls made when navigate through pages. browser's network analysis tool invaluable this. when go page page, post made 'http://www.boerse-frankfurt.de/en/parts/boxes/history/_tickdata_full.m data request.

then goal replicate , loop requests using python. here code started:

import requests r = requests.post('http://www.boerse-frankfurt.de/en/parts/boxes/history/_tickdata_full.m', data={'component_id':'prekop97077bf9dec39f14320bf9d40b636c7c589', 'page':"3", 'page_size':'50', 'boerse_id':'6', 'titel':'tick-data', 'lang':'en', 'text':'locbaec84ecad1b94ad2fd257897c87361', 'items_per_page':'50', 'template':'0', 'pages_total':'50', 'use_external_secu':'1', 'item_count':'2473', 'include_url':'/parts/boxes/history/_tickdata_full.m', 'ag':'291', 'secu':'291', }) print r.text #here data of interest, still needs parsed 

that general idea. put in loop, adding 1 page parameter each time.


Comments