javascript - Click on dropdown menu with selenium, scrap doesn't work -

i want scrape data http://www.squawka.com/match-results. first want choose league in dropdown menu, e.g. americas, german bundesliga. code choose the americas new data isn't load. think there javascript in background not fired up.

here code. use scrapy , selenium chromedirver tested firefox driver no success.

import scrapy  squawka.items import squawkaitem scrapy.http import formrequest, request  selenium import selenium selenium import webdriver import time  class squawkaspider(scrapy.spider):     name = "squawka"     allowed_domains = ["squawka.com"]     start_urls = ["http://www.squawka.com/match-results"]  def __init__(self):     self.driver = webdriver.chrome(executable_path='/users/fabian/chromedriver')  def parse(self, response):     self.driver.get(response.url)     time.sleep(5)     dropdown = self.driver.find_element_by_xpath("//*[@id='league-filter-list']/option[contains(text(), 'the americas')]").click() 

i hope can me.

thank

your problem can resolved combination of wait time , select() statement. here in case, need put wait time. both implicit/explicit wait work here. have tried implicit wait.

i have tried below code , selecting "the americas" drop-down.

import java.util.concurrent.timeunit; import org.openqa.selenium.by; import org.openqa.selenium.webdriver; import org.openqa.selenium.webelement; import org.openqa.selenium.firefox.firefoxdriver; import org.openqa.selenium.support.ui.select;  public class chromedriver {      public static void main(string[] args) throws interruptedexception {         webdriver driver = new firefoxdriver();         driver.manage().timeouts().implicitlywait(15, timeunit.seconds);         driver.manage().window().maximize();         driver.get("http://www.squawka.com/match-results");         system.out.println("entered url");         webelement element1=driver.findelement(by.xpath("//*[@id='league-filter-list']"));         select sel = new select(element1);         sel.selectbyvisibletext("the americas");         system.out.println("the americans selected");     } } 

note:- application designed in such way that, select drop-down , again switch "top 5 european leagues".but able select "the americas" above code.