hello i'm trying collect data website.
on start collecting links on website. after accessing them. problem when accessing multiple pages of website "invalid uri: uri empty" error showing in pages. these of failed urls.
http://syayoyu.com/category/zensinyokudiet http://syayoyu.com/fruitgranola-6759.html http://syayoyu.com/category/diet/fruitgranola accessing browser , accessing them 1 one working ok. when try access in loop error occuring. please tell me wrong it.
this accessing code
httpwebrequest request = (httpwebrequest)httpwebrequest.create(url); request.useragent = "a .net web crawler"; webresponse response = request.getresponse(); stream stream = response.getresponsestream(); streamreader reader = new streamreader(stream); string htmltext = reader.readtoend();
list<string> list = new list<string>(){ @"http://syayoyu.com/category/zensinyokudiet", @"http://syayoyu.com/fruitgranola-6759.html", @"http://syayoyu.com/category/diet/fruitgranola" }; foreach (var url in list) { httpwebrequest request = (httpwebrequest)httpwebrequest.create(url); request.useragent = "a .net web crawler"; webresponse response = request.getresponse(); //stream stream = response.getresponsestream(); //streamreader reader = new streamreader(stream); //string htmltext = reader.readtoend(); //console.writeline(htmltext); using (stream stream = response.getresponsestream()) { using (streamreader reader = new streamreader(stream)) { string htmltext = reader.readtoend(); console.writeline(htmltext); } } } in solution, works fine,so there questions don't know?
Comments
Post a Comment