i using .from() , .size() methods retrieve documents elastic search results.
below sample example -
isearchresponse<dynamic> bresponse = objelasticclient.search<dynamic>(s => s.from(0).size(25000).index("accounts").alltypes().query(query)); recently came across scroll feature of elastic search. looks better approach from() , size() methods fetch large data.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
i looking example on scroll feature in nest api.
can please provide nest example?
thanks, sameer
internal implementation of nest reindex uses scroll move documents 1 index another.
it should starting point.
below can find interesting code github.
var page = 0; var searchresult = this.currentclient.search<t>( s => s .index(fromindex) .alltypes() .from(0) .size(size) .query(this._reindexdescriptor._queryselector ?? (q=>q.matchall())) .searchtype(searchtype.scan) .scroll(scroll) ); if (searchresult.total <= 0) throw new reindexexception(searchresult.connectionstatus, "index " + fromindex + " has no documents!"); ibulkresponse indexresult = null; { var result = searchresult; searchresult = this.currentclient.scroll<t>(s => s .scroll(scroll) .scrollid(result.scrollid) ); if (searchresult.documents.hasany()) indexresult = this.indexsearchresults(searchresult, observer, toindex, page); page++; } while (searchresult.isvalid && indexresult != null && indexresult.isvalid && searchresult.documents.hasany()); also can take @ integration test scroll
[test] public void searchtypescan() { var scanresults = this.client.search<elasticsearchproject>(s => s .from(0) .size(1) .matchall() .fields(f => f.name) .searchtype(searchtype.scan) .scroll("2s") ); assert.true(scanresults.isvalid); assert.false(scanresults.fieldselections.any()); assert.isnotnullorempty(scanresults.scrollid); var results = this.client.scroll<elasticsearchproject>(s=>s .scroll("4s") .scrollid(scanresults.scrollid) ); var hitcount = results.hits.count(); while (results.fieldselections.any()) { assert.true(results.isvalid); assert.true(results.fieldselections.any()); assert.isnotnullorempty(results.scrollid); var localresults = results; results = this.client.scroll<elasticsearchproject>(s=>s .scroll("4s") .scrollid(localresults.scrollid)); hitcount += results.hits.count(); } assert.areequal(scanresults.total, hitcount); }
Comments
Post a Comment