scala - How to use RDD collect method to process each row of RDD in array format? -


//making rdd  val logdata = sc.textfile(sampledata).cache()  //making logdataarray[string]   var logdataarray = logdata.collect; 

but throwing me error:

java.lang.nullpointerexception     @ org.apache.spark.rdd.rdd.collect(rdd.scala:717)     @ com.travel$.com$travel$$isconnected$1(travel.scala:58) 

before using logdata.collect have check size of logdata println(logdata.count). give 1168 record size.

if writing line ( var logdataarray = logdata.collect )in function, possible sc not in scope , getting null. everytime function invoked, execute collect method.

try adding line outside function ie. directly in main method. working me.


Comments