scala - How to convert DataFrame to Json? -


i have huge json file, small part follows:

{     "socialnews": [{         "admintagids": "",         "fileids": "",         "departmenttagids": "",         ........         ........         "comments": [{             "commentid": "",             "newsid": "",             "entityid": "",             ....             ....         }]     }]     .....     } 

i have applied lateral view exlode on socialnews follows:

val rdd = sqlcontext.jsonfile("file:///home/ashish/test") rdd.registertemptable("social") val result = sqlcontext.sql("select * social lateral view explode(socialnews) social comment") 

now want convert result (dataframe) json , save file, not able find scala api conversion. there standard library or way figure out?

val result: dataframe = sqlcontext.read.json(path) result.write.json("/yourpath") 

the method write in class dataframewriter , should accessible on dataframe objects. make sure rdd of type dataframe , not of deprecated type schemardd. can explicitly provide type definition val data: dataframe or cast dataframe todf().


Comments