: org.apache.spark.sparkexception: job aborted due stage failure: task 0 in stage 5.0 failed 1 times, recent failure: lost task 0.0 in stage 5.0 (tid 5, localhost): org.apache.spark.sparkexception: data of type java.util.arraylist cannot used my rdd made avro file contains key "mylist". how change such uses tuple of tuples instead? there way create new rdd convert current arraylist of maps tuple of tuple of maps instead of arraylist of maps?
i.e. instead of [{"a":"vala", "a1":"vala1"},{"b":"valb", "b1":"valb1"}], change (({"a":"vala", "a1":"vala1"},{"b":"valb","b1":"valb1"}))
you should able call tuple on list:
>>> tuple([{"a":"vala", "a1":"vala1"},{"b":"valb", "b1":"valb1"}]) ({'a': 'vala', 'a1': 'vala1'}, {'b1': 'valb1', 'b': 'valb'})
Comments
Post a Comment