i using spark-1.3.1 (pyspark) , have generated table using sql query. have object dataframe. want export dataframe object (i have called "table") csv file can manipulate , plot columns. how export dataframe "table" csv file?
thanks!
if data frame fits in driver memory , want save local files system can convert spark dataframe local pandas dataframe using topandas method , use to_csv:
df.topandas().to_csv('mycsv.csv') otherwise can use spark-csv:
spark 1.3
df.save('mycsv.csv', 'com.databricks.spark.csv')spark 1.4+
df.write.format('com.databricks.spark.csv').save('mycsv.csv')
in spark 2.0+ can use csv data source directly:
df.write.csv('mycsv.csv')
Comments
Post a Comment