I have a spark streaming application which produces a dataset for every minute.
I need to save/overwrite the results of the processed data.
When I tried to overwrite the dataset org.apache.hadoop.mapred.FileAlreadyExistsException stops the execution.
I set the Spark property
set("spark.files.overwrite","true") , but there is no luck.
How to overwrite or Predelete the files from spark?