java - how to resolve issue with uploading files to s3 from amazon emr cluster -


i see error message every time job processing.

error: com.amazonaws.util.ioutils.copy(ljava/io/inputstream;ljava/io/outputstream;)j 

my job has uploaded particular file s3 though job creates 5 files in total simultaneously , files supposed uploaded s3. 1 of files getting partially/fully uploaded s3. not sure whether conflict dependency tree of workflow scheduler or code using. can see there 3 dependencies resolved the:

<groupid>com.amazonaws</groupid> <artifactid>aws-java-sdk</artifactid>.  

one 1.9.26 , 2 other lower version numbers.

could 1 reason above issue? if yes how resolve it? if no, other suggestions should trying do?

the jars have used reside on s3. when run same workflow launching job manually , not through scheduler, see process runs fine on cluster meant manual runs. in no mans land trying figure out. please help!


Comments