i running vm in pseudo mode. due resource related issues (name node in safe mode, not able leave) had format , restart namenode of cloudera 4.x. didn't have other choice. used steps provided here:
writing hdfs replicated 0 nodes instead of minreplication (=1)
after able use get/put command in hdfs means have read/write permission. now, when try submit job, getting following exception.
exception in thread "main"org.apache.hadoop.ipc.remoteexception(java.io.ioexception): java.io.ioexception: no valid local directories in property: mapred.local.dir @ org.apache.hadoop.mapred.jobtracker.submitjob(jobtracker.java:3491) @ org.apache.hadoop.mapred.jobtracker.submitjob(jobtracker.java:3459) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) @ java.lang.reflect.method.invoke(method.java:597) @ org.apache.hadoop.ipc.writablerpcengine$server$writablerpcinvoker.call(writablerpcengine.java:474) @ org.apache.hadoop.ipc.rpc$server.call(rpc.java:898) @ org.apache.hadoop.ipc.server$handler$1.run(server.java:1693) @ org.apache.hadoop.ipc.server$handler$1.run(server.java:1689) @ java.security.accesscontroller.doprivileged(native method) @ javax.security.auth.subject.doas(subject.java:396) @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1332) @ org.apache.hadoop.ipc.server$handler.run(server.java:1687) caused by: java.io.ioexception: no valid local directories in property: mapred.local.dir @ org.apache.hadoop.conf.configuration.getlocalpath(configuration.java:1678) @ org.apache.hadoop.mapred.jobconf.getlocalpath(jobconf.java:500) @ org.apache.hadoop.mapred.jobinprogress.<init>(jobinprogress.java:409) @ org.apache.hadoop.mapred.jobtracker.submitjob(jobtracker.java:3489) ... 13 more @ org.apache.hadoop.ipc.client.call(client.java:1160) @ org.apache.hadoop.ipc.writablerpcengine$invoker.invoke(writablerpcengine.java:225) @ org.apache.hadoop.mapred.$proxy10.submitjob(unknown source) @ org.apache.hadoop.mapred.jobclient$2.run(jobclient.java:973) @ org.apache.hadoop.mapred.jobclient$2.run(jobclient.java:896) @ java.security.accesscontroller.doprivileged(native method) @ javax.security.auth.subject.doas(subject.java:396) @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1332) @ org.apache.hadoop.mapred.jobclient.submitjobinternal(jobclient.java:896) @ org.apache.hadoop.mapreduce.job.submit(job.java:531) @ org.apache.hadoop.mapreduce.job.waitforcompletion(job.java:561) @ clustering.mapreduce.kmeansclusteringjob.main(kmeansclusteringjob.java:69) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25) @ java.lang.reflect.method.invoke(method.java:597) @ org.apache.hadoop.util.runjar.main(runjar.java:208)** when searched above exception found multiple links stating mapred.local.dir should defined , if not set hadoop.tmp.dir used. explicitly set mapred.local.dir in mapred-site.xml , given full permission default folder (/var/lib/hadoop-hdfs/cache). problem still persists.
can please in solving issue? regards
didn't give proper permission local directory -- marking community wiki answer provided in comments
Comments
Post a Comment