i new hadoop , conducting poc single node. second time has happened , not know how solve. happens after map reduce have been stopped ctrl + c. sqlserver importing data hbase sqoop , after error , ctrl + c, nunhum job works, gets stuck in accepted. excludes jobs applications. restarted entire server. resource manager configured accordingly (10gb memory, 2cores), not know happens.
[root@hadoop01 /]# hadoop version hadoop 2.6.0.2.2.6.0-2800 subversion git@github.com:hortonworks/hadoop.git -r acb70ecfae2c3c5ab46e24b0caebceaec16fdcd0 compiled jenkins on 2015-05-18t20:21z compiled protoc 2.5.0 source checksum a25c30f622eb057f47e2155f78dba5e command run using /usr/hdp/2.2.6.0-2800/hadoop/hadoop-common-2.6.0.2.2.6.0-2800.jar [root@hadoop01 fausto.branco]# cat /etc/centos-release centos release 6.6 (final) sudo -u hdfs sqoop import –connect “jdbc:sqlserver://sqlserverip:1433;database=db_poc;username=hdpteste;password=xxxxxx” \ –hbase-create-table \ –hbase-table vw_hdp_arquivo \ –hbase-row-key “id_arquivo, valor” \ –column-family cf_name \ –driver com.microsoft.sqlserver.jdbc.sqlserverdriver \ –table “dbo.vw_hdp_arquivo” -m 1 15/07/14 19:43:51 info zookeeper.zookeeper: session: 0x14e8e1385000001 closed 15/07/14 19:43:51 info zookeeper.clientcnxn: eventthread shut down 15/07/14 19:43:51 info impl.timelineclientimpl: timeline service address: hxxp://hadoop01.poc.local:8188/ws/v1/timeline/ 15/07/14 19:43:51 info client.rmproxy: connecting resourcemanager @ hadoop01.poc.local/xxx.xxx.xxx.xxx:8050 15/07/14 19:43:53 info db.dbinputformat: using read commited transaction isolation 15/07/14 19:43:53 info mapreduce.jobsubmitter: number of splits:1 15/07/14 19:43:54 info mapreduce.jobsubmitter: submitting tokens job: job_1436900603357_0002 15/07/14 19:43:54 info impl.yarnclientimpl: submitted application application_1436900603357_0002 15/07/14 19:43:54 info mapreduce.job: url track job: hxxp://hadoop01.poc.local:8088/proxy/application_1436900603357_0002/ 15/07/14 19:43:54 info mapreduce.job: running job: job_1436900603357_0002 [root@hadoop01 fausto.branco]# yarn application -status application_1436900603357_0002 15/07/14 19:45:05 info impl.timelineclientimpl: timeline service address: hxxp://hadoop01.poc.local:8188/ws/v1/timeline/ 15/07/14 19:45:06 info client.rmproxy: connecting resourcemanager @ hadoop01.poc.local/xxx.xxx.xxx.xxx:8050 application report : application-id : application_1436900603357_0002 application-name : dbo.vw_hdp_arquivo.jar application-type : mapreduce user : hdfs queue : default start-time : 1436903034500 finish-time : 0 progress : 0% state : accepted final-state : undefined tracking-url : n/a rpc port : -1 host : n/a aggregate resource allocation : 0 mb-seconds, 0 vcore-seconds diagnostics : [root@hadoop01 fausto.branco]# hadoop job -status job_1436900603357_0002 deprecated: use of script execute mapred command deprecated. instead use mapred command it. 15/07/14 19:46:19 info impl.timelineclientimpl: timeline service address: hxxp://hadoop01.poc.local:8188/ws/v1/timeline/ 15/07/14 19:46:19 info client.rmproxy: connecting resourcemanager @ hadoop01.poc.local/xxx.xxx.xxx.xxx:8050 job: job_1436900603357_0002 job file: /user/hdfs/.staging/job_1436900603357_0002/job.xml job tracking url : hxxp://hadoop01.poc.local:8088/proxy/application_1436900603357_0002/ uber job : false number of maps: 0 number of reduces: 0 map() completion: 0.0 reduce() completion: 0.0 job state: prep retired: false reason failure: counters: 0
found!
analysing item: aggregate resource allocation : 0 mb-seconds, 0 vcore-seconds
i see node state unhealthy , found message in log:
health-report: 1/1 local-dirs bad: /hadoop/yarn/local; 1/1 local-dirs bad: /hadoop/yarn/log;
so put more disk , worked
Comments
Post a Comment