i trying run simple twitter sentiment analysis code running fine upto don't know changed giving me error. command line has required parameters in place including --class --master --jars etc. thing did different run sudo apt-get install 7-jdk command , updated java version. running spark 1.3.1 java updation should not problem...i think. when run commands sbt assembly or sbt run error saying assembly not know command.
here's command line:
./bin/spark-submit --class sentimenter --master local[4] --jars /home/ubuntu/spark/spark-example-master/target/scala-2.10/sentiment_analysis-assembly-1.0.jar
and here's output get:
error: must specify primary resource (jar or python file) run --help usage or --verbose debug output using spark's default log4j profile: org/apache/spark/log4j-defaults.properties
any suggestions great!
from spark-submit --help:
usage: spark-submit [options] <app jar | python file> [app arguments]
you need add jar or python file on command line, after options. so, in example, you'd need do
./bin/spark-submit --class sentimenter --master local[4] /home/ubuntu/spark/spark-example-master/target/scala-2.10/sentiment_analysis-assembly-1.0.jar note removed --jars. can add additional jars (dependencies). if have 1 jar, that's primary resource , goes on command line without --jars option.
Comments
Post a Comment