i trying generate, sbt assembly, single project several jars. each containing of dependencies.
so far have found this qa close looking for. don't need have separate configs, when run assembly, want generate different jars.
to more concrete. want generate:
- one jar code , general dependencies
- one jar hadoop dependencies <- problem, don't know how say, generate jar has dependencies.
- one jar scala
without going deep complex sbt configurations, try approach. hadoop dependencies being standard, mark them provided in build exclude them.
"org.apache.hadoop" % "hadoop-client" % "2.6.0" % "provided" for scala, library jar standard , can downloaded separately "user". remove fat jar, use following setting (assembly 0.13.0):
assemblyoption in assembly := (assemblyoption in assembly).value.copy(includescala = false) the user of fat jar aked provide both scala , hadoop libraries in classpath.
for example, when using spark correct approach these 2 libraries both provided spark running environment. same logic applies hadoop mapreduce environment.
Comments
Post a Comment