Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10077

Multiple output fails when being used with HDFS sink and anything else

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.4.2
    • Fix Version/s: 3.5.0, 3.4.3
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      HDFSSink can now be used alongside other sinks in a Hydrator pipeline.
    • Rank:
      1|hzzeqf:

      Description

      multiple outputs considers hdfs sink to be a dataset and tries to validate the hdfs path as a dataset name.

      2016-06-23 02:18:44,269 - ERROR [MapReduceRunner-phase-1:c.c.c.i.a.r.b.MapReduceRuntimeService@299] - Exception when submitting MapReduce Job: job=phase-1,=namespaceId=default, applicationId=stream_to_hdfs_batch_v4, program=phase-1, runid=cdaffdd1-38e8-11e6-8b6f-42010a800010
      co.cask.tephra.TransactionFailureException: Failed to execute method beforeSubmit inside a transaction
      	at co.cask.cdap.data2.transaction.Transactions.execute(Transactions.java:177) ~[co.cask.cdap.cdap-data-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.beforeSubmit(MapReduceRuntimeService.java:466) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.startUp(MapReduceRuntimeService.java:204) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) [com.google.guava.guava-13.0.1.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService$1$1.run(MapReduceRuntimeService.java:386) [co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
      Caused by: java.lang.IllegalArgumentException: Name 'hdfs://10.128.0.16:8020/tmp/hdfs_sink_v3' must consist only of ASCII letters, numbers, _, or -.
      	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) ~[com.google.guava.guava-13.0.1.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.dataset.output.MultipleOutputs.assertValidName(MultipleOutputs.java:143) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.dataset.output.MultipleOutputs.addNamedOutput(MultipleOutputs.java:125) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.setOutputsIfNeeded(MapReduceRuntimeService.java:669) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.access$400(MapReduceRuntimeService.java:124) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService$2.call(MapReduceRuntimeService.java:475) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService$2.call(MapReduceRuntimeService.java:466) ~[co.cask.cdap.cdap-app-fabric-3.4.2.jar:na]
      	at co.cask.cdap.data2.transaction.Transactions.execute(Transactions.java:174) ~[co.cask.cdap.cdap-data-fabric-3.4.2.jar:na]
      	... 5 common frames omitted
      

        Attachments

          Activity

            People

            • Assignee:
              ali.anwar Ali Anwar
              Reporter:
              russellsavage Russ Savage
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: