Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10284

Wrangler transform doesn't handle escaped sequences (\t, \n, etc)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      Fixed an issue in the Wrangler transform with the handling of escaped characters.
    • Rank:
      1|hzzrdr:

      Description

      I wrangled the browser info from the Omniture UseCase, dropped it in Hydrator, connected a source and sink, but when I preview, I get the following exception:

      2016-12-20 11:08:48,587 - ERROR [Thread-189:o.a.h.m.LocalJobRunnerWithFix$Job@562] - Job job_local943138009_0004 failed
      java.lang.Exception: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalArgumentException: Could not transform input: sun.org.mozilla.javascript.internal.WrappedException: Wrapped java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 (<Unknown source>#4) in <Unknown source> at line number 4
      	at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job.runTasks(LocalJobRunnerWithFix.java:465) ~[co.cask.cdap.cdap-app-fabric-4.0.0.jar:na]
      	at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job.run(LocalJobRunnerWithFix.java:524) ~[co.cask.cdap.cdap-app-fabric-4.0.0.jar:na]
      java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalArgumentException: Could not transform input: sun.org.mozilla.javascript.internal.WrappedException: Wrapped java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 (<Unknown source>#4) in <Unknown source> at line number 4
      	at co.cask.cdap.etl.batch.PipeTransformDetail.process(PipeTransformDetail.java:51) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.PipeTransformExecutor.runOneIteration(PipeTransformExecutor.java:42) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.TransformRunner.transform(TransformRunner.java:158) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.ETLMapReduce$ETLMapper.map(ETLMapReduce.java:358) ~[cdap-etl-batch-4.0.0.jar:na]
      	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapperWrapper.run(MapperWrapper.java:119) ~[na:na]
      	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
      	at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job$MapTaskRunnable.run(LocalJobRunnerWithFix.java:243) ~[co.cask.cdap.cdap-app-fabric-4.0.0.jar:na]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_79]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_79]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_79]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_79]
      	at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_79]
      Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Could not transform input: sun.org.mozilla.javascript.internal.WrappedException: Wrapped java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 (<Unknown source>#4) in <Unknown source> at line number 4
      	at co.cask.cdap.etl.batch.PipeTransformDetail.process(PipeTransformDetail.java:51) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.TransformEmitter.emit(TransformEmitter.java:49) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.common.TrackedEmitter.emit(TrackedEmitter.java:45) ~[cdap-etl-core-4.0.0.jar:na]
      	at co.cask.hydrator.plugin.batch.source.StreamBatchSource.transform(StreamBatchSource.java:175) ~[1482260915330-0/:na]
      	at co.cask.hydrator.plugin.batch.source.StreamBatchSource.transform(StreamBatchSource.java:56) ~[1482260915330-0/:na]
      	at co.cask.cdap.etl.common.preview.LimitingTransform.transform(LimitingTransform.java:44) ~[cdap-etl-core-4.0.0.jar:na]
      	at co.cask.cdap.etl.common.TrackedTransform.transform(TrackedTransform.java:61) ~[cdap-etl-core-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.PipeTransformDetail.process(PipeTransformDetail.java:46) ~[cdap-etl-batch-4.0.0.jar:na]
      	... 13 common frames omitted
      Caused by: java.lang.IllegalArgumentException: Could not transform input: sun.org.mozilla.javascript.internal.WrappedException: Wrapped java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 (<Unknown source>#4) in <Unknown source> at line number 4
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.transform(JavaScriptTransform.java:166) ~[1482260915330-0/:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.transform(JavaScriptTransform.java:58) ~[1482260915330-0/:na]
      	at co.cask.cdap.etl.common.TrackedTransform.transform(TrackedTransform.java:61) ~[cdap-etl-core-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.PipeTransformDetail.process(PipeTransformDetail.java:46) ~[cdap-etl-batch-4.0.0.jar:na]
      	... 20 common frames omitted
      Caused by: javax.script.ScriptException: sun.org.mozilla.javascript.internal.WrappedException: Wrapped java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 (<Unknown source>#4) in <Unknown source> at line number 4
      	at com.sun.script.javascript.RhinoScriptEngine.invoke(RhinoScriptEngine.java:300) ~[na:1.7.0_79]
      	at com.sun.script.javascript.RhinoScriptEngine.invokeFunction(RhinoScriptEngine.java:258) ~[na:1.7.0_79]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.transform(JavaScriptTransform.java:164) ~[1482260915330-0/:na]
      	... 23 common frames omitted
      Caused by: sun.org.mozilla.javascript.internal.WrappedException: Wrapped java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 (<Unknown source>#4)
      	at sun.org.mozilla.javascript.internal.Context.throwAsScriptRuntimeEx(Context.java:1808) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.MemberBox.invoke(MemberBox.java:196) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.NativeJavaMethod.call(NativeJavaMethod.java:245) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.Interpreter.interpretLoop(Interpreter.java:1706) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.Interpreter.interpret(Interpreter.java:849) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.InterpretedFunction.call(InterpretedFunction.java:162) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.ContextFactory.doTopCall(ContextFactory.java:430) ~[na:1.7.0_79]
      	at com.sun.script.javascript.RhinoScriptEngine$1.superDoTopCall(RhinoScriptEngine.java:116) ~[na:1.7.0_79]
      	at com.sun.script.javascript.RhinoScriptEngine$1.doTopCall(RhinoScriptEngine.java:109) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.ScriptRuntime.doTopCall(ScriptRuntime.java:3160) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.InterpretedFunction.call(InterpretedFunction.java:160) ~[na:1.7.0_79]
      	at com.sun.script.javascript.RhinoScriptEngine.invoke(RhinoScriptEngine.java:294) ~[na:1.7.0_79]
      	... 25 common frames omitted
      Caused by: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
      	at co.cask.cdap.etl.batch.PipeTransformDetail.process(PipeTransformDetail.java:51) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.TransformEmitter.emit(TransformEmitter.java:49) ~[cdap-etl-batch-4.0.0.jar:na]
      	at co.cask.cdap.etl.common.TrackedEmitter.emit(TrackedEmitter.java:45) ~[cdap-etl-core-4.0.0.jar:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform$JSEmitter.emit(JavaScriptTransform.java:185) ~[1482260915330-0/:na]
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_79]
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_79]
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
      	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
      	at sun.org.mozilla.javascript.internal.MemberBox.invoke(MemberBox.java:167) ~[na:1.7.0_79]
      	... 35 common frames omitted
      Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
      	at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_79]
      	at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_79]
      	at co.cask.wrangler.api.Row.getValue(Row.java:112) ~[1482260916744-0/:na]
      	at co.cask.wrangler.steps.Upper.execute(Upper.java:51) ~[1482260916744-0/:na]
      	at co.cask.wrangler.steps.Upper.execute(Upper.java:28) ~[1482260916744-0/:na]
      	at co.cask.wrangler.internal.DefaultPipeline.execute(DefaultPipeline.java:59) ~[1482260916744-0/:na]
      	at co.cask.wrangler.internal.DefaultPipeline.execute(DefaultPipeline.java:36) ~[1482260916744-0/:na]
      	at co.cask.wrangler.Wrangler.transform(Wrangler.java:131) ~[1482260916744-0/:na]
      	at co.cask.wrangler.Wrangler.transform(Wrangler.java:46) ~[1482260916744-0/:na]
      	at co.cask.cdap.etl.common.TrackedTransform.transform(TrackedTransform.java:61) ~[cdap-etl-core-4.0.0.jar:na]
      	at co.cask.cdap.etl.batch.PipeTransformDetail.process(PipeTransformDetail.java:46) ~[cdap-etl-batch-4.0.0.jar:na]
      	... 43 common frames omitted
      

        Attachments

          Activity

            People

            • Assignee:
              russellsavage Russ Savage
              Reporter:
              russellsavage Russ Savage
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: