Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12848

If a non-nullable field is null, the pipeline fails with error: "null"

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.3.1
    • Fix Version/s: None
    • Component/s: Pipeline Plugins, Pipelines
    • Labels:
    • Rank:
      1|i009cv:

      Description

      In my case, I have a Javascript plugin that is supposed to set a non-nullable field in its output schema. However, it does not set that field. Now the pipeline fails without indicating why: all it gives me is a stack trace and an NPE:

      java.lang.NullPointerException: null
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.decodeSimpleType(JavaScriptTransform.java:300) ~[1510242060273-0/:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.decode(JavaScriptTransform.java:248) ~[1510242060273-0/:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.decodeRecord(JavaScriptTransform.java:273) ~[1510242060273-0/:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.access$100(JavaScriptTransform.java:62) ~[1510242060273-0/:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform$JSEmitter.emit(JavaScriptTransform.java:193) ~[1510242060273-0/:na]
      	at jdk.nashorn.internal.scripts.Script$Recompilation$5$1037AAA$\^eval\_.transform(<eval>:17) ~[na:na]
      	at jdk.nashorn.internal.scripts.Script$Recompilation$4$40$\^eval\_.dont_name_your_function_this(<eval>:1) ~[na:na]
      	at jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637) ~[nashorn.jar:na]
      	at jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494) ~[nashorn.jar:na]
      	at jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393) ~[nashorn.jar:na]
      	at jdk.nashorn.api.scripting.ScriptObjectMirror.callMember(ScriptObjectMirror.java:199) ~[nashorn.jar:na]
      	at jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:386) ~[nashorn.jar:na]
      	at jdk.nashorn.api.scripting.NashornScriptEngine.invokeFunction(NashornScriptEngine.java:190) ~[nashorn.jar:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.transform(JavaScriptTransform.java:172) ~[1510242060273-0/:na]
      	at co.cask.hydrator.plugin.transform.JavaScriptTransform.transform(JavaScriptTransform.java:59) ~[1510242060273-0/:na]
      	at co.cask.cdap.etl.common.plugin.WrappedTransform$6.call(WrappedTransform.java:107) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.plugin.WrappedTransform$6.call(WrappedTransform.java:104) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.plugin.Caller$1.call(Caller.java:30) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.plugin.StageLoggingCaller.call(StageLoggingCaller.java:40) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.plugin.WrappedTransform.transform(WrappedTransform.java:104) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.TrackedTransform.transform(TrackedTransform.java:74) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.batch.UnwrapPipeStage.consumeInput(UnwrapPipeStage.java:44) ~[cdap-etl-batch-4.3.1.jar:na]
      	at co.cask.cdap.etl.batch.UnwrapPipeStage.consumeInput(UnwrapPipeStage.java:32) ~[cdap-etl-batch-4.3.1.jar:na]
      	at co.cask.cdap.etl.batch.PipeStage.consume(PipeStage.java:44) ~[cdap-etl-batch-4.3.1.jar:na]
      	at co.cask.cdap.etl.batch.PipeEmitter.emit(PipeEmitter.java:89) ~[cdap-etl-batch-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.TrackedEmitter.emit(TrackedEmitter.java:57) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.cdap.etl.common.plugin.UntimedEmitter.emit(UntimedEmitter.java:64) ~[cdap-etl-core-4.3.1.jar:na]
      	at co.cask.hydrator.plugin.batch.source.AbstractFileBatchSource.transform(AbstractFileBatchSource.java:187) ~[1510242060273-0/:na]
      

      But in case we know what went wrong: field "foo" in the output schema is non-nullable but its value is null. The exception should have that information.

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              andreas Andreas Neumann
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: