Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-4005

TableSink should ensure that the schema only has simple types, during deployment of adapter

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.3.0
    • Component/s: CDAP, ETL
    • Labels:
    • Rank:
      1|hzz1tr:

      Description

      TableSink should ensure that the schema only has simple types, during deployment of adapter.
      If a TableSink has a non-simple type, then during runtime of the pipeline, the pipeline will fail with the error:

      Exception thrown in BatchDriver Mapper: {}
      java.lang.IllegalArgumentException: only simple types are supported (boolean, int, long, float, double, bytes).
              at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) ~[co.cask.cdap.cdap-cli-3.2.0.jar:na]
              at co.cask.cdap.etl.common.RecordPutTransformer.validateAndGetType(RecordPutTransformer.java:119) ~[cdap-etl-common-3.2.0.jar:na]
              at co.cask.cdap.etl.common.RecordPutTransformer.setField(RecordPutTransformer.java:78) ~[cdap-etl-common-3.2.0.jar:na]
              at co.cask.cdap.etl.common.RecordPutTransformer.toPut(RecordPutTransformer.java:64) ~[cdap-etl-common-3.2.0.jar:na]
              at co.cask.cdap.etl.batch.sink.TableSink.transform(TableSink.java:85) ~[na:na]
              at co.cask.cdap.etl.batch.sink.TableSink.transform(TableSink.java:42) ~[na:na]
              at co.cask.cdap.etl.batch.ETLMapReduce$SingleOutputSink.write(ETLMapReduce.java:443) ~[ETLWorkflow.8919208b-86d3-4dc3-9243-5b0952d9056d/:na]
              at co.cask.cdap.etl.batch.ETLMapReduce$ETLMapper.map(ETLMapReduce.java:361) ~[ETLWorkflow.8919208b-86d3-4dc3-9243-5b0952d9056d/:na]
              at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) [org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
              at co.cask.cdap.internal.app.runtime.batch.MapperWrapper.run(MapperWrapper.java:102) [co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
              at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) [org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
              at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) [org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
              at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job$MapTaskRunnable.run(LocalJobRunnerWithFix.java:243) [co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_75]
              at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_75]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
      

      Additionally, explore fails on the table created for it:

      "Target schema must only contain simple fields (boolean, int, long, float, double, bytes, string)"
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shankar Shankar Selvam
                Reporter:
                ali.anwar Ali Anwar
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: