Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12811

Field of name "date" causes explore failure

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.3.1
    • Fix Version/s: None
    • Component/s: Explore, Pipelines
    • Labels:
    • Rank:
      1|i0095z:

      Description

      When creating the sink, the SQL statement fails. I assume it is because 'date' is a reserved word. That means, the field name quoting does not work properly. Looking at the generated SQL create statement, we can see that the field names in the partitioning schema are quoted correctly, but not the fields of the actual schema.

      2017-11-07 09:35:02,583 - INFO  [explore.service-executor-20:h.q.p.ParseDriver@190] - Parsing command: CREATE EXTERNAL TABLE IF NOT EXISTS dataset_inspects (id bigint, score float, date string) PARTITIONED BY (`name` STRING, `year` STRING, `month` STRING) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' LOCATION 'hdfs://amr28827-1000.dev.continuuity.net:8020/cdap/namespaces/default/data/inspects' TBLPROPERTIES ('avro.schema.literal'='{"type":"record","name":"output","fields":[{"name":"id","type":["long","null"]},{"name":"score","type":["float","null"]},{"name":"date","type":["string","null"]}]}', 'cdap.name'='inspects', 'cdap.version'='4.3.1-1506281018006')
      

      Perhaps this has to do with the Avro form of the schema?

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              andreas Andreas Neumann
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: