Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-13040

Avro fileset plugins create bad hive tables with reserved keywords.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0.0, 4.3.3
    • Component/s: Pipeline Plugins
    • Labels:
      None
    • Release Notes:
      Fixed avro fileset plugins so that reserved hive keywords can be used as column names
    • Rank:
      1|i00afr:

      Description

      Apparently, when creating avro tables in hive, you don't actually need to specify the columns and types, but just need to set avro.schema.literal as a table property.

      Filesets that set the explore.schema dataset property can cause table creation to fail if reserved keywords are in the schema (like 'user'). Apparently, this is not an issue if the avro.schema.literal is set without setting explore.schema. All avro fileset plugins should therefore not be setting explore.schema.

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: