Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10285

Hydrator plugins not compatible with develop

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: Applications, Pipelines
    • Labels:
      None
    • Rank:
      1|hzz327:

      Description

      I tried using some transforms from the hydrator-plugins repo. The ui jsons are not correct there. For example, CSVParser.json looks like:

      {
        "id": "CSVParser",
        "groups": {
          "position": [ "group1" ],
          "group1": {
            "display": "CSV Parser",
            "position": [ "field", "format", "schema" ],
            "fields": {
              "field": {
                "widget": "textbox",
                "label": "Input field to Parse",
                "properties": {
                  "width": "large"
                }
              },
              "format": {
                "widget": "select",
                "label": "Formats",
                "properties": {
                  "values" : [ "DEFAULT", "EXCEL", "MYSQL", "RFC4180", "TDF" ],
                  "default": "DEFAULT"
                }
              },
              "schema" : {
                "widget": "schema",
                "label": "Schema",
                "description" : "Schema for parsed CSV Record",
                "schema-types" : [ "boolean", "int", "long", "float", "double", "bytes", "string" ],
                "schema-default-type" : "string"
              }
            }
          }
        }
      }
      

      If you try using this json, the UI will complain that there is a missing field. Apparently this is because schema can't be listed as a field, but needs to be in a special outputschema key:

      {
        "id": "CSVParser",
        "groups": {
          "position": [ "group1" ],
          "group1": {
            "display": "CSV Parser",
            "position": [ "field", "format" ],
            "fields": {
              "field": {
                "widget": "textbox",
                "label": "Input field to Parse",
                "properties": {
                  "width": "large"
                }
              },
              "format": {
                "widget": "select",
                "label": "Formats",
                "properties": {
                  "values" : [ "DEFAULT", "EXCEL", "MYSQL", "RFC4180", "TDF" ],
                  "default": "DEFAULT"
                }
              }
            }
          }
        },
        "outputschema": {
          "schema" : {
            "widget": "schema",
            "schema-types" : [ "boolean", "int", "long", "float", "double", "bytes", "string", "map<string, string>" ],
            "schema-default-type" : "string"
          }
        }
      }
      

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: