Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-14397

After Ingest data into GCSFile, the output is not readable by Preparation

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.1.0
    • Component/s: Data Prep
    • Labels:
      None
    • Release Notes:
      Files without extension will be treated as text files.
    • Rank:
      1|i00i13:

      Description

      Use Case:

      1. Create a pipeline that will write into GCS.
      2. Try to use Data Preparation to further transform the output.

      Currently, there's no way to wrangle the output file that is written to GCS. Dataprep only able to process file with certain extensions. 

       

      Ideally, dataprep should be able to take a folder (HDFS folder?) and sample data within that folder. 

        Attachments

          Activity

            People

            • Assignee:
              vinisha Vinisha Shah
              Reporter:
              edwin Edwin Elia
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: