Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-15135

Pipelines should validate arguments earlier

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 6.0.0
    • Fix Version/s: None
    • Component/s: Datasets, Pipelines
    • Rank:
      1|i00miv:

      Description

      A pipeline that writes to BigQuery, it will create an external dataset to track lineage, metadata etc. for the sink. If the reference name given in the sink properties is not a valid dataset name, that throws an exception, which is expected. However, it should do so, when the pipeline is deployed, or right when it is started. In this case, the pipeline had 5 phases, and only when the 5th phase started, the error was raised. After many minutes of running. 

      java.lang.IllegalArgumentException: Name 'doubleclick.impressions-da43a7e4-2296-44d2-9787-a3aede518679' must consist only of ASCII letters, numbers, _, or -.
       

        Attachments

          Activity

            People

            • Assignee:
              trishka Trishka
              Reporter:
              andreas Andreas Neumann
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: