Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-16174

Delete temporary tables after pipeline completes

    Details

    • Rank:
      1|i00t0v:

      Description

      Today when we run a pipeline with BQ source, reading from a partitioned table and writing to another BQ table, we see temporary tables created with expiration of 1 day.

       

      We shouldn't force the expiration time to be one day, if the pipeline completes we should delete them instead of forcing user to wait for a day for the table to be gone. We already do clean up some of the GCS buckets we create in other plugins. 

        Attachments

          Activity

            People

            • Assignee:
              yaojie Yaojie Feng
              Reporter:
              ajai Ajai Narayan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: