Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-6386

Updating a dataset should not disable and re-enable explore

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.5.0, 3.4.3, 3.3.5, 3.3.4
    • Fix Version/s: 3.5.0
    • Component/s: Datasets, Explore
    • Labels:
      None
    • Release Notes:
      Fixed an issue where updating the properties of a dataset deleted all of its partitions in Hive.
    • Rank:
      1|hzzfif:

      Description

      Updating a dataset currently disables Explore, then calls upgrade() on the dataset, then enables Explore again.

      Disabling, however, drops the existing Hive table, and that loses all the existing partitions. Enabling explore does not add these partitions back, it only creates an empty Hive table.

      That means that every time the properties for a partitioned file set are updated, it loses all its partitions in Hive forever.

      The same happens when creating a dataset and dataset.unchecked.upgrade is true in cdap-site.xml, and that means, with this flag on, redeploying an app deletes all partitions in the datasets created by this app.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                andreas Andreas Neumann
                Reporter:
                andreas Andreas Neumann
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: