Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-3890

Dataset Service performance degradation

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.2.0, 3.1.0, 3.0.0
    • Fix Version/s: 3.5.0
    • Component/s: Datasets
    • Labels:
    • Release Notes:
      Improved performance of Dataset Service.
    • Rank:
      1|hzz14n:

      Description

      After CDAP 2.8.1 release, we added at least two expensive operations to getDataset call in DatasetService -

      1. Check whether namespace exists. This makes a remote call to app-fabric service, which in turn makes a call to Dataset Service to fetch app meta table.
      2. Register usage. This again makes a remote call to DatasetService to fetch usage dataset.

      These are causing every getDataset call to become at least 3 getDataset calls, and causing timeouts.

      This also makes a getDataset call depend on another getDataset call, thus bringing in dependency between Dataset Service executor threads. This can lead to executor threads waiting for a call that can never get executed due to unavailability of executor threads.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                andreas Andreas Neumann
                Reporter:
                poorna Poorna Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: