Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-14398

Browsing folder with a lot of folders(files) under it in wrangler renders CDAP very slow

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.1.0
    • Component/s: Data Prep
    • Labels:
      None
    • Release Notes:
      Limiting max number of files showed in the S3 and GCS file browser to 1000.
    • Rank:
      1|i00i1b:

      Description

      Steps to reproduce:

      • Go to dataprep
      • Create an S3 connection
      • Navigate to a folder that potentially has 100s or 1000s of folders or files underneath.

      Issues

      • The UI shows the spinning icon forever
      • The backend accumulates a lot of logs. (The logs were rolled over to 5+ files within minutes)
      • CDAP becomes super slow
      • Eventually it gives up and CDAP shutsdown

      The above instance mentions only about S3. It could potentially happen with GCS or any other connection which has a lot of data. 

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vinisha Vinisha Shah
                Reporter:
                ajai Ajai Narayan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: