Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-13816

Whole file ingest plugin should not fail if the directory is empty

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Rank:
      1|i00etz:

      Description

      If the source directory is empty, the whole file ingest tries to initialize a priority queue with 0 size which throws an illegal argument exception. 

       

      https://github.com/data-integrations/whole-file-ingest/blob/develop/src/main/java/co/cask/hydrator/plugin/batch/file/MetadataInputFormat.java#L124

       

      The code path should return empty splits in this case without trying to initialize a 0 size priority queue https://docs.oracle.com/javase/7/docs/api/java/util/PriorityQueue.html#PriorityQueue(int)

       

      as a workaround the number of splits can be set to a large value (ex: 1000) so that priority queue is initialized as 1

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              sree Sreevatsan Raman
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: