Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-6995

Log Saver clean-up job to remove old files that do not have metadata

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Log
    • Labels:
      None
    • Release Notes:
      Adding capability to clean up log files which does not have corresponding metadata.
    • Rank:
      1|hzzjp3:

      Description

      Since file create/delete operations and HBase metadata update operations cannot be done in the same transaction, there are instances where some log files may not get cleaned up. It would be good to have a clean-up job that periodically scans the logs namespace directory for files much older than retention time and deletes them if it cannot find metadata for those files in HBase.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vinisha Vinisha Shah
                Reporter:
                poorna Poorna Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: