Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-7031

Incorrect log statement in run record corrector

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.6.0, 3.5.1, 3.4.3
    • Fix Version/s: 4.0.0
    • Component/s: App Fabric
    • Labels:
    • Release Notes:
      Fix misleading log messages from the RunRecord corrector.
    • Rank:
      1|hzzjzb:

      Description

      There is a race condition in run record corrector:
      1. Program is running according to run record (and it actually is).
      2. Next, it checks live-info to see if the program is still running. By this point, the program could have stopped/completely successfully. We don't see it in the live-info as running, so run record corrector things it has to correct the run record.
      3. It logs "Fixing RunRecord <runId> in program...", and attempts to update the run record to ERROR status.
      https://github.com/caskdata/cdap/blob/release/3.4/cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/services/ProgramLifecycleService.java#L649-L653
      4. Fortunately, it is using a compareAndSet method, to handle this race condition, so the run record isn't actually updated. I can see from the UI that it is in COMPLETED state. However, the log statement is very misleading.

      The fix could be to have compareAndSet return a boolean value, with which we could use to avoid logging the statement saying that the run record is being corrected.

        Attachments

          Activity

            People

            • Assignee:
              ali.anwar Ali Anwar
              Reporter:
              ali.anwar Ali Anwar
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: