Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-7117

Support transaction in checkpoint for Spark Streaming

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.0, 3.4.0
    • Fix Version/s: 4.0.0
    • Component/s: None
    • Labels:
      None
    • Release Notes:
      Added support for checkpointing in Spark Streaming programs
    • Rank:
      1|hzzknz:

      Description

      While CDAP supports checkpointing in Spark Streaming, transaction is not supported. This is because functions applied on the DStream need to be serializable and SparkExecutionContext is not.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                terence Terence Yim
                Reporter:
                terence Terence Yim
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: