Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-8152

Streaming source should have a way to access spark execution context

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0
    • Component/s: None
    • Labels:
      None
    • Release Notes:
      StreamingSource plugins now have access to the CDAP SparkExecutionContext to read from datasets and streams.
    • Rank:
      1|hzzswf:

      Description

      As of now we do not expose the spark execution context to the hydrator streaming source. We should have a getter to expose that in default in DefaultStreamingContext
      https://github.com/caskdata/cdap/blob/develop/cdap-app-templates/cdap-etl/hydrator-spark-core/src/main/java/co/cask/cdap/etl/spark/streaming/DefaultStreamingContext.java#L30

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ashau Albert Shau
                Reporter:
                abhinav Abhinav Bansal
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: