Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5.0
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      Added a streaming pipeline type to Hydrator for realtime pipelines run using Spark Streaming.
    • Rank:
      1|hzzffb:

      Description

      This cdap-data-streams artifact will be an experimental Hydrator application that uses Spark Streaming as the execution engine. Logically, it will be for "realtime" pipelines that operate on streams of data.

      It will support many of the existing plugin types:

      transform
      sparkcompute
      joiner
      batchaggregator
      batchsink

      And introduce a new plugin type:

      streamsource

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: