Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-964

Simplify data ingestion to stream

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: App Fabric, Datasets
    • Labels:
      None
    • Rank:
      1|hzyjdz:

      Description

      Currently one always need to write a Program (usually a Flow) to get data from external source (e.g. Kafka) and persist it to dataset so that multiple programs can share and process the same dataset.

      As part of ZAP, the platform should have the capability to support common data source for ingesting data into stream. It should be designed with a pluggable architecture in mind so that it's easy to add new source/sink type. Also, it should be possible to define simply data cleansing logic before writing into sink.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gokul Gokul Gunasekaran
                Reporter:
                terence Terence Yim
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: