Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-5675

CDAP should define a new abstraction for tracking external sources/sinks

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: CDAP
    • Labels:
      None
    • Rank:
      1|hzzb93:

      Description

      In CDAP-5424, we use datasets to track lineage of external sources/sinks. That a short term fix, because reusing datasets abstraction leads to confusion with names and semantics.
      For eg:

      • Given a dataset name we cannot figure out if it is a regular dataset or represents an external entity.
      • Using an external dataset in a map reduce leads to different semantics than datasets due to CDAP-5674

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bhooshan Bhooshan Mogal
                Reporter:
                poorna Poorna Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: