Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-7281

PartitionedFileSet accepts partition keys that contain extra fields

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.1
    • Fix Version/s: 3.6.0
    • Component/s: API, Datasets
    • Labels:
      None
    • Release Notes:
      Improved the PartitionedFileSet to validate the schema of a partition key. Note that this will break code that uses incorrect partition keys, which was previously silently ignored.
    • Rank:
      1|hzzlyf:

      Description

      Accepting partition keys that contain extra fields creates the impression that these fields are used and stored by the dataset, and also retrievable. However, these extra fields are simply ignored.

      PartitionedFileSet should reject partition keys that contain extra fields, and the documentation should be updated to make that clear. Also, it would be useful to have a PartitionKey.builder(PartitionKey) that enforces the validity of all fields.

        Attachments

          Activity

            People

            • Assignee:
              andreas Andreas Neumann
              Reporter:
              andreas Andreas Neumann
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: