Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-2878

Semantics of dataset TTL are confusing

    Details

    • Release Notes:
      [Note: Please remove this JIRA from known-issues in Release notes.]
    • Rank:
      1|hzyvbj:

      Description

      For tables, we have a property

        /**
         * Property set to configure time-to-live on data within this dataset. The value given is in milliseconds.
         * Once a cell's data has surpassed the given value in age,
         * the cell's data will no longer be visible and may be garbage collected.
         */
      String PROPERTY_TTL = "dataset.table.ttl";
      

      This is documented in the javadoc and in the developer guide to be in milliseconds.

      However, when creating (or updating) a table through the REST API, the ttl is interpreted as seconds, and the DatasetInstanceHandler converts it to milliseconds; and when querying the properties of a dataset, it converts it back to seconds before returning it as JSON.

      This means that the two APIs to create a dataset (REST and Java) assign a different meaning to this property: If creating the dataset through REST, it has to be in seconds; if creating it in the application's configure method using createDataset(), it must be in milliseconds.

      That is inconsistent and counter-intuitive.

      It gets more complicated when a custom dataset has its own property that it uses to set the TTL for an embedded table. If that property does not have the exact same name (`dataset.table.ttl`), then it will not undergo the same translation in the HTTP handler.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shankar Shankar Selvam
                Reporter:
                andreas Andreas Neumann
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: