Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10567

Python transform does not decode simple types correctly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Pipelines
    • Labels:
    • Sprint:
      App Eng Sprint 2
    • Release Notes:
      Python evaluator can now handle float and double data types
    • Rank:
      1|hzzn5z:

      Description

      Python transform fails with the following error when a float type is used in output schema.

      This is because the python evaluator doesn't handle simple types correctly. Fix is to use the following code while handling float type

      case LONG:
              return ((Number) object).longValue();
      case FLOAT:
             return ((Number) object).floatValue();
       case BYTES:
              List byteArr = jsObject2List(object);
              byte[] output = new byte[byteArr.size()];
              for (int i = 0; i < output.length; i++) {
                // everything is a number
                output[i] = ((Number) byteArr.get(i)).byteValue();
              }
              return output;
       case DOUBLE:
              // case so that if it's not really a double it will fail. This is possible for unions,
              // where we don't know what the actual type of the object should be.
              return ((Number) object).doubleValue();
      

      https://github.com/caskdata/hydrator-plugins/blob/develop/core-plugins/src/main/java/co/cask/hydrator/plugin/transform/PythonEvaluator.java

        Attachments

          Activity

            People

            • Assignee:
              abhinav Abhinav Bansal
              Reporter:
              sree Sreevatsan Raman
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: