Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-16075

BQTables with decimal types does not work with arithmetic operations

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 6.3.0
    • Component/s: Data Prep
    • Rank:
      1|i00scv:

      Description

      Applying the following transformations on decimal type fails with wrangler

      filter-row-if-true trip_distance < 1.0
      filter-row-if-true passenger_count < 2.0
      filter-row-if-true total_amount <= 0.0
      set-column cost_per_mile trip_distance / total_amount
      set-column cost_per_passenger passenger_count / total_amount
      

      Error:

      {"stages":[{"icon":"icon-DataPreparation","type":"transform","plugin":{"name":"Wrangler","artifact":{"name":"wrangler-transform","version":"4.1.1","scope":"SYSTEM"},"properties":{"field":"*","precondition":"false","threshold":"1","workspaceId":"0ecf731f-6475-49a2-9335-05e819397557","directives":"filter-row-if-true trip_distance < 1.0\nfilter-row-if-true passenger_count < 2.0\nfilter-row-if-true total_amount <= 0.0\nset-column cost_per_mile trip_distance / total_amount\nset-column cost_per_passenger passenger_count / total_amount","schema":"{\"type\":\"record\",\"name\":\"etlSchemaBody\",\"fields\":[{\"name\":\"vendor_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"pickup_datetime\",\"type\":[\"string\",\"null\"]},{\"name\":\"dropoff_datetime\",\"type\":[\"string\",\"null\"]},{\"name\":\"passenger_count\",\"type\":[\"long\",\"null\"]},{\"name\":\"trip_distance\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"rate_code\",\"type\":[\"string\",\"null\"]},{\"name\":\"store_and_fwd_flag\",\"type\":[\"string\",\"null\"]},{\"name\":\"payment_type\",\"type\":[\"string\",\"null\"]},{\"name\":\"fare_amount\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"extra\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"mta_tax\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"tip_amount\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"tolls_amount\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"imp_surcharge\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"total_amount\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"pickup_location_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"dropoff_location_id\",\"type\":[\"string\",\"null\"]},{\"name\":\"cost_per_mile\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]},{\"name\":\"cost_per_passenger\",\"type\":[{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":38,\"scale\":35},\"null\"]}]}"},"label":"Wrangler"}}]}
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vinisha Vinisha Shah
                Reporter:
                sree Sreevatsan Raman
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: