Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10355

Database Batch Source doesn't handle $CONDITIONS when getting a schema

    Details

    • Sprint:
      App Eng Sprint 2
    • Release Notes:
      Database Batch Source now handles $CONDITIONS when getting a schema
    • Rank:
      1|hzzcnz:

      Description

      The documentation (http://docs.cask.co/cdap/3.4.0/en/cdap-apps/hydrator/hydrator-plugins/batchsources/database.html) for the Database Batch Source includes the importQuery:

      The SELECT query to use to import data from the specified table. You can specify an arbitrary number of columns to import, or import all columns using *. The Query should contain the ‘$CONDITIONS’ string. For example, ‘SELECT * FROM table WHERE $CONDITIONS’. The ‘$CONDITIONS’ string will be replaced by ‘splitBy’ field limits specified by the bounding query. The ‘$CONDITIONS’ string is not required if numSplits is set to one.
      

      If you set a "$CONDITIONS" string, then when you use the "Get Schema" button, that exact same "Import Query" will be used, but it will fail because the "$CONDITIONS" string will not be replaced correctly.

      See the attached screenshot.

      I am not sure what the correct fix is. Should the $CONDITIONS string be replaced, and that is what is missing? Does the user need to edit the query, and better instructions are required?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shashank Shashank
                Reporter:
                John John Jackson
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: