Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-13554

Malformed json exceptions in Mapreduce context config while deserializing plugins map

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.0.0, 4.3.5
    • Component/s: None
    • Labels:
    • Release Notes:
      Fixed a bug that caused mapreduce pipelines to fail when using too many macros
    • Rank:
      1|i00dfz:

      Description

      In MapReduceContextConfig class, while getting plugins map and other properties, We are getting values from hConf using `hConf.get` instead of `hConf.getRaw`. `hConf.get` method will try to substitute the values and this might cause malformed JSON exception when the variable values get substituted. we wouldn't want substitution to happen and we should use `getRaw` instead.

      2018-05-31 17:21:21,551 - ERROR [main:c.c.c.i.a.r.b.d.MapReduceContainerLauncher@120] - Exception raised when calling org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(String[]) method
      com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Unterminated object at line 1 column 2432
              at com.google.gson.Gson.fromJson(Gson.java:818) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.Gson.fromJson(Gson.java:768) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.Gson.fromJson(Gson.java:717) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at co.cask.cdap.internal.app.runtime.batch.MapReduceContextConfig.getPlugins(MapReduceContextConfig.java:190) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at co.cask.cdap.internal.app.runtime.batch.MapReduceClassLoader$Parameters.<init>(MapReduceClassLoader.java:244) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at co.cask.cdap.internal.app.runtime.batch.MapReduceClassLoader$Parameters.<init>(MapReduceClassLoader.java:240) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at co.cask.cdap.internal.app.runtime.batch.MapReduceClassLoader$Parameters.<init>(MapReduceClassLoader.java:236) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at co.cask.cdap.internal.app.runtime.batch.MapReduceClassLoader.<init>(MapReduceClassLoader.java:96) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_171]
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_171]
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_171]
              at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_171]
              at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_171]
              at co.cask.cdap.internal.app.runtime.batch.distributed.MapReduceContainerLauncher.launch(MapReduceContainerLauncher.java:84) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(Unknown Source) [launcher.jar:na]
      Caused by: com.google.gson.stream.MalformedJsonException: Unterminated object at line 1 column 2432
              at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1505) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:480) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.stream.JsonReader.peek(JsonReader.java:414) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at co.cask.cdap.internal.app.ApplicationSpecificationAdapter$AppSpecTypeAdapterFactory$1.read(ApplicationSpecificationAdapter.java:226) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at co.cask.cdap.internal.app.ApplicationSpecificationAdapter$AppSpecTypeAdapterFactory$1.read(ApplicationSpecificationAdapter.java:200) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172) ~[com.google.code.gson.gson-2.2.4.jar:na]
              at co.cask.cdap.internal.app.ApplicationSpecificationAdapter$AppSpecTypeAdapterFactory$1.read(ApplicationSpecificationAdapter.java:227) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at co.cask.cdap.internal.app.ApplicationSpecificationAdapter$AppSpecTypeAdapterFactory$1.read(ApplicationSpecificationAdapter.java:200) ~[co.cask.cdap.cdap-app-fabric-4.3.4.jar:na]
              at com.google.gson.Gson.fromJson(Gson.java:803) ~[com.google.code.gson.gson-2.2.4.jar:na]
              ... 14 common frames omitted 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ashau Albert Shau
                Reporter:
                shankar Shankar Selvam
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: