Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-4967

Record schema serialization/deserialization error

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.2
    • Component/s: App Fabric
    • Labels:
      None
    • Release Notes:
      Fixed a schema parsing bug that prevented use of schemas where a record is used as a top level field and also inside another record field.
    • Rank:
      1|hzz6w7:

      Description

      There is some bug either in schema serialization or schema deserialization for records.

      The following test case fails:

        @Test
        public void testCodec() throws UnsupportedTypeException, IOException {
          ReflectionSchemaGenerator reflectionSchemaGenerator = new ReflectionSchemaGenerator();
          Schema schema = reflectionSchemaGenerator.generate(SomeObject.class);
          Schema.parseJson(schema.toString());
        }
      
        private static class SomeObject {
          Record x;
          OuterRecord y;
        }
      
        private static class Record {
          int x;
        }
      
        private static class OuterRecord {
          Record rec;
        }
      

      with an exception:

      java.lang.IllegalArgumentException: Undefined schema co.cask.cdap.etl.batch.mock.DummyTest$Record
      	at co.cask.cdap.api.data.schema.Schema.resolveSchema(Schema.java:821)
      	at co.cask.cdap.api.data.schema.Schema.resolveSchema(Schema.java:809)
      	at co.cask.cdap.api.data.schema.Schema.populateRecordFields(Schema.java:772)
      	at co.cask.cdap.api.data.schema.Schema.<init>(Schema.java:394)
      	at co.cask.cdap.api.data.schema.Schema.recordOf(Schema.java:332)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.readRecord(SchemaTypeAdapter.java:218)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.read(SchemaTypeAdapter.java:109)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.readUnion(SchemaTypeAdapter.java:133)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.read(SchemaTypeAdapter.java:88)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.readInnerSchema(SchemaTypeAdapter.java:234)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.readRecord(SchemaTypeAdapter.java:214)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.read(SchemaTypeAdapter.java:109)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.read(SchemaTypeAdapter.java:62)
      	at co.cask.cdap.internal.io.SchemaTypeAdapter.read(SchemaTypeAdapter.java:48)
      	at com.google.gson.TypeAdapter.fromJson(TypeAdapter.java:256)
      	at com.google.gson.TypeAdapter.fromJson(TypeAdapter.java:269)
      	at co.cask.cdap.api.data.schema.Schema.parseJson(Schema.java:134)
      	at co.cask.cdap.etl.batch.mock.DummyTest.testCodec(DummyTest.java:34)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ashau Albert Shau
                Reporter:
                ashau Albert Shau
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: