Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-4865

Metadata Kafka Publishing is not working in SDK

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.3.1
    • Fix Version/s: 3.3.1
    • Component/s: Metadata
    • Labels:
      None
    • Release Notes:
      Hide
      Enhanced the CDAP SDK to be able to publish metadata updates to an external Kafka, identified by the configuration parameter metadata.updates.kafka.broker.list. Publishing can be enabled by setting metadata.updates.publish.enabled to true. Updates are published to the Kafka topic identified by metadata.updates.kafka.topic.
      Show
      Enhanced the CDAP SDK to be able to publish metadata updates to an external Kafka, identified by the configuration parameter metadata.updates.kafka.broker.list. Publishing can be enabled by setting metadata.updates.publish.enabled to true. Updates are published to the Kafka topic identified by metadata.updates.kafka.topic.
    • Rank:
      1|hzz69b:

      Description

      Built SDK from release/3.3 branch and started CDAP and I see the following messages in cdap-debug.log :

      2016-02-02 15:06:49,227 - DEBUG [Endure-Service-:c.c.c.c.s.RetryOnStartFailureService$1@66] - Exception raised when starting service 
      java.lang.NoClassDefFoundError: org/apache/twill/internal/kafka/client/ByteBufferEncoder
              at co.cask.cdap.data2.metadata.publisher.KafkaMetadataChangePublisher$1.get(KafkaMetadataChangePublisher.java:69) ~[co.cask.cdap.cdap-data-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.metadata.publisher.KafkaMetadataChangePublisher$1.get(KafkaMetadataChangePublisher.java:64) ~[co.cask.cdap.cdap-data-fabric-3.3.1-SNAPSHOT.jar:na]
              at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:116) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.data2.metadata.publisher.KafkaMetadataChangePublisher.publish(KafkaMetadataChangePublisher.java:88) ~[co.cask.cdap.cdap-data-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.publish(DefaultMetadataStore.java:425) ~[co.cask.cdap.cdap-data-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.metadata.store.DefaultMetadataStore.addTags(DefaultMetadataStore.java:157) ~[co.cask.cdap.cdap-data-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.metadata.system.AbstractSystemMetadataWriter.write(AbstractSystemMetadataWriter.java:83) ~[co.cask.cdap.cdap-data-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.writeSystemMetadata(ArtifactRepository.java:781) ~[co.cask.cdap.cdap-app-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.addArtifact(ArtifactRepository.java:413) ~[co.cask.cdap.cdap-app-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.addSystemArtifact(ArtifactRepository.java:609) ~[co.cask.cdap.cdap-app-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.addSystemArtifacts(ArtifactRepository.java:589) ~[co.cask.cdap.cdap-app-fabric-3.3.1-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.runtime.artifact.SystemArtifactLoader$1$1.doStart(SystemArtifactLoader.java:45) ~[co.cask.cdap.cdap-app-fabric-3.3.1-SNAPSHOT.jar:na]
              at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.common.service.RetryOnStartFailureService$1.run(RetryOnStartFailureService.java:62) ~[co.cask.cdap.cdap-common-3.3.1-SNAPSHOT.jar:na]
      

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              gokul Gokul Gunasekaran
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: