Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-6382

Refactor NamespaceAdmin to make it lightweight

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.0
    • Fix Version/s: 3.5.0
    • Component/s: Namespaces
    • Labels:
      None
    • Rank:
      1|hzzfhj:

      Description

      For enabling custom namespace mapping we need to fetch namespace metadata from different modules through NamespaceAdmin.

      The current implementation of NamespaceAdmin depends on a lot of stuff for its creation because NamespaceAdmin performs various operation like creating underlying namespace, deleting artifacts and other stuff on namespace delete.

      Also, StandaloneMain now gets an instance of TrackerAppCreationService from the injector which created a circular dependency because of various module now depending on NamespaceAdmin.

      For example for HDFS:

      1) Tried proxying co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository to support a circular dependency, but it is not an interface.
        while locating co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository
          for parameter 11 at co.cask.cdap.internal.app.namespace.DefaultNamespaceAdmin.<init>(DefaultNamespaceAdmin.java:96)
        while locating co.cask.cdap.internal.app.namespace.DefaultNamespaceAdmin
        at co.cask.cdap.app.guice.AppFabricServiceRuntimeModule$AppFabricServiceModule.configure(AppFabricServiceRuntimeModule.java:291)
        while locating co.cask.cdap.common.namespace.NamespaceAdmin
          for parameter 2 at co.cask.cdap.common.namespace.DefaultNamespacedLocationFactory.<init>(DefaultNamespacedLocationFactory.java:49)
        while locating co.cask.cdap.common.namespace.DefaultNamespacedLocationFactory
        while locating co.cask.cdap.common.namespace.NamespacedLocationFactory
          for parameter 1 at co.cask.cdap.internal.app.runtime.artifact.ArtifactStore.<init>(ArtifactStore.java:165)
        at co.cask.cdap.app.guice.AppFabricServiceRuntimeModule$AppFabricServiceModule.configure(AppFabricServiceRuntimeModule.java:289)
        while locating co.cask.cdap.internal.app.runtime.artifact.ArtifactStore
          for parameter 1 at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.<init>(ArtifactRepository.java:96)
        while locating co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository
          for parameter 1 at co.cask.cdap.TrackerAppCreationService.<init>(TrackerAppCreationService.java:69)
        while locating co.cask.cdap.TrackerAppCreationService
      

      To resolve this we should split the NamespaceAdmin operations into parts.
      One which will support definition operations (list/get/exists) and another will support manipulation operation (create/delete/deleteDatasets/updateProperties) this will allow other modules which just need to fetch NamespaceMeta to look up custom mapping depend on the first lightweight implementation and the second implementation can then depend on other modules to perform different operations. This is how we plan to resolve the circular dependency in guice injection for StandaloneMain.

      Note: This does not happen in distributed mode since MasterServiceMain does use TrackerApp like StandaloneMain.

      This change will also be needed for Hbase and Hive custom namespace mapping

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rsinha Rohit Sinha
                Reporter:
                rsinha Rohit Sinha
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: