Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-15883

Dataproc provisioner fails on legacy networks

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Cloud Provisioner
    • Labels:
      None
    • Rank:
      1|i00r4f:

      Description

      The dataproc provisioner will check that the configured network contains subnets before trying to create the cluster. Legacy VPC networks do not have subnets, so the provisioner will fail with something like:

      2019-09-09 13:38:38,316 - ERROR [provisioning-service-3:i.c.c.i.p.t.ProvisioningTask@151] - PROVISION task failed in REQUESTING_CREATE state for program run program_run:default.GCSCopyTest.-SNAPSHOT.workflow.DataPipelineWorkflow.1fc53344-d307-11e9-b1cc-42010ad20012.
      java.lang.IllegalArgumentException: Network 'default' in project 'xxxxx' does not contain any subnets. Please create a subnet or use a different network.
              at io.cdap.cdap.runtime.spi.provisioner.dataproc.DataprocClient.fromConf(DataprocClient.java:163) ~[na:na]
              at io.cdap.cdap.runtime.spi.provisioner.dataproc.DataprocProvisioner.createCluster(DataprocProvisioner.java:166) ~[na:na]
              at io.cdap.cdap.internal.provision.task.ClusterCreateSubtask.execute(ClusterCreateSubtask.java:43) ~[na:na]
              at io.cdap.cdap.internal.provision.task.ProvisioningSubtask.execute(ProvisioningSubtask.java:54) ~[na:na]
              at io.cdap.cdap.internal.provision.task.ProvisioningTask.lambda$executeOnce$0(ProvisioningTask.java:123) ~[na:na]
              at io.cdap.cdap.common.service.Retries.callWithRetries(Retries.java:183) ~[na:na]
              at io.cdap.cdap.common.service.Retries.callWithInterruptibleRetries(Retries.java:257) ~[na:na]
              at io.cdap.cdap.internal.provision.task.ProvisioningTask.executeOnce(ProvisioningTask.java:123) ~[na:na]
              at io.cdap.cdap.internal.provision.ProvisioningService.lambda$null$16(ProvisioningService.java:567) [na:na]
              at io.cdap.cdap.internal.provision.ProvisioningService.callWithProgramLogging(ProvisioningService.java:782) [na:na]
              at io.cdap.cdap.internal.provision.ProvisioningService.lambda$null$17(ProvisioningService.java:565) [na:na]
              at io.cdap.cdap.common.async.KeyedExecutor$2.run(KeyedExecutor.java:99) ~[na:na]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_181]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_181]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_181]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_181]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_181]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_181]
              at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
      
      

        Attachments

          Activity

            People

            • Assignee:
              trishka Trishka
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: