Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-14494

Provisioning tasks fail with firewall rule error on dataproc

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: 5.1.0
    • Component/s: Cloud Provisioner
    • Labels:
      None
    • Rank:
      1|i00imf:

      Description

      Pipelines fail while provisioning on dataproc clusters. Stacktrace below:

       
      java.io.IOException: Failed to connect to host REDACTED. Ensure that GCP Firewall Ingress Rules exist that allow ssh on port 22.
      at co.cask.cdap.runtime.spi.provisioner.dataproc.DataprocProvisioner.createSSHSession(DataprocProvisioner.java:214) ~[na:na]
      at co.cask.cdap.runtime.spi.provisioner.dataproc.DataprocProvisioner.initializeCluster(DataprocProvisioner.java:202) ~[na:na]
      at co.cask.cdap.internal.provision.task.ClusterInitializeSubtask.execute(ClusterInitializeSubtask.java:47) ~[na:na]
      at co.cask.cdap.internal.provision.task.ProvisioningSubtask.execute(ProvisioningSubtask.java:54) ~[na:na]
      at co.cask.cdap.internal.provision.task.ProvisioningTask.lambda$execute$0(ProvisioningTask.java:108) ~[na:na]
      at co.cask.cdap.common.service.Retries.callWithRetries(Retries.java:176) ~[na:na]
      at co.cask.cdap.common.service.Retries.callWithInterruptibleRetries(Retries.java:249) ~[na:na]
      at co.cask.cdap.internal.provision.task.ProvisioningTask.execute(ProvisioningTask.java:108) ~[na:na]
      at co.cask.cdap.internal.provision.ProvisioningService.lambda$null$18(ProvisioningService.java:576) [na:na]
      at co.cask.cdap.internal.provision.ProvisioningService.runWithProgramLogging(ProvisioningService.java:775) [na:na]
      at co.cask.cdap.internal.provision.ProvisioningService.lambda$createProvisionTask$19(ProvisioningService.java:572) [na:na]
      at co.cask.cdap.common.async.KeyedExecutor.lambda$submit$0(KeyedExecutor.java:62) ~[na:na]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_181]
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_181]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_181]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_181]
      at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
      Caused by: java.io.IOException: com.jcraft.jsch.JSchException: java.net.ConnectException: Connection timed out (Connection timed out)
      at co.cask.cdap.common.ssh.DefaultSSHSession.<init>(DefaultSSHSession.java:80) ~[na:na]
      at co.cask.cdap.internal.provision.DefaultSSHContext.createSSHSession(DefaultSSHContext.java:114) ~[na:na]
      at co.cask.cdap.runtime.spi.ssh.SSHContext.createSSHSession(SSHContext.java:92) ~[na:na]
      at co.cask.cdap.runtime.spi.ssh.SSHContext.createSSHSession(SSHContext.java:80) ~[na:na]
      at co.cask.cdap.runtime.spi.provisioner.dataproc.DataprocProvisioner.createSSHSession(DataprocProvisioner.java:211) ~[na:na]
      ... 16 common frames omitted
      Caused by: com.jcraft.jsch.JSchException: java.net.ConnectException: Connection timed out (Connection timed out)
      at com.jcraft.jsch.Util.createSocket(Util.java:349) ~[com.jcraft.jsch-0.1.54.jar:na]
      at com.jcraft.jsch.Session.connect(Session.java:215) ~[com.jcraft.jsch-0.1.54.jar:na]
      at com.jcraft.jsch.Session.connect(Session.java:183) ~[com.jcraft.jsch-0.1.54.jar:na]
      at co.cask.cdap.common.ssh.DefaultSSHSession.<init>(DefaultSSHSession.java:77) ~[na:na]
      ... 20 common frames omitted
      Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
      at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_181]

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              sree Sreevatsan Raman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: