Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-15369

Dataproc runtime fails if project has os login enabled

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.0.0, 5.1.2
    • Fix Version/s: 6.0.0
    • Component/s: Cloud Provisioner
    • Labels:
    • Release Notes:
      Fix Dataproc runtime for GCP projects where OS Login is enabled.
    • Rank:
      1|i00nyf:

      Description

      All pipelines using dataproc profile against a project will fail if os login is enabled in that project:

      2019-05-09 21:48:43,123 - ERROR [runtime-startup-4:i.c.c.i.a.r.d.r.RemoteExecutionTwillRunnerService@543] - Fail to start program run program_run:default.F2F.-SNAPSHOT.workflow.DataPipelineWorkflow.db1234f7-72a3-11e9-9830-2aed5aec4f65
      java.io.IOException: com.jcraft.jsch.JSchException: Auth fail
      	at io.cdap.cdap.common.ssh.DefaultSSHSession.<init>(DefaultSSHSession.java:82) ~[na:na]
      	at io.cdap.cdap.internal.app.runtime.distributed.remote.RemoteExecutionTwillPreparer.lambda$start$0(RemoteExecutionTwillPreparer.java:429) ~[na:na]
      	at io.cdap.cdap.internal.app.runtime.distributed.remote.RemoteExecutionTwillRunnerService$ControllerFactory.lambda$create$0(RemoteExecutionTwillRunnerService.java:519) ~[na:na]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_212]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_212]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_212]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_212]
      	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
      Caused by: com.jcraft.jsch.JSchException: Auth fail
      	at com.jcraft.jsch.Session.connect(Session.java:519) ~[com.jcraft.jsch-0.1.54.jar:na]
      	at com.jcraft.jsch.Session.connect(Session.java:183) ~[com.jcraft.jsch-0.1.54.jar:na]
      	at io.cdap.cdap.common.ssh.DefaultSSHSession.<init>(DefaultSSHSession.java:79) ~[na:na]
      	... 7 common frames omitted 

      To reproduce this, set the project-level metadata (at http://console.cloud.google.com/compute/metadata) to:

      Key Value
      enable-oslogin true

      A workaround for this is to set the value to false or to remove the key completely, as it then defaults to false.

        Attachments

          Activity

            People

            • Assignee:
              ali.anwar Ali Anwar
              Reporter:
              ali.anwar Ali Anwar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: