Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-16822

Dataproc provisioner breaks if staging bucket is deleted

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 6.3.0
    • Component/s: Cloud Provisioner
    • Labels:
      None
    • Rank:
      1|i00xcn:

      Description

      I have the most recent build of the sandbox, and my pipeline runs fail within a few seconds with:

      2020-05-14 15:10:41,852 - ERROR [provisioning-service-2:i.c.c.i.p.t.ProvisioningTask@151] - PROVISION task failed in REQUESTING_CREATE state for program run program_run:default.t.-SNAPSHOT.workflow.DataPipelineWorkflow.bc504d34-962f-11ea-b486-000000f78a44.
      com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Google Cloud Storage bucket does not exist 'dataproc-ad333953-e9aa-4f85-a072-3f500781d65e-us-east1'.
      	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:49) ~[na:na]
      	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[na:na]
      	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[na:na]
      	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97) ~[na:na]
      	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68) ~[na:na]
      	at com.google.common.util.concurrent.Futures$4.run(Futures.java:1123) ~[com.google.guava.guava-13.0.1.jar:na]
      	at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435) ~[na:na]
      	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900) ~[com.google.guava.guava-13.0.1.jar:na]
      	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811) ~[com.google.guava.guava-13.0.1.jar:na]
      	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675) ~[com.google.guava.guava-13.0.1.jar:na]
      	at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:520) ~[na:na]
      	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:495) ~[na:na]
      	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[na:na]
      	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[na:na]
      	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[na:na]
      	at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700) ~[na:na]
      	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[na:na]
      	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[na:na]
      	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[na:na]
      	at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:399) ~[na:na]
      	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:510) ~[na:na]
      	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:66) ~[na:na]
      	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:630) ~[na:na]
      	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:518) ~[na:na]
      	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:692) ~[na:na]
      	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:681) ~[na:na]
      	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[na:na]
      	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[na:na]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_221]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_221]
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_221]
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_221]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_221]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_221]
      	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_221]
      Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Google Cloud Storage bucket does not exist 'dataproc-ad333953-e9aa-4f85-a072-3f500781d65e-us-east1'.
      	at io.grpc.Status.asRuntimeException(Status.java:533) ~[na:na]
      	... 24 common frames omitted
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vinisha Vinisha Shah
                Reporter:
                ashau Albert Shau
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: