A pipeline that writes to BigQuery, it will create an external dataset to track lineage, metadata etc. for the sink. If the reference name given in the sink properties is not a valid dataset name, that throws an exception, which is expected. However, it should do so, when the pipeline is deployed, or right when it is started. In this case, the pipeline had 5 phases, and only when the 5th phase started, the error was raised. After many minutes of running.