One reason for the flakiness appears to be that testStartProgramWithDisabledRuntimeArgs() - which is misnamed because it tests a disabled profile - starts a workflow but does not properly wait for it to finish. This workflow is then still running when the test shuts down. That appears to trigger a deadlock in MapReduce (see
CDAP-14076) that can cause the build to hang.
The method should be renamed and it needs to use Tasks.waitFor() to assert that the workflow has finished.