To reproduce this issue, create a pipeline with a macro and deploy it. When you try to run the pipeline, the 'Runtime Arguments' modeless will pop up. If you check the 'Provided' checkbox for the macro and click 'Save', it isn't actually saved. If you now try to run the pipeline, the modeless will still pop up, with the Provided checkbox unchecked. See attached gif.