For high stakes systems an alternative I've used that gives much of the benefits while avoiding any risk is Continous Deployment to a staging environment - that way every change is instantly available for manual testing but there's still a deliberate "go live" moment with an extra human in the loop
@simon See, my experience is that a (dedicated) staging environment *feels* like it reduces risk, but actually increases it: you’re more likely to treat production as a pet, and batch up changes from fear, where spinning up dedicated test environments before merging changes (or, better still, blue-green deploying) actually tests for issues explicitly. But, I’ll grant, requires more trust in the process, as well as more process being built.
@stevemarshall yeah, I totally buy that. I think the staging environment option is good for when the test suite for an older system just isn't thorough enough to be trustworthy
@simon @stevemarshall the one time I worked on a system that had explicitly designed for a staging system for testing purposes had then (before my time there) also spun out testing, qa, qa2, qa3, etc, all of which showing subtle differences, all of which gave little confidence when a prod deploy happened such that manual QA had to happen AGAIN anyway
@simon Even then, I’d push for not having “a staging environment” and instead spinning up a new environment for each pull request. So instead of one staging environment, one per change. Does require the ability to spin up and tear down environments, though, and confidence that they won’t cost the earth.