AI Image Pipeline Rollback

Definition. Reverting an image pipeline to a previous model or parameter set, and reprocessing the affected assets, when a change is found to degrade quality. Rollback is complicated by the fact that outputs are already published, so it requires both a code revert and a reprocessing job.

How it works. Keep the previous model and parameters deployable. On detecting a regression, revert the configuration immediately to stop further bad output. Then identify affected assets by the provenance record and reprocess them. Verify a sample before reprocessing the full set, since a second bad change compounds the problem.

Example. A model update degrades text rendering. The configuration is reverted in 10 minutes, stopping new bad output. Provenance shows 8,400 assets were processed with the new model. A sample of 50 is reprocessed with the old model and verified, then the full 8,400 are reprocessed over 3 hours. Without provenance, the team could not have identified which assets were affected.

Use case. Rollback capability is required for any image pipeline, and it is the control that limits the impact of a bad change. It is also the reason to record provenance for every asset.

Warnings. Reverting the configuration without reprocessing leaves bad assets published. Never reprocess the full set without verifying a sample first. And never assume you can identify affected assets without provenance.

Limitations. Reprocessing is expensive and slow at scale, and it may not be possible if the previous model is no longer available. Rollback also cannot undo the exposure of bad assets that were already seen by users.

← Browse all glossary terms