AI Image Processing Retry Policy

Definition. The rules governing how a failed image processing job is retried, including which errors are retryable, the backoff, and the maximum attempts. Image processing failures are often transient, such as a GPU out-of-memory or a timeout, and a well-designed policy recovers most of them.

How it works. Classify errors: transient, such as timeouts and resource exhaustion, versus permanent, such as corrupt input or unsupported format. Retry transient errors with backoff and a small cap. For out-of-memory, retry with a smaller tile or batch size rather than identically. Route permanent failures to a quarantine with the reason, and never retry them.

Example. A batch of 50,000 images has 1,100 failures. Analysis shows 780 are out-of-memory on very large images and 320 are corrupt files. The retry policy retries out-of-memory with a reduced tile size, recovering 741 of the 780. The 320 corrupt files are quarantined with the reason, and 39 remain unrecoverable and are reported. Total recovered is 67 percent of failures.

Use case. Retry policy design is required for any image pipeline at scale, and it is the difference between a batch that loses 2 percent of items and one that loses 0.1 percent. It is also a cost control, since blind retries waste compute.

Warnings. Retrying an out-of-memory error identically will fail again, so the retry must change a parameter. Never retry a corrupt file, since it will never succeed. And never retry without a cap, since a persistent failure will consume the batch.

Limitations. Error classification requires the processing library to return distinguishable errors, which is not always the case. Parameter adjustment on retry also complicates the pipeline, since the output may differ from the standard path.

← Browse all glossary terms