Pattern: Parallelization
Pattern: Parallelization
Category: Orchestration Source: FOR-0012 Status: Documented
When to Use
When a large job can be split into independent chunks that do not depend on each other and can be processed simultaneously. Best when throughput matters more than sequential reasoning, and when results can be merged afterward.
How It Works
- A coordinator agent receives a large task
- It decomposes the task into independent subtasks
- It checks available resources and spawns parallel workers, one per subtask
- Each worker processes its subtask independently
- Results are collected, normalized to a common format, and merged
- A final synthesis step produces the unified output
Example
A digital talent that monitors multiple social media channels for a client. Rather than checking each platform sequentially, parallel workers each monitor one channel (Twitter, LinkedIn, Instagram). Results are collected, normalized into a common alert format, and merged into a daily digest. Total processing time equals the slowest single channel, not the sum of all channels.
Tradeoffs
| Pro | Con |
|---|---|
| Significant speedup for independent tasks | Subtasks must be truly independent — dependencies break it |
| Scales horizontally by adding workers | Merging results adds complexity |
| Failure in one worker does not block others | Harder to debug than sequential processing |
| Efficient use of available resources | Cost scales linearly with worker count |
Factory Usage
Not yet used explicitly in the factory, but a natural fit for future production line stages where multiple independent validations or generations could run simultaneously (e.g., parallel QA checks on different aspects of a digital talent).