Mastering Async and Concurrency in .NET - Part 4: Parallel Processing Patterns

Introduction

In Part 3, we settled the debate about async and threads. Now we’re going to explore parallel processing patterns - when to use Parallel.ForEach vs Task.WhenAll, understanding execution order, and choosing the right tool for your workload.

Parallel.ForEach vs Task.WhenAll: The Core Distinction

Both Parallel.ForEach and Task.WhenAll leverage the .NET ThreadPool, but they serve fundamentally different purposes:

Feature	Parallel.ForEach	Task.WhenAll
Primary Use Case	CPU-bound data parallelism	I/O-bound async operations
Execution Model	Synchronous, blocks calling thread	Asynchronous, non-blocking
Thread Control	Limited (MaxDegreeOfParallelism)	Explicit task creation
Partitioning	Automatic chunking	Manual (no built-in)
Best For	Number-crunching, image processing	Web requests, file I/O, DB calls

Important: Task.WhenAll Is Async, Not Parallel (CPU Sense)

Task.WhenAll enables asynchronous concurrency, not CPU parallelism.
When used with I/O-bound operations, Task.WhenAll:
Frees threads during waits (no threads blocked during I/O)
May use zero threads during the wait period
Does NOT guarantee simultaneous execution on different CPU cores
This is fundamentally different from Parallel.ForEach, which:
Explicitly uses multiple CPU threads
Keeps threads busy with CPU-bound work
Distributes work across multiple cores

Parallel.ForEach: CPU-Bound Data Parallelism

Design Goals

Parallel.ForEach is part of the Task Parallel Library (TPL) designed for CPU-intensive work that can be broken into independent iterations.

Example: Image Processing

public void ProcessImages(string[] imagePaths)
{
    var options = new ParallelOptions
    {
        MaxDegreeOfParallelism = Environment.ProcessorCount
    };

    Parallel.ForEach(imagePaths, options, imagePath =>
    {
        // CPU-intensive: load, resize, apply filters
        var image = LoadImage(imagePath);
        ResizeImage(image);
        ApplyFilters(image);
        SaveImage(image);
    });
}

Why this works:

Each iteration is CPU-bound
Work saturates available cores
No I/O waits that would waste threads

Performance Profile

// Scenario: Process 10,000 images (CPU-bound)
// Each image takes 50ms to process

// Sequential: 10,000 × 50ms = 500 seconds
foreach (var image in images)
{
    ProcessImage(image);
}

// Parallel.ForEach on 8-core machine: ~500s / 8 = 62.5 seconds
Parallel.ForEach(images, image =>
{
    ProcessImage(image);
});

Speedup: ~8x (near-linear with core count)

Task.WhenAll: Concurrent Async Operations

Design Goals

Task.WhenAll is for waiting on multiple async operations to complete concurrently.

Example: API Calls

public async Task<IEnumerable<WeatherForecast>> GetWeatherForecastAsync()
{
    var tasks = new List<Task<IEnumerable<WeatherForecast>>>();

    // Start all operations (don't await yet!)
    var result1 = FetchWeatherAsync("London");
    var result2 = FetchWeatherAsync("Paris");
    var result3 = FetchWeatherAsync("Berlin");

    tasks.Add(result1);
    tasks.Add(result2);
    tasks.Add(result3);

    // Wait for all to complete
    var combinedResults = await Task.WhenAll(tasks);
    return combinedResults.SelectMany(cr => cr);
}

Why this works:

Each operation is I/O-bound (network calls)
Threads released during network waits
All operations run concurrently
Calling thread not blocked

Performance Profile

// Scenario: Call 3 APIs, each takes 2 seconds

// Sequential: 3 × 2s = 6 seconds
var result1 = await DownloadFile1Async(); // 2 seconds
var result2 = await DownloadFile2Async(); // 2 seconds
var result3 = await DownloadFile3Async(); // 2 seconds

// Concurrent with Task.WhenAll: 2 seconds (max of all)
var task1 = DownloadFile1Async();
var task2 = DownloadFile2Async();
var task3 = DownloadFile3Async();
var results = await Task.WhenAll(task1, task2, task3);

Speedup: ~3x (limited by slowest operation)

Common Pitfalls

❌ Pitfall 1: Task.WhenAll Without Throttling

// BAD: Creates millions of tasks - thread pool exhaustion!
var tasks = millionsOfItems.Select(item => ProcessItemAsync(item));
await Task.WhenAll(tasks);

Solution: Use Throttling

// GOOD: Process in batches with SemaphoreSlim
var semaphore = new SemaphoreSlim(10); // Max 10 concurrent
var tasks = new List<Task>();

foreach (var item in millionsOfItems)
{
    await semaphore.WaitAsync();
    tasks.Add(Task.Run(async () =>
    {
        try
        {
            await ProcessItemAsync(item);
        }
        finally
        {
            semaphore.Release();
        }
    }));
}

await Task.WhenAll(tasks);

❌ Pitfall 2: Parallel.ForEach in Web Apps

// BAD: Exhausts ThreadPool in web app
[HttpPost]
public IActionResult ProcessItems(List<Item> items)
{
    Parallel.ForEach(items, item =>
    {
        ProcessItem(item); // CPU-intensive work
    });

    return Ok();
}

Why it’s bad:

Exhausts thread pool threads
Other requests starved
Poor scalability

Solution: Use Background Service

// GOOD: Offload to background service
[HttpPost]
public async Task<IActionResult> ProcessItemsAsync(List<Item> items)
{
    await _backgroundJobQueue.EnqueueAsync(items);
    return Accepted(); // 202 - processing started
}

Parallel.ForEachAsync (.NET 6+)

What It Is

Parallel.ForEachAsync is the async-aware version of Parallel.ForEach introduced in .NET 6.

Example: Mixed CPU and I/O Work

public async Task ProcessImagesFromStorageAsync(IEnumerable<string> imageIds)
{
    var options = new ParallelOptions
    {
        MaxDegreeOfParallelism = Environment.ProcessorCount
    };

    await Parallel.ForEachAsync(imageIds, options, async (imageId, ct) =>
    {
        // I/O-bound: Download from cloud storage
        var imageData = await blobStorage.DownloadAsync(imageId, ct);

        // CPU-bound: Process image
        var processedImage = await Task.Run(() =>
            ProcessImage(imageData), ct);

        // I/O-bound: Upload result
        await blobStorage.UploadAsync(processedImage, ct);
    });
}

Benefits:

Handles both I/O and CPU work naturally
Controlled parallelism with MaxDegreeOfParallelism
Clean async/await syntax
Built-in cancellation support

Task.WhenAny: Processing First Completed

What It Is

Task.WhenAny returns when any of the provided tasks completes (not all).

Use Case 1: Timeout Pattern

public async Task<string> DownloadWithTimeoutAsync(string url, TimeSpan timeout)
{
    using var cts = new CancellationTokenSource(timeout);

    var downloadTask = httpClient.GetStringAsync(url, cts.Token);
    var timeoutTask = Task.Delay(timeout, cts.Token);

    var completedTask = await Task.WhenAny(downloadTask, timeoutTask);

    if (completedTask == timeoutTask)
    {
        throw new TimeoutException($"Download from {url} timed out");
    }

    return await downloadTask;
}

Use Case 2: Racing Multiple Services

public async Task<WeatherData> GetWeatherFromFastestSourceAsync(string city)
{
    using var cts = new CancellationTokenSource();

    var task1 = GetWeatherFromServiceAAsync(city, cts.Token);
    var task2 = GetWeatherFromServiceBAsync(city, cts.Token);
    var task3 = GetWeatherFromServiceCAsync(city, cts.Token);

    var completedTask = await Task.WhenAny(task1, task2, task3);

    // Cancel remaining tasks
    cts.Cancel();

    return await completedTask;
}

Execution Order and Task Scheduling

Understanding execution order is critical for correct concurrent programming.

Execution Order Summary

Construct	Start Order	Execution Order	Completion Order	Result Order
Task.WhenAll	Sequential (task creation)	Non-deterministic (concurrent)	Non-deterministic	Preserves creation order
Parallel.ForEach	Non-deterministic	Non-deterministic (partitioned)	Non-deterministic	N/A (no return array)
Parallel.ForEachAsync	Non-deterministic	Non-deterministic (partitioned)	Non-deterministic	N/A (no return array)

Critical: Task.WhenAll Preserves Result Order

// Even though tasks complete in random order...
var results = await Task.WhenAll(
    SlowTaskAsync(),   // Finishes 3rd (300ms)
    FastTaskAsync(),   // Finishes 1st (100ms)
    MediumTaskAsync()  // Finishes 2nd (200ms)
);

// Results array ALWAYS matches creation order:
// results[0] = SlowTask result
// results[1] = FastTask result
// results[2] = MediumTask result

Why this matters:

var users = new[] { "Alice", "Bob", "Charlie" };
var tasks = users.Select(user => GetUserDataAsync(user));
var results = await Task.WhenAll(tasks);

// results[0] is ALWAYS Alice's data
// results[1] is ALWAYS Bob's data
// results[2] is ALWAYS Charlie's data
// Even if Bob's API call finished first!

TPL, PLINQ, and the ThreadPool

What is TPL?

Task Parallel Library (TPL) is a set of APIs that simplify parallel and concurrent programming:

Namespace: System.Threading.Tasks
Key Types: Task, Task<T>, Parallel, TaskFactory
Built on: Managed ThreadPool
Purpose: Simplify parallel programming

What is PLINQ?

Parallel LINQ (PLINQ) is a parallel implementation of LINQ:

// Sequential LINQ
var results = numbers
    .Where(n => IsPrime(n))
    .Select(n => n * 2)
    .ToList();

// Parallel LINQ (PLINQ)
var results = numbers
    .AsParallel()                    // ← Enable parallelism
    .Where(n => IsPrime(n))
    .Select(n => n * 2)
    .ToList();

// With ordering preserved
var results = numbers
    .AsParallel()
    .AsOrdered()                     // ← Preserve order (performance cost!)
    .Where(n => IsPrime(n))
    .Select(n => n * 2)
    .ToList();

ThreadPool Usage Comparison

Construct	Uses ThreadPool?	How?
Parallel.ForEach	✅ Yes	Via default task scheduler
Parallel.ForEachAsync	✅ Yes	Via default task scheduler
Task.Run	✅ Yes	Queues work on ThreadPool
Task.WhenAll (I/O)	⚠️ Partially	No threads during I/O wait
Task.WhenAll (CPU)	✅ Yes	If using Task.Run
PLINQ (.AsParallel())	✅ Yes	Via TPL/ThreadPool
async/await (I/O)	❌ No	No thread during await

Decision Matrix

Use Parallel.ForEach When:

✅ CPU-bound work that benefits from parallelism ✅ Processing a large collection in parallel ✅ Each iteration is independent ✅ Operations are synchronous ✅ Want automatic chunking and load balancing

Examples:

Image/video processing
Data transformation
Complex calculations
Batch processing

Use Task.WhenAll When:

✅ I/O-bound operations (network, disk, database) ✅ Need fine-grained control over task creation ✅ Mixing CPU and I/O operations ✅ Want to avoid blocking the calling thread ✅ Server applications requiring high scalability

Examples:

Multiple API calls
Database queries
File I/O operations
Email sending

Use Parallel.ForEachAsync When:

✅ Processing a collection with async operations ✅ Need throttling (MaxDegreeOfParallelism) ✅ Mixed CPU and I/O work ✅ Want clean async/await syntax ✅ Need built-in cancellation

Examples:

Batch API calls
Processing files from cloud storage
Data transformation pipelines

Use Task.WhenAny When:

✅ Need result from first completed operation ✅ Implementing timeouts ✅ Racing multiple sources ✅ Processing results as they arrive ✅ Interruptible async operations

Examples:

API timeout patterns
Multi-source redundancy
Progressive UI updates

Real-World Pattern: Combining Both

public async Task<ProcessingResult> ProcessWithTimeoutPerItemAsync(
    IEnumerable<int> itemIds,
    TimeSpan timeoutPerItem)
{
    var results = new ConcurrentBag<ItemResult>();
    var failures = new ConcurrentBag<(int id, string error)>();

    var options = new ParallelOptions
    {
        MaxDegreeOfParallelism = 5
    };

    await Parallel.ForEachAsync(itemIds, options, async (id, ct) =>
    {
        // Use Task.WhenAny for per-item timeout
        var processTask = ProcessItemAsync(id, ct);
        var timeoutTask = Task.Delay(timeoutPerItem, ct);

        var completedTask = await Task.WhenAny(processTask, timeoutTask);

        if (completedTask == timeoutTask)
        {
            failures.Add((id, "Timeout"));
        }
        else
        {
            try
            {
                var result = await processTask;
                results.Add(result);
            }
            catch (Exception ex)
            {
                failures.Add((id, ex.Message));
            }
        }
    });

    return new ProcessingResult
    {
        SuccessCount = results.Count,
        Failures = failures.ToList()
    };
}

Pattern Benefits:

Parallel processing with Parallel.ForEachAsync
Per-item timeout with Task.WhenAny
Controlled concurrency
Graceful failure handling

Key Takeaways

Parallel.ForEach: Synchronous, CPU-bound, automatic partitioning, blocks until complete
Task.WhenAll: Asynchronous, I/O-bound, manual control, non-blocking
Parallel.ForEachAsync: Async-friendly parallel processing with throttling (.NET 6+)
Task.WhenAny: Returns when first task completes (timeouts, racing)
Choose based on workload: CPU-bound → Parallel, I/O-bound → async patterns
In web apps: Prefer async patterns over Parallel.ForEach
Always throttle: When creating many tasks with Task.WhenAll
TPL and PLINQ: Both built on ThreadPool for efficient resource usage

References

Microsoft Docs - Task Parallel Library
Microsoft Docs - PLINQ
Microsoft Docs - Parallel.ForEachAsync