ParEVO: Agentic Evolutionary Synthesis of Parallel Algorithms for Irregular Data

Software developers are increasingly hitting the limits of single-thread execution. Extracting the vast compute potential from modern multi-core devices demands writing highly optimized parallel algorithms. While Large Language Models (LLMs) have demonstrated success in producing code for simple, sequential logic, synthesizing non-trivial parallel code—particularly for intricate graph structures and irregular data—remains a persistent challenge requiring precise synchronization, correct abstractions, and optimized memory access patterns.

To solve this, we are proud to introduce ParEVO: a comprehensive framework bridging the gap between language models and high-performance parallel coding execution. ParEVO equips base LLMs with the primitives of safe parallel software libraries and provides a robust evolutionary agent capable of iterating on those primitives toward deterministic hardware optimality.

The ParEVO Pipeline

Standard coding benchmarks like HumanEval evaluate models on sequential loops but critically fail to measure run-time scaling, such as calculating the theoretical limit of Work-Span on complex data. We overcome this limitation using a multi-stage approach.

We began by building The Parlay-Instruct Corpus, a uniquely curated dataset containing 13,820 tasks generated via an automated "Critic-Refine" pipeline. The dataset strictly filters out poor solutions, insisting on empirically performant code that explicitly handles underlying Work-Span primitives effectively.

Using this high-quality corpus, we present highly Specialized Fine-Tuned Models that internalize data-parallel abstractions (e.g. Map, Reduce, Scan) natively. We release bespoke, heavily fine-tuned variants of DeepSeek (6.7B), Gemini-2.5-Pro, and dual Qwen3 (30B) models spanning C++ (ParlayLib) and safe Rust (Rayon) ecosystems.

The Evolutionary Coding Agent

Even with fine-tuning, the "last mile" of parallel coding typically results in frustrating data races and compiler errors when attempting complex algorithms out of zero-shot generation. ParEVO introduces a groundbreaking Evolutionary Coding Agent (ECA) to conquer this.

Taking inspiration from genetic algorithms and MAP-Elites exploration, our ECA iteratively mutates initial model guesses. It completely detaches from standard unit-test constraints, instead evaluating code fitness explicitly using performance tests, catching insidious data-races, verifying functional correctness, and explicitly optimizing for maximal execution speedups.

Robust Performance Results

In evaluations spanning the comprehensive ParEval suite, ParEVO consistently crushes baseline benchmarks:

• Average Speedup: An outstanding 106× average performance increase relative to un-optimized code.
• Maximal Scaling: Up to a peak of 1103× acceleration depending on available hardware topology.
• Complex Graphs: On notoriously difficult irregular graph configurations, ParEVO still drove a highly resilient 13.6× speedup.

Crucially, the evolutionary process consistently matches or explicitly beats the theoretical asymptotic efficiency of manually hand-tuned solutions delivered by human domain experts.