ParEVO: Synthesizing Code for Irregular Data: High-Performance Parallelism through Agentic Evolution

Liu Yang Affiliation: Department of Computer Science, Yale University, New Haven, CT, USA Zeyu Nie Affiliation: Department of Computer Science, Yale University, New Haven, CT, USA Andrew Liu Affiliation: Department of Computer Science, Yale University, New Haven, CT, USA Ruomu Zou Affiliation: Department of Computer Science, Yale University, New Haven, CT, USA Deniz Altinbüken Affiliation: Google DeepMind, Mountain View, CA, USA Amir Yazdanbakhsh Affiliation: Google DeepMind, Mountain View, CA, USA Quanquan C. Liu Affiliation: Department of Computer Science, Yale University, New Haven, CT, USA

Abstract

Parallelizing code for irregular data structures (sparse graphs, unbalanced trees, non-uniform meshes) is notoriously hard, and current LLMs fail catastrophically on such tasks, generating code riddled with race conditions, deadlocks, and poor scaling. We address this with ParEVO, a framework for synthesizing high-performance parallel algorithms for irregular data, built on three contributions: the Parlay-Instruct Corpus of 13,820 tasks generated via a Critic-Refine” pipeline that filters for empirically performant uses of Work-Span primitives; specialized DeepSeek, Qwen, and Gemini models fine-tuned to the semantics of the ParlayLib library; and an Evolutionary Coding Agent (ECA) that repairs the last mile” of correctness using compiler and profiler feedback. On the ParEval benchmark, ParEVO achieves an average $107\times$ speedup and a $13.6\times$ speedup on highly complex irregular graph problems, outperforming commercial models like GPT-5-Thinking and Gemini-3-Pro, while matching expert human-written baselines and reaching up to a $4.1\times$ speedup on kernels such as Maximal Independent Set. This demonstrates that AI-driven agents can effectively navigate the complex landscape of high-performance computing. Source code and datasets are available at https://github.com/WildAlg/ParEVO (41).

Keywords:

LLM for Code, High Performance Computing, Parallelism, Evolutionary Algorithms

1 Introduction

The breakdown of Dennard scaling and the subsequent stagnation of single-core frequency scaling have fundamentally shifted the computing paradigm. Performance improvements in modern software are now almost exclusively driven by parallelism, whether through multi-core CPUs, GPUs, or distributed clusters (48). While “regular” parallelism (e.g., dense matrix multiplication) is well-understood and supported by mature libraries, irregular parallelism remains a grand challenge in High-Performance Computing (HPC).

Irregular algorithms, which operate on graph structures, sparse matrices, or adaptive meshes, are characterized by unpredictable memory access patterns and dynamic work distribution. In these regimes, the computational cost of processing a node or element depends on runtime data, making static load balancing ineffective. Writing efficient code for these problems requires sophisticated techniques like work-stealing, dynamic scheduling, and lock-free synchronization (38).

Current Large Language Models (LLMs) struggle profoundly with this domain. Trained primarily on sequential Python or standard C++ code from GitHub, they exhibit strong “sequential bias.” When asked to parallelize a graph traversal, they often attempt to wrap a standard Breadth-First Search (BFS) in a naive parallel loop (#pragma omp parallel for), ignoring the race conditions inherent in updating the ‘visited’ array. Alternatively, they may introduce coarse-grained locks that serialize execution, rendering the parallel code slower than its sequential counterpart (29).

We argue that the solution lies not in teaching LLMs to write low-level threading primitives (like ‘pthreads’ or ‘std::thread’), which are error-prone and hard to compose, but in leveraging high-level algorithmic primitives. ParlayLib (9) provides a suite of such primitives (e.g., ‘filter’, ‘pack’, ‘scan’, ‘sort’, ‘reduce’) that abstract away the complexities of scheduler management. By training LLMs to map natural language intent to these primitives, we can generate code that is correct by construction and mathematically provable to scale.

To this end, we introduce ParEVO, an end-to-end system for synthesizing high-performance parallel code. We detail the following main contributions:

•

Data-Centric Synthesis: We introduce the Parlay-Instruct corpus, a dataset of 13,820 parallel coding tasks. Unlike previous datasets scraped from GitHub (which often contain broken code), our data is synthesized via a “Teacher-Student” pipeline and verified against a ground-truth compiler oracle. We provide a novel performance dataset generation technique focused on graph problems curated from the selection of well-known programming competitions curated by the online-judge DMOJ (21).
•

DeepSeek-Parlay, Qwen-Parlay, Qwen-Rust, and Gemini-2.5-Parlay¹¹ 1 https://huggingface.co/qqggez/deepseek-parlay-6.7b, https://huggingface.co/qqggez/qwen3-30b-sft-stage2-merged, https://huggingface.co/YangLiuWillow/qwen3_rust_dpo_final_merged: We release a fine-tuned 6.7B parameter Deepseek model for C++ (40), two fine-tuned 30B parameter Qwen3 models—one for C++ (42) and one for Rust (43)—and a Gemini-2.5-Pro model fine-tuned for C++. These models outperform some larger closed-source and open-source models on parallel reasoning tasks by internalizing the data structures, semantics, primitives, and algorithms of the state-of-the-art ParlayLib library (9) and safe parallel Rust patterns.
•

Evolutionary Refinement: We formalize both the data synthesis step and the final code generation process as an evolutionary search over the space of Abstract Syntax Trees (ASTs). Our agent generates a population of candidate solutions, compiles them, runs them against performance tests, and uses the error logs (or performance profiles) as “fitness functions” to drive mutation and crossover operations in the prompt space.
•

The Correctness-Speedup Trade-off: We identify an “alignment tax” for concurrent programming. Our evaluation reveals that fine-tuning enables models to write significantly safer code at the expense of slightly slower peak performance (i.e. higher Pass@1 rates with lower Speedup@1 rates). This trade-off occurs because fine-tuned models learn to conservatively avoid raw, risky atomics in favor of stable, high-level primitives (such as parlay::unique).

Specifically, our paper succeeds in the following task:

ParEVO democratizes parallel computing for irregular data by fine-tuning LLMs on verified primitives and deploying an evolutionary agent to iteratively optimize code based on runtime performance feedback.

All code can be found at https://github.com/WildAlg/ParEVO (41).

Conflict of Interest Disclosure.

D.A. and A.Y. are employed by Google DeepMind, which develops the Gemini family of models. This paper evaluates and uses Gemini models, including Gemini-2.5-Flash, Gemini-2.5-Pro, Gemini-3-Pro, and a fine-tuned Gemini-2.5-Pro variant referred to as Gemini-2.5-Parlay. This work was supported in part by a Google Academic Research Award.

2 Related Work

LLMs for Code Generation. Large Language Models have fundamentally shifted the landscape of software engineering, achieving remarkable success in sequential code completion (28), summarization (3), and translation (23). Evaluation metrics have similarly evolved from surface $n$ -gram overlap to structure-aware measures like CodeBLEU (47), which better correlate with functional correctness. However, current models struggle with complex planning and reasoning tasks (29), a limitation that is magnified in High-Performance Computing (HPC). 38 demonstrated via the ParEval benchmark (24) that while LLMs can generate syntactic structures for frameworks like Kokkos and MPI, they often fail to capture the semantic nuances of synchronization and race conditions. Recently, ParEval-Repo (19; 20) extended this evaluation to repository-level HPC translation tasks (e.g., multi-file codebases, build systems), highlighting that scaling beyond individual kernels introduces qualitatively different failure modes. Our work addresses this by moving beyond general-purpose pre-training, targeting the qualitatively harder regime of parallel and irregular algorithms where correctness requires respecting concurrency and performance depends on minimizing span.

Automated Parallelization and HPC Translation. Prior efforts in automated parallelization have largely focused on translating serial loops to OpenMP directives. BabelTower (60) previously tackled auto-parallelized program translation from sequential C to CUDA via a learning-based framework leveraging large-scale corpora and back-translation with reranking. OMPGPT (14) fine-tunes GPT-Neo to predict pragmas for regular loops, while AutoParLLM (35) uses Graph Neural Networks to guide LLM generation based on parallelism patterns. 58 attempts unsupervised translation between languages and their HPC extensions but lacks a feedback mechanism for correctness. To address correctness risks such as subtle parallel bugs that often arise in serial-to-CUDA/OpenMP translation, MuSL (31; 32; 30) proposed a mutual-supervision loop where a translator and a test-generator co-evolve: the tester synthesizes unit tests to filter translations, and the translator produces code to improve the tester. More recently, UniPar (8) introduced a multi-agent framework for translating code between serial, OpenMP, and CUDA formats. While UniPar evaluates functional correctness (achieving 33%), it does not explicitly optimize for or benchmark the runtime scalability (work-span) of the generated algorithms. In contrast, ParEVO specifically targets irregular data, such as graph traversals and sparse matrix operations, where correct translation is insufficient, and performance speedup via parallelism, software engineering techniques, and performant algorithms and data structures is key. Recent advances have further specialized LLMs for parallel domains. For instance, 13 successfully fine-tuned base models on the HPC-Instruct dataset to target low-resource parallel languages, demonstrating that smaller, specialized models can match proprietary models on the ParEval benchmark. Similarly, frameworks like MARCO (46) and PerfCoder (62) utilize multi-agent reasoning and execution trajectories to separate code generation from performance tuning. However, while these frameworks primarily target traditional imperative paradigms like OpenMP and CUDA, ParEVO specifically targets the algorithmic complexities of irregular data by grounding the model in the composable semantics of ParlayLib.

Structured Reasoning and Agentic Coding. To transcend the stochastic limitations of single-shot generation, frameworks like Reflexion (51) use verbal reinforcement to iteratively correct failures. More recently, this paradigm has been extended via evolutionary search. Building upon this, EvoTune (56) augments LLM-based evolutionary program search by periodically updating the model via reinforcement learning on search-derived signals. Similarly, AI tree search systems (7) embed LLM-based code mutation within a search procedure to maximize a measurable quality metric. To benchmark these search processes, AlgoTune (45; 44) introduced a suite for numerical programs and evaluated an agent that iterates by editing, compiling, timing, and selecting the fastest valid variant. Concurrent open-source works such as OpenEvolve (50) have demonstrated the efficacy of coupling LLMs with genetic algorithms (6; 33; 39) and Quality-Diversity metrics (e.g., MAP-Elites) to prevent diversity collapse during program synthesis. ParEVO brings this evolutionary paradigm to the HPC domain, replacing standard unit-test fitness functions with rigorous hardware profiling.

Abstractions for Irregular Parallelism. A core theme in parallel algorithmics is that abstraction choice determines accessibility. The classic work-span model (11) and work-stealing schedulers (10) provide a principled foundation for nested parallelism. High-level libraries like ParlayLib (9; 4) expose this theory through composable primitives (e.g., scan, reduce, filter), making provably efficient algorithms more accessible. Similarly, specialized abstractions such as GraphIt (63; 25) separate algorithm specification from scheduling choices to enable systematic performance tuning for irregular graph workloads, while Ligra (53) provides a lightweight shared-memory graph processing framework with simple vertex/edge mapping primitives and density-adaptive traversal strategies. Benchmarks like PBBS (52; 5) and Rusty-PBBS (1) formalize the evaluation of these irregular workloads. ParEVO leverages these insights by training models to target primitive-based code-writing within the Parlay ecosystem, ensuring that generated code is not just a parallel loop, but a structurally sound parallel algorithm capable of handling load imbalance inherent in irregular data (48; 12).

Test-Time Compute and Execution Feedback. A growing consensus indicates that standard Supervised Fine-Tuning (SFT) and text-based reflection are insufficient for generating highly optimized code. Consequently, the field has rapidly shifted toward integrating real-machine execution feedback into the LLM reasoning loop. Using empirical hardware profiling as a direct reward signal drastically improves kernel efficiency (22; 36; 34). Crucially, 55 applied test-time program search to the ParEval benchmark and empirically proved that LLMs exhibit a severe capability gap when attempting to act as their own “verifiers” for parallel code. This limitation directly motivates ParEVO’s Evolutionary Coding Agent (ECA), which sidesteps the unreliable “LLM-as-a-judge” paradigm in favor of treating deterministic compilers and sanitizers as ground-truth adversarial critics.

ML for Compiler/Performance Optimization.

ML increasingly replaces heuristics for phase ordering, register allocation, auto-vectorization (18; 59; 26), and tensor scheduling (15; 64). Foundation models (17; 57) optimize IR using MCTS. For parallelization, OMPGPT, AutoParLLM, and UniPar predict pragmas for regular loops. However, they operate at lower stack levels, lack robust feedback, or target regular matrices. ParEVO uniquely synthesizes high-level parallel abstractions for irregular data where vectorization fails.

Program Synthesis/Repair with Feedback.

Agentic workflows increasingly use execution environments as oracles (61; 16; 37; 65). While text-based critique and RL use compiler diagnostics, ParEVO advances this by tying selection pressure to scalable runtimes and problem-specific tests written by human experts that are designed to catch data races that lead to incorrect output and code that degrade performance.

3 Methodology: The ParEVO System

ParEVO is composed of three distinct stages: (1) Data Synthesis through Evolutionary Search, (2) Supervised Fine-Tuning, and (3) Inference-Time Evolutionary Search.

3.1 Stage 1: The Parlay-Instruct Fine-Tuning Dataset Corpus

The primary bottleneck for training “HPC-aware” LLMs is data scarcity. High-quality parallel C++ code is rare on GitHub compared to React components or Python scripts. We generated a synthetic dataset which incorporates parallel performance constructs, syntax, software engineering techniques, data structures, and algorithms, using a “Teacher-Student-Critic” pipeline via OpenEvolve (49). This synthetic dataset contains three parts: (1) the ParlayLib primitives, (2) DMOJ slow-fast code comparison pairs, and (3) DMOJ problem-solution pairs with labeled status, runtime performance, and any compiler or runtime error messages.

3.1.1 Seed Generation and Mutation

We manually authored 593 “golden” examples covering ParlayLib’s core primitives and 20 problems from DMOJ (21). We then used Gemini-3-Pro (the “Teacher”) to mutate these seeds. We defined three mutation operators $\mathcal{M}$ :

1.

Type Mutation ( $\mathcal{M}_{type}$ ): Changes the underlying data type (e.g., ‘int’ $\rightarrow$ ‘std::string’ or custom ‘struct Point’). This forces the model to learn C++ template instantiation rules.
2.

Constraint Mutation ( $\mathcal{M}_{cons}$ ): Adds logical predicates (e.g., “Sort only odd numbers” $\rightarrow$ requiring a ‘filter’ then ‘sort’). This forces the composition of primitives.
3.

Algorithmic Mutation ( $\mathcal{M}_{algo}$ ): Transforms the problem structure, e.g., converting a ‘reduce’ problem into a ‘scan’ (prefix sum) problem.

We ran 5 passes of 10 mutations per seed, sampled from the operators $\{\mathcal{M}_{type},\mathcal{M}_{cons},\mathcal{M}_{algo}\}$ defined above, plus an additional 50 tasks targeting complex primitives (e.g., parlay::delayed::filter). This produced an initial pool of 29,700 candidates.

3.1.2 The Critic Loop: Rejection Sampling

Let $P$ be a generated problem and $C$ be the generated code. We accept $(P,C)$ into the dataset if and only if:

\text{Compile}(C)\land\text{UnitTest}(C)

(1)

That is, we only accept code that compiles and passes the unit tests. Compiling and executing each candidate against its unit test discarded 15,880 candidates that failed to compile or timed out. This filtration process yielded 13,820 verified instruction-tuning pairs, which we partitioned into a fine-tuning training set of size 13,120 and a held-out test set of 700 pairs for evaluation.

Performance Optimization Dataset.

To enable the model to reason about runtime efficiency, we curated a benchmark of 20 challenging graph problems from the DMOJ competitive programming platform (21). We synthesized optimization trajectories for these problems using the OpenEvolve framework (49) powered by Gemini-3-Pro. The data generation process followed the following novel protocol:

1.

Agent Initialization: The agent was provided with the problem description and ParlayLib documentation, with a dual objective function minimizing both test failures and execution time.
2.

Trajectory Extraction: We recorded the agent’s iterative refinements, extracting pairs of solutions $(C_{\text{base}},C_{\text{opt}})$ from the evolutionary history.
3.

Speedup Threshold: To ensure high-quality training signal, we filtered for pairs where the optimized solution $C_{\text{opt}}$ achieved a runtime speedup of at least $1.2\times$ over $C_{\text{base}}$ .

We constructed pairwise comparison examples using the solution pairs identified in the previous step. To eliminate positional bias, we randomized the assignment of “Code A” and “Code B” so that the faster implementation appears in either position with equal probability. The model is trained to identify the more performant solution using the following format:

Instruction: Determine which of the two code solutions has better performance.

Input:
Code A: [Source Code]
Code B: [Source Code]

Output: [Label of the Faster Solution]

A concrete example of this comparison format is provided in Figure 3.

While learning on performance edits has been used in (54), our dataset is distinct in its focus on the complex, global transformations required for irregular parallelism, rather than the local sequential optimizations primarily targeted in the prior work.

Rust Parlay Primitives

Given that the distinct Rust primitives were insufficient to constitute a robust fine-tuning dataset, we opted to include them directly in the context window. This approach allowed us to leverage the models’ pattern-matching and in-context learning capabilities without the need for parameter updates. To support this process, we integrated a full suite of Parlay-equivalent Rust primitives derived from RPB (2). Furthermore, to support the generation of higher-complexity algorithms, we manually implemented the delayed execution primitives in Rust and supplied them as immutable reference implementations within the system prompt.

Rust Evolutionary Dataset.

To train the evolutionary coding agent for the Rust domain, we constructed a specialized dataset derived from the DMOJ benchmark execution logs. We aggregated the raw logs to extract code solutions, runtime metrics, and error traces. The data underwent a rigorous cleaning pipeline: we first filtered out irrelevant infrastructure failures (e.g., permission errors) and removed the held-out test set. We then deduplicated the remaining entries, prioritizing successful submissions while retaining a diverse set of failing attempts characterized by distinct error messages. The final corpus was serialized into JSONL format, where each entry explicitly pairs a problem description with the corresponding code, execution status, runtime performance, and any resulting compiler or runtime error messages. This rich metadata distinguishes our dataset from standard code corpora, enabling the model to learn both correct optimization patterns and specific error-correction strategies. Such a detailed corpus of training data is necessary for Rust given that Rust is notoriously difficult to use for irregular parallelism; hence, the available training data (including errors and compile-time messages) is rare for this language in the available base models.

3.2 Stage 2: Fine-Tuning DeepSeek, Gemini-2.5, Qwen3 for ParlayLib and Rust RPB

We selected DeepSeek-6.7b-base and Qwen3-Coder-30B-A3B-Instruct as our open-source backbones due to their strong performance on standard C++. These models represent a tiered architecture strategy: DeepSeek-6.7b serves as our efficient, lightweight baseline, while Qwen3 acts as our high-capacity large model. We fine-tuned the model using Low-Rank Adaptation (LoRA) (27) to minimize compute costs while preserving the base model’s reasoning capabilities. We selected Gemini-2.5-Pro as our third base model due to its extensive context window for handling complex, long-context scenarios. All three models underwent fine-tuning to align them with our specific domain requirements.

Training Configuration.

We configured the training pipeline according to model scale. For DeepSeek-6.7b-base, we executed single-stage Supervised Fine-Tuning (SFT) on an NVIDIA RTX 5000 Ada machine. We targeted the query and value projections using LoRA ( $r=8,\alpha=16$ ) and trained on a combined dataset of ParlayLib syntax and ‘slow-fast’ performance pairs (FP16, learning rate $2\text{e-}4$ ).

For the larger Qwen3-Coder-30B-A3B, we implemented a dual-stage alignment pipeline on an NVIDIA H200 GPU. The first stage established domain capability via SFT on ParlayLib syntax and standard DMOJ solutions, using QLoRA ( $r=16,\alpha=32$ ) across all linear attention and MLP layers. The second stage applied Direct Preference Optimization (DPO) to explicitly suppress failure modes. In this phase, we trained on contrastive triplets (pairing passing solutions against failing or inefficient implementations) using a reduced learning rate of $5\text{e-}6$ and $\beta=0.1$ .

Evaluation Environment.

Performance benchmarks were conducted on a dual-socket compute node featuring two Intel Xeon Platinum 8562Y+ processors (64 physical cores total). To ensure consistent comparisons across frameworks, all experiments use 32 threads for OpenMP, ParlayLib, and Rust unless otherwise specified.

3.3 Stage 3: Evolutionary Coding Agent (ECA)

Evolutionary Search Strategy.

To transcend the stochastic limitations of single-shot generation, we deploy an evolutionary agent that iteratively refines code for both correctness and performance. We model this process as a directed population-based search in the discrete space of possible programs. See Figure 1 for a diagram of the workflow.

Figure 1: Overview of the ParEVO Framework. The system integrates human expert context (problem formulation, parallel tooling) with an evolutionary LLM agent. The cycle iteratively refines candidate parallel algorithms through a rigorous evaluation framework (correctness verification, stress testing, and performance profiling), using metrics to guide the selection of the next population via MAP-Elites.

The agent maintains a diverse population of candidate solutions, each associated with specific performance metrics (test coverage, execution time) and diagnostic artifacts (compiler logs, failure reasons, and targeted refinement instructions). The search initializes with either a baseline functional solution or a raw problem description. We define the fitness function $f(x)$ for a candidate solution $x$ as:

f(x)=\begin{cases}0&\text{if }x\text{ fails compilation or tests}\\ \frac{1}{T(x)+\varepsilon}&\text{if }x\text{ passes, where }T(x)\text{ is runtime}\end{cases}

(2)

Furthermore, candidate solutions that trigger execution timeouts or yield inconsistent outputs across the five stress-test rounds are strictly assigned a fitness of 0.

A critical design choice in our evolutionary loop is reliance on deterministic external tools—specifically compilers and stress tests—rather than LLM-based static analysis. Because LLMs process code as text tokens, they natively fail to capture inter-thread timing and synchronization structures, making them prone to hallucinating data races. Furthermore, 55 finds LLMs to be fundamentally unreliable verifiers of low-level parallel code. By executing each candidate five times on exceedingly large inputs (on the order of 10 million vertices), we force latent concurrency bugs to manifest as observable failures, providing a rigorous empirical filter that penalizes unsafe memory accesses.

In each generation, the agent selects survivors to populate the context window for the next iteration. To balance exploitation and exploration, we select the top $k=3$ solutions by fitness (performance) and $d=5$ diverse solutions via MAP-Elites. The MAP-Elites archive is organized as a grid indexed by two behavioral dimensions:

•

Complexity: The character length of the source code, discretized into bins. Shorter programs fall into lower bins, helping retain solutions of varying structural complexity.
•

Diversity: The mean character-level edit (Levenshtein) distance between a candidate and all other programs currently in the population. Programs structurally dissimilar to the rest of the population receive high diversity scores and occupy higher bins.

A new candidate replaces an incumbent in a grid cell only if its fitness is strictly higher, enforcing a strict quality-diversity invariant. These selected candidates, along with their diagnostic artifacts, prompt the LLM to synthesize the next generation of improved code. The process terminates by returning the candidate with the maximum fitness score.

3.4 Supported Languages

To demonstrate the versatility of our approach, in this paper, we use our ParEVO on two languages: C++ and Rust. For C++, we use our ParEVO system to fine-tune models on ParlayLib (9). For Rust, we use our ParEVO system to fine-tune models on RPB: Rust Parallel Benchmarks Suite (1; 2). For both C++ and Rust, our methods lead to improved performance.

3.5 Benchmarking Suite

We evaluate our framework across four distinct benchmarks to assess both generation quality and runtime performance. First, we compare our fine-tuned models against state-of-the-art local and commercial LLMs using the ParEval (38) library. Second, we measure absolute performance against expert human baselines, utilizing C++ solutions from PBBSBench (52) and Rust implementations from RPB (2). Finally, to test generalization, we evaluate on a held-out set of DMOJ competitive programming problems. In this setting, we compare the runtime of code generated by ParEVO against official contest solutions, demonstrating significant speedups.

4 Experimental Results

4.1 Experimental Setup

Hardware. All experiments were conducted on a dual-socket compute node equipped with two Intel Xeon Platinum 8562Y+ processors (64 physical cores total) and 512GB DDR5 ECC RAM. An NVIDIA H200 GPU was utilized solely for inference.

Benchmarks. We evaluated on:

1.

ParEval: The testing suite of (38).
2.

PBBSBench & RPB: Expert-written C++ and Rust baselines (52; 2).
3.

DMOJ: A held-out set of competitive programming problems. (21)

4.2 Main Results: ParEval Performance

Methodological Note on Expected Speedup. In traditional systems literature, the geometric mean is typically used to average normalized execution times of a static benchmark suite across different hardware. However, in the context of zero-shot code generation over a large distribution of tasks (ParEval), we conceptualize performance formally as an expected capability reward. Specifically, we report the arithmetic mean of Speedup@1 to represent the expected speedup ( $\mathbb{E}[S]$ ) a user would experience when querying the model with a random task from the problem domain. This aligns directly with standard machine learning evaluation practices for reporting expected test-time rewards over a distribution, as opposed to summarizing the total execution time of a fixed static workload.

Table 1 presents the performance of local and commercial models. Our fine-tuned models (Gemini-2.5-Parlay and DeepSeek-Parlay) significantly outperform their base counterparts. Notably, Gemini-2.5-Parlay achieves an average $107\times$ speedup over the baseline provided by ParEval, driven by its ability to generate valid, compilable parallel code (Build@1 0.81 vs 0.25 of the state-of-the-art Gemini 3.0 Pro). Even our smallest fine-tuned model, DeepSeek-Parlay (with 6.7b parameters) is able to beat the commercial state-of-the-art Gemini-3-Pro. The Speedup@1 metric is the arithmetic mean of the expected best performance speedups across all 59 problems relative to a sequential baseline. The exceptionally high 107.43x mean for Gemini-2.5-Parlay is driven by heavy-tailed performance on specific irregular tasks (e.g., 34_scan_largest_contiguous_subarray_sum), where the model discovered an optimal prefix-sum algorithm while the baseline sequential implementation is heavily bottlenecked by nested loops.

Execution Model	Code	Sched.	Build@1	Pass@1	Speedup@1 (AM)	Speedup@1 (GM)
Claude Opus 4.5	Parlay	Parlay	0.97	0.05	0.42	1.13
GPT-5 Thinking	Parlay	Parlay	0.93	0.05	0.43	1.13
Gemini-2.5-Flash	Parlay	Parlay	0.57	0.29	6.88	2.75
Gemini-2.5-Pro	Parlay	Parlay	0.84	0.54	85.34	5.47
Gemini-3-Pro	Parlay	Parlay	0.25	0.23	3.68	1.53
Gemini-2.5-Parlay	Parlay	Parlay	0.81	0.58	107.43	6.33
DeepSeek-6.7B-Base	Parlay	Parlay	0.89	0.11	0.73	1.14
DeepSeek-Syntax	Parlay	Parlay	0.85	0.12	2.42	1.38
DeepSeek-Parlay	Parlay	Parlay	0.81	0.26	126.16	3.14
Qwen3-Parlay	Parlay	Parlay	0.50	0.33	5.01	1.87
DeepSeek-Coder-V2-Lite-Base	Parlay	Parlay	0.80	0.09	1.30	1.19
Qwen2.5-Coder-32B	Parlay	Parlay	0.93	0.11	5.02	1.82
Qwen2.5-Coder-32B-Instruct	Parlay	Parlay	0.61	0.41	7.00	2.22
DeepSeek-Coder-V2-Lite-Base	Rust	Rayon	0.73	0.29	3.47	0.58
DeepSeek-Coder-V2-Lite-Instruct	Rust	Rayon	0.40	0.02	0.15	1.04
Qwen2.5-Coder-32B	Rust	Rayon	0.82	0.45	4.43	0.63
Qwen2.5-Coder-32B-Instruct	Rust	Rayon	0.63	0.49	4.61	0.57
Qwen3-Coder-30B-Instruct	Rust	Rayon	0.61	0.50	4.57	0.46
Qwen3-Rust	Rust	Rayon	0.64	0.46	4.62	0.35
StarCoder2-15B	Rust	Rayon	0.77	0.27	2.46	0.79
Gemini-3-Pro	Rust	Rayon	0.97	0.82	7.32	0.69

Table 1: ParEval results (temperature

=0.2

). Code denotes the parallel programming language used, and Scheduler the parallel runtime. Parlay code uses the ParlayLib library, with either ParlayLib’s internal scheduler or OpenMP as the scheduling backend. Rust code uses Rayon’s dynamic work-stealing scheduler. Our fine-tuned models achieve orders-of-magnitude improvements in speedup.

Impact of Fine-tuning on Code Quality.

As illustrated in Figure 5, the most consistent effect of fine-tuning is on runtime performance: ParEVO produces faster code in eleven of twelve categories, frequently by an order of magnitude, indicating that the model learns to select more efficient parallel patterns by using ParlayLib primitives rather than merely valid ones. The effect on Build@1 and Pass@1 is category-dependent. Fine-tuning yields its largest correctness gains precisely where the base model was weakest at expressing the appropriate ParlayLib idiom—most notably graph ( $\texttt{Build@1}:0.62\rightarrow 0.97$ , $\texttt{Pass@1}:0.42\rightarrow 0.76)$ and histogram ( $\texttt{Pass@1}:0.19\rightarrow 0.63$ ), while incurring mild regressions on categories the base already handled well (e.g., fft, geometry, search).

4.3 Semantic Alignment via Fine-Tuning

A critical advantage of ParEVO is its ability to learn the correct semantics of parallel primitives. In the complex number sorting task (Figure 21 in Appendix), the base model failed completely (Build@1 = 0), struggling with C++ custom comparators. The fine-tuned model not only compiled (Build@1 = 1) but achieved a speedup of $17.5\times$ . This suggests that the model has learned to navigate the complex type system of ParlayLib.

4.4 Performance Analysis: Strong Scaling

Code correctness is insufficient for HPC; the solution must also scale. Figure 2 demonstrates strong scaling up to 64 cores. For regular parallelism like Discrete Fourier Transform, our model generates code that scales near-linearly ( $40\times$ speedup), abstracting away complex synchronization that typically hinders manual implementations. The performance drop observed at 64 cores for Figure 2(d) is due to parallel overhead and thread contention, where the generated code reveals that the LLM attempted to maintain local dynamic queues inside a nested parallel BFS loop.

(a) Maximal Matching (Rust)

(b) Min. Spanning Forest (Rust)

(d) Largest Component (C++)

Figure 2: Strong Scaling results. (c) Algorithms like Discrete Fourier Transform show excellent scaling with ParEVO’s generated code, reaching nearly

40\times

speedup on 64 cores.

4.5 Comparison vs. Expert Baselines

We benchmarked our generated solutions against expert human implementations from PBBSBench (C++) and RPB (Rust). As shown in Table 2, ParEVO matches or exceeds expert performance. For Maximal Independent Set, the generated Rust solution achieved a $4.1\times$ speedup over the baseline by identifying a superior parallel strategy. We demonstrate the maximum speedups we can gain by using Gemini-3-Pro with our ParEVO evolutionary strategy described in Section 3.3.

Table 2: Runtime Comparison: PBBS & RPB at Thread=32, best speedup across test inputs. Baseline code is state-of-the-art human-written code. We also demonstrate the speedup against one thread, labeled Speedup (1T).

Problem	Model/Method	Language	Runtime (s)	Speedup (1T)	Speedup (Base)
Maximal Independent Set	Baseline	Rust	0.31876	$1.116\times$	–
Maximal Independent Set	PAREVO (GEMINI)	Rust	0.07728	$0.938\times$	$4.125\times$
Maximal Matching	Baseline	Rust	0.20646	$21.723\times$	–
Maximal Matching	PAREVO (GEMINI)	Rust	0.1928	$21.43286835\times$	$1.0708\times$
Minimum Spanning Forest	Baseline	Rust	0.41968	$13.427\times$	–
Minimum Spanning Forest	PAREVO (GEMINI)	Rust	0.38004	$13.87\times$	$1.1043\times$
Spanning Forest	Baseline	Rust	0.11571	$10.378\times$	–
Spanning Forest	PAREVO (GEMINI)	Rust	0.08865	$15.482\times$	$1.3052\times$
Minimum Spanning Forest	Baseline	C++	1.24	$22.633\times$	–
Minimum Spanning Forest	PAREVO (GEMINI)	C++	1.169	$23.689\times$	$1.061\times$
Histogram	Baseline	C++	$0.051$	$27.59\times$	–
Histogram	PAREVO (GEMINI)	C++	$0.019$	$>13.94\times$	$2.68421\times$

4.6 Ablation Study: Evolutionary Agent

To isolate the contribution of the Evolutionary Coding Agent (ECA), we evaluated performance with the agent disabled. Table 3 confirms that iterative refinement is crucial: 30 iterations of ECA yield a $2.2\times$ performance multiplier over single-shot generation.

For this ablation study, we utilized a reserved set of training problems sourced from DMOJ.²² 2 The DMOJ training problems are available at https://github.com/WildAlg/ParEVO/tree/main/code-contests-dataset In Table 3, the base performance ( $1.00\times$ ) corresponds to baseline.cpp, which we define as the very first solution that passes all tests (this is not necessarily the solution from iteration 1 if the early attempts fail). The reported speedup for the ECA configurations is calculated as the average of the relative speedups achieved across all datasets within this training corpus.

Table 3: Impact of Evolutionary Refinement on Speedup. 30 iterations of ECA yield a

2.2\times

speedup over the first valid solution.

Configuration	Speedup
Gemini-3-Pro (No ECA)	$1.00\times$ (baseline)
Gemini-3-Pro + ECA (10 iter)	$1.498\times$
Gemini-3-Pro + ECA (30 iter)	$\mathbf{2.218\times}$

While the actual prompt in each iteration contains more context (other iterations/metrics) by the default template of openevolve, the structural system prompt we specified for the ECA is provided in Appendix A.

4.7 Analysis: The Correctness-Speedup Trade-off

A deeper analysis of Graph problems (Table 4) reveals a trade-off. Fine-tuning increases correctness (Pass@1 $0.42\to 0.76$ ) by enforcing safe API usage, but sometimes degrades speedup ( $21\times\rightarrow 13\times$ ) as it favors stable, high-level primitives over risky, fine-grained atomic operations.

Table 4: Performance on Graph Problems. Fine-tuning improves reliability (Pass@1) but favors safer, slightly slower algorithms.

Model	Build@1	Pass@1	Speedup@1
Gemini-2.5-Pro	0.62	0.42	21.76
Gemini-2.5-Parlay	0.97	0.76	13.67

5 Discussion and Limitations

5.1 The Role of Abstraction in Parallelization

Our findings suggest that the efficacy of LLM parallel code generation depends heavily on the abstraction level provided by the target intermediate representation (IR). We argue that the superior performance of ParEVO on ParlayLib stems from an alignment of abstraction. Imperative models like OpenMP force the LLM to manage global state and explicit synchronization: tasks that maximize the “state-tracking” burden on the attention mechanism and increase the probability of race conditions.

In contrast, ParlayLib functions as a high-level parallel DSL. Its functional primitives (e.g., map, reduce, scan) encapsulate complex scheduling logic and enforce immutability. This reduces parallelization to local transformations (mapping serial loops to equivalent functional constructs) which aligns naturally with the token-local prediction of Transformer models.

By training our models to target ParlayLib’s composable primitives, ParEVO aligns the optimization objective with the token-local reasoning capabilities of the Transformer architecture, yielding code that is both mathematically sound and highly performant.

5.2 Limitations and Future Directions

•

Architectural Scope: ParEVO is currently optimized for shared-memory multicore architectures. It does not address the distributed memory paradigm (e.g. MPI/PGAS), where communication latency and data partitioning introduce distinct optimization constraints.
•

Inference Latency vs. Runtime Efficiency: The Evolutionary Coding Agent trades inference-time compute for execution-time speedup. While generating multiple candidates and compiling them is costly, we argue this is an acceptable amortized cost for HPC kernels that may run trillions of times over their lifecycle.
•

Domain Generalization: On some benchmarks, the model produces “confident hallucinations” when applying learned parallel patterns to unfamiliar algorithmic domains. Future work will explore integrating formal verification tools into the evolutionary loop to constrain these semantic errors.

6 Conclusion

We have presented ParEVO, a framework that bridging modern generative AI and high-performance computing. By curating a specialized dataset of parallel primitives and fine-tuning models to internalize the Work-Depth cost model, we achieve state-of-the-art results on the ParEval, surpassing both commercial LLMs and traditional heuristics.

Crucially, our results reveal that syntax generation alone is insufficient for HPC. The integration of an Evolutionary Coding Agent—which treats the compiler and runtime profiler as adversarial critics—is essential for traversing the optimization landscape. This work sets a precedent for AI-Driven Performance Engineering: moving beyond code completion to systems that actively reason about scalability, correctness, and the complex interplay between algorithms and hardware.

Acknowledgements

We thank Lin Zhong for helpful discussions and Rust resources, Ramla Ijaz for helpful discussions, and Roger Fu for compiling and providing to us the publicly available test cases for the competitive programming problems we used. We also thank the extended team at Google DeepMind who supported this research direction. Amir Yazdanbakhsh and Deniz Altınbüken contributed to this paper in an advisory capacity.

This work was supported in part by the National Science Foundation (NSF) under Grant #CCF-2453323 and a Google Academic Research Award.

Impact Statement

This paper presents work whose goal is to advance the field of Machine Learning and High Performance Computing. By enabling easier access to efficient parallel programming, this work could reduce the energy footprint of large-scale computations as well as make parallel computing accessible to non-experts. However, as with all code generation tools, there is a risk of generating subtle bugs in critical systems if not properly verified. We recommend human oversight for mission-critical applications.

References

Abdi et al. (2023a) J. Abdi, G. Zhang, and M. C. Jeffrey Brief announcement: is the problem-based benchmark suite fearless with rust?. In Proceedings of the 35th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’23), External Links: Document Cited by: §2, §3.4.
Abdi et al. (2023b) J. Abdi, G. Zhang, and M. C. Jeffrey Rusty-PBBS: rust problem based benchmark suite. GitHub. Note: https://github.com/mcj-group/rpbGitHub repository Cited by: §3.1.2, §3.4, §3.5, item 2.
Ahmed and Devanbu (2022) T. Ahmed and P. Devanbu Learning code summarization from a small and local dataset. External Links: 2206.00804, Link Cited by: §2.
Anderson et al. (2020) D. Anderson, G. Blelloch, L. Dhulipala, T. Tseng, wheatman, L. Hübschle, R. Yesantharao, X. Dong, and aheydon-google ParlayLib: a toolkit for programming parallel algorithms on shared-memory multicore machines (github repository; cmu parlay group). External Links: Link Cited by: §2.
Anderson et al. (2022) D. Anderson, G. E. Blelloch, L. Dhulipala, M. Dobson, and Y. Sun The problem-based benchmark suite (PBBS), V2. In Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’22), External Links: Document Cited by: §2.
Assumpção et al. (2025) H. Assumpção, D. Ferreira, L. Campos, and F. Murai CodeEvolve: an open source evolutionary coding agent for algorithm discovery and optimization. arXiv preprint arXiv:2510.14150. Cited by: §2.
Aygün et al. (2025) E. Aygün, A. Belyaeva, G. Comanici, M. Coram, H. Cui, J. Garrison, R. Johnston, A. Kast, C. Y. McLean, P. Norgaard, Z. Shamsi, D. Smalling, J. Thompson, S. Venugopalan, B. P. Williams, C. He, S. Martinson, M. Plomecka, L. Wei, Y. Zhou, Q. Zhu, M. Abraham, E. Brand, A. Bulanova, J. A. Cardille, C. Co, S. Ellsworth, G. Joseph, M. Kane, R. Krueger, J. Kartiwa, D. Liebling, J. Lueckmann, P. Raccuglia, X. (. Wang, K. Chou, J. Manyika, Y. Matias, J. C. Platt, L. Dorfman, S. Mourad, and M. P. Brenner An ai system to help scientists write expert-level empirical software. arXiv preprint arXiv:2509.06503. External Links: 2509.06503, Document, Link Cited by: §2.
Bitan et al. (2025) T. Bitan, T. Kadosh, E. Kaplan, S. Meiri, L. Chen, P. Morales, N. Hasabnis, and G. Oren UniPar: a unified llm-based framework for parallel and accelerated code translation in hpc. In 2025 IEEE High Performance Extreme Computing Conference (HPEC), Cited by: §2.
Blelloch et al. (2020) G. E. Blelloch, D. Anderson, and L. Dhulipala ParlayLib: A Toolkit for Parallel Algorithms on Shared-Memory Multicore Machines. In Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures, SPAA ’20, New York, NY, USA, pp. 507–509. External Links: ISBN 9781450369350, Link, Document Cited by: 2nd item, §1, §2, §3.4.
Blumofe and Leiserson (1999) R. D. Blumofe and C. E. Leiserson Scheduling multithreaded computations by work stealing. J. ACM 46 (5), pp. 720–748. External Links: ISSN 0004-5411, Link, Document Cited by: §2.
Brent (1974) R. P. Brent The parallel evaluation of general arithmetic expressions. Journal of the ACM 21 (2), pp. 201–206. Cited by: §2.
Bronson et al. (2013) N. Bronson, Z. Amsden, G. Cabrera, P. Chakka, P. Dimov, H. Ding, J. Ferris, A. Giardullo, S. Kulkarni, H. Li, M. Marchukov, D. Petrov, L. Puzar, Y. J. Song, and V. Venkataramani TAO: facebook’s distributed data store for the social graph. In Proceedings of the 2013 USENIX Conference on Annual Technical Conference, USENIX ATC’13, USA, pp. 49–60. Cited by: §2.
Chaturvedi (2024) A. Chaturvedi HPCCoder-v2: efficient fine-tuning of small language models for high-performance computing. arXiv preprint arXiv:2410.20527. Cited by: §2.
Chen et al. (2024a) L. Chen, A. Bhattacharjee, N. Ahmed, N. Hasabnis, G. Oren, V. Vo, and A. Jannesari OMPGPT: a generative pre-trained transformer model for openmp. In Euro-Par 2024: Parallel Processing, pp. 121–134. External Links: ISBN 9783031695773, ISSN 1611-3349, Link, Document Cited by: §2.
Chen et al. (2018) T. Chen, L. Zheng, E. Yan, Z. Jiang, T. Moreau, L. Ceze, C. Guestrin, and A. Krishnamurthy Learning to optimize tensor programs. In Advances in Neural Information Processing Systems (NeurIPS), pp. 3393–3404. Cited by: §2.
Chen et al. (2024b) X. Chen, M. Lin, N. Schärli, and D. Zhou Teaching large language models to self-debug. In International Conference on Learning Representations (ICLR), Cited by: §2.
Cummins et al. (2025) C. Cummins, V. Seeker, D. Grubisic, B. Rozière, J. Gehring, G. Synnaeve, and H. Leather LLM compiler: foundation language models for compiler optimization. In Proceedings of the 34th ACM SIGPLAN International Conference on Compiler Construction (CC), External Links: Document Cited by: §2.
Cummins et al. (2022) C. Cummins, B. Wasti, J. Guo, B. Cui, J. Ansel, S. Gomez, S. Jain, J. Liu, O. Teytaud, B. Steiner, Y. Tian, and H. Leather CompilerGym: robust, performant compiler optimization environments for AI research. In IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pp. 92–105. External Links: Document Cited by: §2.
Davis et al. (2025a) J. H. Davis, D. Nichols, I. Khillan, and A. Bhatele ParEval-repo: a benchmark suite for evaluating llms with repository-level hpc translation tasks. In Proceedings of the International Conference on Parallel Processing (ICPP 2025), External Links: Document, Link Cited by: §2.
Davis et al. (2025b) J. H. Davis, D. Nichols, I. Khillan, and A. Bhatele ParEval-repo: a benchmark suite for evaluating llms with repository-level hpc translation tasks. arXiv preprint arXiv:2506.20938. External Links: 2506.20938, Document, Link Cited by: §2.
DMOJ Developers (2024) DMOJ Developers DMOJ: modern online judge. Note: https://github.com/DMOJ/online-judgeGitHub repository Cited by: 1st item, §3.1.1, §3.1.2, item 3.
Du et al. (2025) M. Du, L. A. Tuan, Y. Liu, Y. Qing, D. Huang, X. He, Q. Liu, Z. Ma, and S. Ng Afterburner: reinforcement learning facilitates self-improving code efficiency optimization. arXiv preprint arXiv:2505.23387. Cited by: §2.
Eniser et al. (2024) H. F. Eniser, H. Zhang, C. David, M. Wang, M. Christakis, B. Paulsen, J. Dodds, and D. Kroening Towards translating real-world code with llms: a study of translating to rust. External Links: 2405.11514, Link Cited by: §2.
Foundry et al. (2024) P. C. Foundry, D. Nichols, Y. Zi, Z. Xie, and H. Menon ParEval: parallel code evaluation benchmark (github repository). External Links: Link Cited by: §2.
GraphIt-DSL et al. (2018) GraphIt-DSL, Y. Zhang, T. Manlaibaatar, A. Brahmakshatriya, ykenny1, R. Baghdadi, E. Furst, G. Siegfried, B. Wade, and L. Dhulipala GraphIt-dsl/graphit: graphit compiler and dsl implementation (github repository). External Links: Link Cited by: §2.
Haj-Ali et al. (2020) A. Haj-Ali, N. K. Ahmed, T. L. Willke, Y. S. Shao, K. Asanovic, and I. Stoica NeuroVectorizer: end-to-end vectorization with deep reinforcement learning. In Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Optimization (CGO), pp. 242–255. External Links: Document Cited by: §2.
Hu et al. (2022) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), External Links: Link Cited by: §3.2.
Husein et al. (2025) R. A. Husein, H. Aburajouh, and C. Catal Large language models for code completion: a systematic literature review. Computer Standards & Interfaces 92, pp. 103917. External Links: ISSN 0920-5489, Document, Link Cited by: §2.
Kambhampati et al. (2024) S. Kambhampati, K. Valmeekam, L. Guan, M. Verma, K. Stechly, S. Bhambri, L. Saldyt, and A. Murthy LLMs can’t plan, but can help planning in llm-modulo frameworks. External Links: 2402.01817, Link Cited by: §1, §2.
kcxain (2025) kcxain Kcxain/translator-qwen3-0.6b: musl c-to-cuda translator model (hugging face). External Links: Link Cited by: §2.
Ke et al. (2025) C. Ke, R. Zhang, S. Wang, L. Ding, G. Li, Y. Wen, S. Zhang, R. Xu, J. Qin, J. Guo, C. Wang, L. Li, Q. Guo, and Y. Chen Mutual-supervised learning for sequential-to-parallel code translation. arXiv preprint arXiv:2506.11153. External Links: 2506.11153, Document, Link Cited by: §2.
Ke (2025) C. Ke Kcxain/musl: code repository for mutual-supervised learning for sequential-to-parallel code translation. External Links: Link Cited by: §2.
Khrulkov et al. (2025) V. Khrulkov, A. Galichin, D. Bashkirov, D. Vinichenko, O. Travkin, R. Alferov, A. Kuznetsov, and I. Oseledets GigaEvo: an open source optimization framework powered by llms and evolution algorithms. arXiv preprint arXiv:2511.17592. Cited by: §2.
Lei et al. (2025) K. Lei, H. Yang, H. Zhang, X. You, K. Zhang, Z. Luan, Y. Liu, and D. Qian PRAGMA: a profiling-reasoned multi-agent framework for automatic kernel optimization. arXiv preprint arXiv:2511.06345. Cited by: §2.
Mahmud et al. (2025) Q. I. Mahmud, A. TehraniJamsaz, H. D. Phan, L. Chen, M. Capotă, T. L. Willke, N. K. Ahmed, and A. Jannesari AutoParLLM: GNN-guided context generation for zero-shot code parallelization using LLMs. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), L. Chiruzzo, A. Ritter, and L. Wang (Eds.), Albuquerque, New Mexico, pp. 11821–11841. External Links: Link, Document, ISBN 979-8-89176-189-6 Cited by: §2.
Merouani et al. (2025) M. Merouani, I. K. Bernou, and R. Baghdadi Agentic auto-scheduling: an experimental study of llm-guided loop optimization. arXiv preprint arXiv:2511.00592. Cited by: §2.
Ni et al. (2023) A. Ni, S. Iyer, D. Radev, V. Stoyanov, W. Yih, S. I. Wang, and X. V. Lin LEVER: learning to verify language-to-code generation with execution. In International Conference on Machine Learning (ICML), Cited by: §2.
Nichols et al. (2024) D. Nichols, J. H. Davis, Z. Xie, A. Rajaram, and A. Bhatele Can large language models write parallel code?. In Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing (HPDC ’24), External Links: Document, Link Cited by: Figure 17, Figure 17, Appendix D, §1, §2, §3.5, item 1.
Novikov et al. (2025) A. Novikov, N. Vũ, M. Eisenberger, E. Dupont, P. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. R. Ruiz, A. Mehrabian, M. P. Kumar, A. See, S. Chaudhuri, G. Holland, A. Davies, S. Nowozin, P. Kohli, and M. Balog AlphaEvolve: a coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131. Cited by: §2.
ParEVO (2026a) ParEVO DeepSeek-parlay-6.7b: a fine-tuned model for parallel algorithmic reasoning. External Links: Link Cited by: 2nd item.
ParEVO (2026b) ParEVO ParEVO project repository (code/data for parevo; includes parlay-instruct artifacts). External Links: Link Cited by: §1, Abstract.
ParEVO (2026c) ParEVO Qwen3-30b-sft-stage2-merged. Hugging Face. External Links: Link Cited by: 2nd item.
ParEVO (2026d) ParEVO Qwen3-rust-dpo: a fine-tuned model for safe parallel rust. External Links: Link Cited by: 2nd item.
Press et al. (2025a) O. Press, B. Amos, H. Zhao, Y. Wu, S. K. Ainsworth, D. Krupke, P. Kidger, T. Sajed, B. Stellato, J. Park, N. Bosch, E. Meril, A. Steppi, A. Zharmagambetov, F. Zhang, D. Perez-Pineiro, A. Mercurio, N. Zhan, T. Abramovich, K. Lieret, H. Zhang, S. Huang, M. Bethge, and O. Press AlgoTune benchmark dataset. External Links: Link Cited by: §2.
Press et al. (2025b) O. Press, B. Amos, H. Zhao, Y. Wu, S. K. Ainsworth, D. Krupke, P. Kidger, T. Sajed, B. Stellato, J. Park, N. Bosch, E. Meril, A. Steppi, A. Zharmagambetov, F. Zhang, D. Perez-Pineiro, A. Mercurio, N. Zhan, T. Abramovich, K. Lieret, H. Zhang, S. Huang, M. Bethge, and O. Press AlgoTune: can language models speed up general-purpose numerical programs?. arXiv preprint arXiv:2507.15887. External Links: 2507.15887, Document, Link Cited by: §2.
Rahman (2025) Md. Rahman MARCO: multi-agent reasoning for code optimization. arXiv preprint arXiv:2501.12345. Cited by: §2.
Ren et al. (2020) S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, and S. Ma CodeBLEU: a method for automatic evaluation of code synthesis. External Links: 2009.10297, Link Cited by: §2.
Sahu et al. (2019) S. Sahu, A. Mhedhbi, S. Salihoglu, J. Lin, and M. T. Özsu The ubiquity of large graphs and surprising challenges of graph processing: extended survey. The VLDB Journal 29 (2–3), pp. 595–618. External Links: ISSN 0949-877X, Link, Document Cited by: §1, §2.
Sharma (2025a) A. Sharma OpenEvolve: an open source implementation of google deepmind’s alphaevolve. GitHub. Note: https://github.com/codelion/openevolveAccessed: 2026-01-24 Cited by: §3.1.2, §3.1.
Sharma (2025b) OpenEvolve: an open-source evolutionary coding agent External Links: Link Cited by: §2.
Shinn et al. (2023) N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 36. Cited by: §2.
Shun et al. (2012) J. Shun, G. E. Blelloch, A. Kyrola, H. V. Simhadri, K. Tangwongsan, J. T. Fineman, and P. B. Gibbons Brief announcement: the problem based benchmark suite. In Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’12), External Links: Document Cited by: §2, §3.5, item 2.
Shun and Blelloch (2013) J. Shun and G. E. Blelloch Ligra: a lightweight graph processing framework for shared memory. In Proceedings of the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 135–146. External Links: Document, Link Cited by: §2.
Shypula et al. (2024) A. Shypula, A. Madaan, Y. Zeng, U. Alon, J. Gardner, M. Hashemi, G. Neubig, P. Ranganathan, O. Bastani, and A. Yazdanbakhsh Learning performance-improving code edits. External Links: 2302.07867, Link Cited by: §3.1.2.
Singh et al. (2024) G. Singh, A. Guha, B. Kailkhura, and H. Menon Can test-time compute help LLMs write low-resource parallel code better?. In NeurIPS Workshop on Deep Learning for Code (DL4C), External Links: Link Cited by: §2, §3.3.
Surina et al. (2025) A. Surina, A. Mansouri, L. Quaedvlieg, A. Seddas, M. Viazovska, E. Abbe, and C. Gulcehre Algorithm discovery with llms: evolutionary search meets reinforcement learning. CoRR abs/2504.05108. External Links: 2504.05108, Document, Link Cited by: §2.
Tang et al. (2025) A. S. Tang, C. Priebe, R. Mahapatra, L. Qin, and H. Esmaeilzadeh Reasoning compiler: LLM-guided optimizations for efficient model serving. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2.
TehraniJamsaz et al. (2024) A. TehraniJamsaz, A. Bhattacharjee, L. Chen, N. K. Ahmed, A. Yazdanbakhsh, and A. Jannesari CODEROSETTA: pushing the boundaries of unsupervised code translation for parallel programming. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. External Links: ISBN 9798331314385 Cited by: §2.
Trofin et al. (2021) M. Trofin, Y. Qian, E. Brevdo, Z. Lin, K. Choromanski, and D. Li MLGO: a machine learning guided compiler optimizations framework. arXiv preprint arXiv:2101.04808. Cited by: §2.
Wen et al. (2022) Y. Wen, Q. Guo, Q. Fu, X. Li, J. Xu, Y. Tang, Y. Zhao, X. Hu, Z. Du, L. Li, C. Wang, X. Zhou, and Y. Chen BabelTower: learning to auto-parallelized program translation. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162, pp. 23685–23700. External Links: Link, Document Cited by: §2.
Yang et al. (2023) J. Yang, A. Prabhakar, K. Narasimhan, and S. Yao InterCode: standardizing and benchmarking interactive coding with execution feedback. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Cited by: §2.
Yang (2025) Z. Yang PerfCoder: performance-driven code generation. arXiv preprint arXiv:2502.54321. Cited by: §2.
Zhang et al. (2018) Y. Zhang, M. Yang, R. Baghdadi, S. Kamil, J. Shun, and S. P. Amarasinghe GraphIt: a high-performance graph dsl. Proceedings of the ACM on Programming Languages 2 (OOPSLA), pp. 121:1–121:30. External Links: Document, Link Cited by: §2.
Zheng et al. (2020) L. Zheng, C. Jia, M. Sun, Z. Wu, C. H. Yu, A. Haj-Ali, Y. Wang, J. Yang, D. Zhuo, K. Sen, J. E. Gonzalez, and I. Stoica Ansor: generating high-performance tensor programs for deep learning. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pp. 863–879. Cited by: §2.
Zhou et al. (2024) A. Zhou, K. Yan, M. Shlapentokh-Rothman, H. Wang, and Y. Wang Language agent tree search unifies reasoning, acting, and planning in language models. In Proceedings of the 41st International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, Vol. 235, pp. 62138–62160. Cited by: §2.

Appendix A Evolutionary Coding Agent (ECA) System Prompt

The structural system prompt we specify for the single ECA node is as follows:

Appendix B Examples of Fine-Tuning Dataset

B.1 Example of Event Generation Strategies.

Code A (Efficient): Parallel Map-Scan-Write

⬇

1 // 1. MAP: Count events in parallel (No Locking)

2 parlay::sequence<int> counts(N);

3 parlay::parallel_for(0, N, [&](int r) {

4 int cnt = 0; /* logic checks ... */

5 if (valid) cnt++;

6 counts[r] = cnt;

7 });

9 // 2. SCAN: Calculate offsets (Prefix Sum)

10 auto [offsets, total] = parlay::scan(counts);

12 // 3. WRITE: Parallel Fill (Zero Realloc/Contention)

13 parlay::sequence<Event> evs(total); // Alloc exact size

14 parlay::parallel_for(0, N, [&](int r) {

15 int k = offsets[r];

16 /* logic checks ... */

17 if (valid) evs[k++] = {u, v, t};

18 });

Code B (Inefficient): Sequential Push

⬇

1 // 1. Setup Vector (Heuristic reservation)

2 std::vector<Event> events;

3 events.reserve(2 * N * N); // May still realloc

5 // 2. Iterate Sequentially (Cannot Parallelize)

6 for (int r = 0; r < N; ++r) {

7 for (int c = 0; c < N; ++c) {

8 /* logic checks ... */

10 // BOTTLENECK: Single thread, capacity checks,

11 // and reallocation overhead.

12 if (valid) {

13 events.push_back({u, v, t});

14 }

15 }

16 }

Figure 3: Comparison of Event Generation Strategies. Left: Code A employs a Map-Scan-Write pattern to enable lock-free parallel writing. Right: Code B relies on sequential push_back, preventing parallelization and incurring reallocation costs.

B.2 Example of Hidden Unit Test

Figure 4 is an example of the finetuning data and the corresponding unit test for verification.

Figure 4: A representative sample from the training corpus. Each sample includes a natural language instruction, the ground-truth parallel implementation, and an executable unit test used for verification.

Appendix C Detailed Experimental Data

C.1 Comprehensive ParEval Benchmarks

Table 5 provides the complete breakdown of ‘Build@1‘, ‘Pass@1‘, and ‘Speedup@1‘ metrics across commercial and open-weight models. The fine-tuned ParEVO models consistently outperform baselines in compilation rates and execution speed.

Model	Temp.	Code	Sched.	Build@1	Pass@1	Speedup@1
						AM	GM w/ Out.	GM w/o Out.
Gemini-2.5-Flash	0.2	Parlay	Parlay	0.57	0.29	6.88	2.75	2.41
Gemini-2.5-Pro	0.2	Parlay	Parlay	0.84	0.54	85.34	5.47	3.17
Gemini-3-Pro	0.2	Parlay	Parlay	0.25	0.23	3.68	1.53	1.42
GPT-5 Thinking	0.2	Parlay	Parlay	0.93	0.05	0.43	1.13	1.13
Claude Opus 4.5	0.2	Parlay	Parlay	0.97	0.05	0.42	1.13	1.13
Gemini-2.5-Parlay (ParEVO)	0.2	Parlay	Parlay	0.81	0.58	107.43	6.33	3.74
Qwen3-Parlay (ParEVO)	0.2	Parlay	Parlay	0.50	0.33	5.01	1.87	1.64
DeepSeek-6.7B-Base	0.2	Parlay	Parlay	0.89	0.11	0.73	1.14	1.14
DeepSeek-Syntax	0.2	Parlay	Parlay	0.85	0.12	2.42	1.38	1.29
DeepSeek-Parlay (ParEVO)	0.2	Parlay	Parlay	0.81	0.26	126.16	3.14	1.63
Qwen2.5-Coder-32B	0.2	Parlay	Parlay	0.93	0.11	5.02	1.82	1.59
Qwen2.5-Coder-32B	0.7	Parlay	Parlay	0.93	0.18	11.80	2.65	1.98
Qwen2.5-Coder-32B-Instruct	0.2	Parlay	Parlay	0.61	0.41	7.00	2.22	1.82
Qwen3-Coder-30B-Instruct	0.2	Parlay	Parlay	0.51	0.28	5.15	1.96	1.72
DeepSeek-Coder-V2-Lite-Base	0.2	Parlay	Parlay	0.80	0.09	1.30	1.19	1.19
DeepSeek-Coder-V2-Lite-Base	0.7	Parlay	Parlay	0.92	0.14	4.74	1.71	1.59
StarCoder2-15B	0.2	Parlay	Parlay	0.80	0.27	7.65	2.02	1.62
Gemini-3-Pro	0.2	OMP	OMP	0.78	0.72	445.14	6.75	3.20
Gemini-2.5-Parlay	0.2	OMP	OMP	0.94	0.71	373.78	7.77	3.22
GPT-5 Thinking	0.2	OMP	OMP	0.95	0.06	0.60	1.15	1.15
Claude Opus 4.5	0.2	OMP	OMP	0.97	0.07	0.52	1.14	1.14
Qwen2.5-Coder-32B-Instruct	0.2	OMP	OMP	0.91	0.65	10.47	2.77	1.98
Qwen2.5-Coder-32B-Instruct	0.7	OMP	OMP	0.92	0.65	11.20	3.44	2.46
Qwen3-Coder-30B-Instruct	0.2	OMP	OMP	0.86	0.55	11.88	2.45	1.61
Qwen3-Coder-30B-Instruct	0.7	OMP	OMP	0.91	0.56	11.47	2.53	1.68
Qwen2.5-Coder-32B	0.2	OMP	OMP	0.98	0.35	10.37	2.56	1.61
Qwen2.5-Coder-32B	0.7	OMP	OMP	0.97	0.39	11.51	3.48	2.47
DeepSeek-Coder-V2-Lite-Base	0.2	OMP	OMP	0.82	0.24	5.60	2.25	2.09
DeepSeek-Coder-V2-Lite-Base	0.7	OMP	OMP	0.96	0.31	12.47	3.35	2.21
StarCoder2-15B	0.2	OMP	OMP	0.97	0.26	8.23	1.95	1.47
Gemini-3-Pro	0.2	Rust	Rayon	0.97	0.82	7.32	0.69	0.61
Qwen2.5-Coder-32B-Instruct	0.2	Rust	Rayon	0.63	0.49	4.61	0.57	0.51
Qwen2.5-Coder-32B-Instruct	0.7	Rust	Rayon	0.70	0.48	5.69	0.51	0.44
Qwen3-Rust (ParEVO)	0.2	Rust	Rayon	0.64	0.46	4.62	0.35	0.29
Qwen3-Coder-30B-Instruct	0.2	Rust	Rayon	0.61	0.50	4.57	0.46	0.38
Qwen3-Coder-30B-Instruct	0.7	Rust	Rayon	0.66	0.49	4.94	0.29	0.24
Qwen2.5-Coder-32B	0.2	Rust	Rayon	0.82	0.45	4.43	0.63	0.63
Qwen2.5-Coder-32B	0.7	Rust	Rayon	0.86	0.38	4.35	0.79	0.79
DS-Coder-V2-Lite-Base	0.2	Rust	Rayon	0.73	0.29	3.47	0.58	0.54
DS-Coder-V2-Lite-Base	0.7	Rust	Rayon	0.85	0.25	4.96	0.89	0.78
StarCoder2-15B	0.2	Rust	Rayon	0.77	0.27	2.46	0.79	0.74
StarCoder2-15B	0.7	Rust	Rayon	0.82	0.25	5.67	0.63	0.49
DS-Coder-V2-Lite-Instruct	0.2	Rust	Rayon	0.40	0.02	0.15	1.04	1.04
DS-Coder-V2-Lite-Instruct	0.7	Rust	Rayon	0.48	0.02	0.15	1.04	1.04

Table 5: Comprehensive ParEval results for commercial and local models. Shaded regions distinguish C++/ParlayLib/OMP models (Green) from Rust/Rayon models (Purple). To isolate pure parallel scaling, we also conducted an ablation removing programs with speedups ¿ 32x (indicating algorithmic improvement beyond physical core counts), as denoted by the GM w/o Out. column.

C.2 Metric Breakdown by Problem Type

To understand the specific impact of fine-tuning, we visualize the shift in metrics across problem types. Figure 5 and Figure 6 demonstrate that while fine-tuning universally improves Build and Pass rates, the Speedup gains are most pronounced in the irregular graph and complex arithmetic categories. This applies to Figure 7 except that we see occasional lower Pass rate after finetuning.

(a) Mean Build@1

(b) Mean Pass@1

Figure 5: ParEval Metrics Comparison between Gemini-2.5-Pro and Gemini-2.5-Parlay, where the former is the base model and the latter is the base model finetuned on the Parlay-Instruct and DMOJ datasets. (a-c) highlight that fine-tuning significantly improves the model’s ability to construct valid ParlayLib code, with substantial gains in build and pass rates as well as improved running time over the base model.

(a) Mean Build@1

(b) Mean Pass@1

Figure 6: Impact of Fine-tuning on DeepSeek-6.7B. The fine-tuned model (DeepSeek-Parlay) shows massive gains in pass rate and speedup compared to the base model. The DeepSeek-Syntax model is the finetuned model of DeepSeek-6.7B-Base purely on ParlayLib syntax.

(a) Mean Build@1

(b) Mean Pass@1

Figure 7: Impact of Fine-tuning on Qwen3-Coder-30B-A3B-Instruct. The fine-tuned model (Qwen3-Rust) shows gains in speedup compared to the base model.

C.3 Comparison vs. Expert Human Baselines (PBBS & RPB)

A key contribution of this work is benchmarking against expert human code. Figure 8 compares our best generated solutions against the PBBSBench (C++) and RPB (Rust) baselines. ParEVO solutions frequently match or exceed the human baselines. Figure 8 visualizes the runtime and scalability profiles.

(a) MM Runtime (Rust)

(b) MM Scalability (Rust)

(d) MSF Runtime (Rust)

(e) MSF Scalability (Rust)

(f) MSF Relative Speedup (Rust)

(g) MIS (Rust)

(h) MIS (Rust)

(i) MIS (Rust)

(j) SP (Rust)

(k) SP (Rust)

(l) SP (Rust)

(m) BFS (Rust)

(n) BFS (Rust)

(o) BFS (Rust)

Figure 8: Runtime and Scalability comparisons against expert Rust and C++ baselines. ParEVO solutions track or beat the scalability of hand-optimized code. In (m)-(o), BackForward BFS specifically refers to the new BFS algorithm ParEVO generated, which uses a different method than the baseline implementation that uses multiqueue BFS.

Figure 9: Code comparison for Maximal Matching. Left (Baseline): Uses mixed safe/unsafe access (potential aliasing bugs) and collects results using two separate passes (red highlights). Right (ParEVO): Uses consistent raw pointer access to ensure memory visibility, increases block granularity to 16, and uses a single-pass unzip for result collection (green highlights).

Figure 10: Code comparison for Minimum Spanning Forest (MSF). Left (Baseline): Uses standard indexing for the reservation array (incurring bounds checks) and standard unwrap() (incurring branch checks), highlighted in red. Right (ParEVO): Adopts a “Maximal Unsafe” strategy, converting all data structures to raw pointers. It uses unwrap_unchecked() and pointer arithmetic (.add()) to eliminate all runtime safety checks, highlighted in green. This relies on the assumption that edge indices are always valid, allowing ParEVO to trade runtime safety checks for improved performance.

Figure 11: Code comparison for Maximal Independent Set (MIS). Left (Baseline): Uses unsafe standard Vec<u8> (red), causing undefined behavior (data races) during reads and writing via raw pointers. It uses a small block size (20). Right (ParEVO): Uses Vec<AtomicU8> (green) for correct synchronization using Relaxed ordering. It optimizes throughput with a larger block size (256) and employs a zero-copy cast to convert the atomic vector back to a standard vector at the end.

Figure 12: Code comparison for Spanning Forest. Left (Baseline): Performs a fresh allocation for the reservation array on every call (red) and uses checked indexing/unwrapping inside the hot loop. Right (ParEVO): Implements a memory recycling mechanism via rs_cache (green) to reuse the large reservation vector across calls. It also employs get_unchecked and AtomicUnionFind to eliminate bounds checking and pointer dereference overheads.

Figure 13: Code comparison for BFS. Left (Baseline): Uses a standard asynchronous approach where every edge relaxation requires a CAS loop and a queue push (red), leading to high contention on scale-free graphs. Right (ParEVO): Implements Direction-Optimizing BFS (Ligra-style). It dynamically switches between ”Push” (Sparse) and ”Pull” (Dense) modes based on the frontier density (green), drastically reducing edge checks during the heavy middle levels of the traversal.

C.4 Case Study: The Safety vs. Performance Trade-off

A deeper analysis of the Graph Shortest Path problem reveals a subtle trade-off introduced by fine-tuning. As shown in Figure 14, the base model produces a wide variance of runtimes, occasionally hitting a very fast (but risky) solution using atomic operations. The fine-tuned ParEVO model produces highly consistent but slightly slower code, preferring safe high-level primitives (like ‘parlay::unique’) over raw memory manipulation. The detailed code samples are shown in Figure 15.

(a) Base Model Runtime Distribution

(b) Fine-tuned Runtime Distribution

Figure 14: Runtime Histograms for Graph Shortest Path. The fine-tuned model (b) exhibits tighter variance (reliability) but a higher median runtime due to overhead from safety-focused primitives.

Figure 15: Code comparison for Shortest Path (Problem 19 in ParEval). Left (Baseline): Effectively uses std::atomic and Compare-and-Swap (CAS) to manage visitation state in parallel, resulting in a significantly faster runtime. Right (Finetuned): Chooses a high-overhead initialization step (copying the adjacency matrix via tabulate) and falls back to sequential logic for the queue update loop (red), causing

O(N^{2})

startup cost and serialization bottlenecks. Nonetheless, it shows heavier abstraction usage.

C.5 Case Study: Performance Stability on ParEval Problem 34 (Scan)

Similarly to the Shortest Path problem, we observe a distinct stabilization of performance in the fine-tuned model for ParEval Problem 34 (Scan), as shown in Figure 16. The base model’s runtime distribution Figure 16(a) is somewhat disjointed, with some runs being very slow and others faster. On the other hand, the fine-tuned model Figure 16(b) demonstrates a much tighter, more predictable runtime distribution. This consistency confirms that the fine-tuned ParEVO model systematically converges on stable and reliable parallel patterns.

(a) Base Model Runtime Distribution

(b) Fine-tuned Runtime Distribution

Figure 16: Runtime Histograms for ParEval Problem 34 (Scan). The fine-tuned model (b) exhibits tighter variance and highly predictable performance compared to the wider distribution of the base model (a).

C.6 Failure Modes: Geometric Hallucinations

While fine-tuning improves general syntax, it can induce “confident hallucinations” in domains with specialized logic. In the Convex Hull task (Table 6), the fine-tuned model failed by repeatedly calling a non-existent parlay::convex_hull function, whereas the base model attempted (and occasionally succeeded at) a manual implementation. This highlights the necessity of the ECA’s compiler-feedback loop to catch API hallucinations.

Problem Type	Model	Pass@1	Speedup@1
10_convex_hull	Gemini-2.5-Pro	0.45	1.43
10_convex_hull	DS-Parlay (ParEVO)	0.00	0.00
13_closest_pair_2d	Gemini-2.5-Pro	0.40	74.48
13_closest_pair_2d	DS-Parlay (ParEVO)	0.45	188.03

Table 6: Detailed Geometry Results. The fine-tuned model dominates in Closest Pair but hallucinates APIs in Convex Hull.

Appendix D Prompts

ParEval Prompts

For the ParEval benchmarks, we adopt the prompting specifications outlined by Nichols et al. (38). We utilize a fixed system instruction alongside language-specific templates for C++ and Rust.

The system prompt provided to the model is as follows (see Figure 17).

Figure 17: The prompting strategy adopted from the ParEval paper (38). The templates include specific placeholders (function_name, prompt) populated dynamically during evaluation.

Extending ParEval for Parallel Libraries

Since the original ParEval dataset lacks native support for ParlayLib and Rust, we manually curated task-specific prompts to bridge this gap. These prompts preserve the original problem semantics while explicitly requesting the use of specific parallel frameworks (ParlayLib for C++ and Rayon for Rust). Figure 18 demonstrates how a standard Discrete Fourier Transform (DFT) task is adapted for both languages.

Figure 18: Representative examples of our manual extensions to the ParEval dataset. The prompts are tailored to enforce specific parallel backends while maintaining identical input/output specifications.

PBBSBench Prompting Strategy

We employ two distinct prompting strategies for PBBSBench to evaluate the model’s ability to utilize context:

•

Concise Prompts: These contain only the natural language problem description and the target function signature.
•

Augmented Prompts: These extend the concise version by including definitions for necessary ParlayLib primitives, custom data structures (e.g., Graph), and helper utilities (e.g., Graph_io) defined within the PBBSBench environment.

Figure 19 illustrates an example of the concise prompting format.

Figure 19: An example of the Concise prompt formulation for the PBBSBench maximalIndependentSet task. The model is provided with the function signature and a request to use ParlayLib, but implementation details of the Graph structure are omitted.

RPB Prompting Strategy

The prompting strategy for the RPB benchmarks relies on a composite structure. Each prompt comprises two distinct segments: (1) a context block defining Rust primitives that replicate ParlayLib functionality (e.g., flatten), and (2) the specific problem statement, including allowed libraries and the target function signature.

Figure 20: An example of the RPB prompting template. We inject the full set of parallel primitive definitions (represented by the flatten excerpt in Part 1) prior to the specific task instructions (Part 2) to ground the model in the available Rust-ParlayLib equivalence layer.

Appendix E Fine-tuning on Low-level APIs vs. ParlayLib Token-efficiency

To assess whether the choice of parallel abstraction affects fine-tuning efficiency, we compare deepseek-coder-6.7b-base fine-tuned on our Parlay-Instruct dataset against an equivalent model fine-tuned on a token-matched OpenMP dataset.

The OpenMP dataset was sourced from hpcgroup/hpc-instruct and cleaned to match the 2.14M token budget of Parlay-Instruct, isolating core algorithmic loops to provide a dense learning signal. The filtering process consisted of:

1.

Dropping translation tasks and non-native OpenMP languages (Python, Chapel, OpenCL, CUDA).
2.

Retaining only samples with actual #pragma omp or !$omp directives in the solution.
3.

Dropping outputs from weaker generator models (e.g., Mixtral, DBRX).
4.

Deduplicating by seed, keeping the best model output per seed.
5.

Trimming the longest samples to hit the 2.14M token target, retaining shorter, more focused examples.

Both models were trained with identical SFT+LoRA hyperparameters (LoRA $r=8$ , $\alpha=16$ , dropout $=0.05$ , max_seq_length $=2048$ , LR $=2\times 10^{-4}$ ). Results on three representative PBBS problems are shown in Table 7.

Model	Exec. Model	Problem Name	build@1	pass@1	speedup@1
DeepSeek-Syntax	parlay	06_fft_dft	0.937	0.020	30.754
DeepSeek-Syntax	parlay	16_graph_largest_component	0.856	0.060	3.517
DeepSeek-Syntax	parlay	19_graph_shortest_path	0.919	0.080	2.187
DeepSeek-OMP	omp	06_fft_dft	1.000	0.550	23.180
DeepSeek-OMP	omp	16_graph_largest_component	1.000	0.000	0.000
DeepSeek-OMP	omp	19_graph_shortest_path	0.960	0.100	0.000

Table 7: Token-matched comparison of Parlay-Instruct vs. OpenMP fine-tuning on deepseek-coder-6.7b-base.

Under a fixed token budget, the Parlay-Instruct model achieves higher speedups on successful runs, while the OpenMP model produces more frequently compiling and passing solutions but with limited parallel gains. This suggests that high-level abstractions like ParlayLib provide a more token-efficient signal for learning parallel reasoning, whereas explicit low-level thread and memory management consumes capacity that might otherwise go toward algorithmic structure. Full PBBSBench evaluations will appear in a subsequent revision.

Appendix F Ablation Study on Agent Architecture and Search Strategies

To rigorously evaluate the structural contributions of our evolutionary pipeline, we conducted an ablation study utilizing an open-weight local model (Qwen3-Coder-30B-Instruct). A mid-sized open model is employed to eliminate confounding variables associated with opaque, massive-scale models, and to ensure that any observed performance gains are directly attributable to our architectural advancements—specifically, the integration of MAP-Elites and dynamic execution feedback—rather than sheer parameter count.

The baseline model for all non-finetuned configurations is Qwen3-Coder-30B-Instruct, while our complete pipeline utilizes the fine-tuned Qwen-Parlay. We evaluate the following configurations:

•

Best-of-N: Standard independent sampling without execution-driven feedback.
•

Best-of-N + CoT: Independent sampling augmented with Chain-of-Thought prompting.
•

Self-Refine: A single-lineage, text-only iterative refinement strategy.
•

ECA-base: Our Evolutionary Coding Agent (ECA) pipeline applied to the baseline model, lacking domain-specific fine-tuning.
•

ECA_no_div: The ECA pipeline utilizing runtime execution feedback, but with the MAP-Elites diversity archive disabled.
•

ECA-ft (Full ParEVO): Our proposed methodology, combining the fine-tuned Qwen-Parlay model with the complete, diversity-driven ECA pipeline.

Problem ID	Best-of-N	Best-of-N + CoT	Self-Refine	ECA-base	ECA_no_div	ECA-ft (Full)
coci11c1p3	0.1019	0.1018	0.3200	0.1054	0.1017	0.0297
ccc16j3	0.0026	0.0025	0.0031	0.0030	0.0026	0.0019
coci06c1p5	-	-	0.0040	0.0068	-	0.0069
coci12c1p4	-	0.0265	0.0264	0.0774	0.0267	0.0147
coci17c4p1	0.0030	0.0030	0.0030	0.0032	0.0018	0.0017
coci22c3p1	0.0027	0.0031	0.0032	0.0030	-	0.0030
crci08p1	0.0030	0.0031	0.0018	0.0033	-	0.0017

Table 8: Ablation results evaluating runtime performance (in seconds) across diverse competitive programming challenges. Problems ids refer to problems in the DMOJ dataset. A “-” indicates that the generated solution failed to pass the required unit tests, resulting in no valid runtime.

The results demonstrate that ParEVO’s primary advantage over standard iterative repair lies in its ability to escape local optima. Single-lineage refinement methods (e.g., Self-Refine and ECA_no_div) frequently converge on structurally flawed algorithms, exhaustively attempting syntactic patches without exploring alternative logic. By leveraging MAP-Elites to enforce structural diversity (e.g., varying synchronization primitives and underlying data structures), ECA-ft successfully navigates the broader search space to discover highly optimized parallel implementations where simpler pipelines fail to yield functionally correct solutions.

Appendix G Ablation Study on Iteration Budgets and Fine-Tuning Synergy

This section investigates the combined efficacy of fine-tuning and the Evolutionary Coding Agent (ECA). We evaluate the synergy of fine-tuning alongside the ECA pipeline across various iteration budgets and diversity settings on selected DMOJ problems.

For these experiments, the base model is Gemini-2.5-Pro, and the fine-tuned variant is Gemini-2.5-Parlay. We define the following ablation configurations:

•

ECA-Iter5, ECA-Iter15, ECA-Iter30: The baseline Gemini-2.5-Pro model utilizing the standard ECA pipeline (with a diversity archive of 5 programs) capped at 5, 15, and 30 evolutionary iterations, respectively.
•

ECA-No-Diversity: The baseline Gemini-2.5-Pro model running for 30 iterations, but with the MAP-Elites diversity mechanism disabled.
•

ECA-Finetuned (Full Synergy): Our fully proposed pipeline, pairing the fine-tuned Gemini-2.5-Parlay model with the complete ECA pipeline for 30 iterations.

Problem ID	ECA-Iter5	ECA-Iter15	ECA-Iter30	ECA-No-Diversity	ECA-Finetuned
cco08p4	-	0.00753	0.00698	0.00927	0.00706
coci19c1p3	-	-	-	-	0.02424
coci11c1p3	0.02606	0.02565	0.02534	0.00822	0.02681
coci23c2p2	0.03474	0.03166	0.03130	0.03500	0.03608

Table 9: Ablation results detailing runtime performance (in seconds) on DMOJ problems. A “-” indicates that the generated solution failed to pass the requisite tests, yielding no valid runtime. Iteration configurations dictate the maximum number of evolutionary cycles permitted.

Crucially, ECA-Finetuned emerges as the most robust model configuration. On exceptionally complex algorithms—such as the graph problem coci19c1p3—every base-model configuration failed to generate a valid parallel solution, regardless of the iteration allowance. Only the combination of domain-specific fine-tuning and the evolutionary search successfully navigated the solution space to yield a correct, optimized implementation. Furthermore, this variant maintains highly competitive runtimes across all other evaluated problems.

Additionally, isolating the search parameters reveals two critical methodological trends:

1.

Iteration Budgets: Expanding the evolutionary horizon consistently yields tighter, more optimized execution schedules. For instance, on cco08p4, a constrained budget of 5 iterations fails entirely, 15 iterations secures a valid solution, and 30 iterations discovers the most time-efficient implementation.
2.

Advantage of MAP-Elites Diversity: Disabling the structural diversity constraints (ECA-No-Diversity) reliably traps the agent in local optima. Without the pressure to explore alternative concurrency primitives or data structures, the search can sometimes lead to invalid or degraded solutions.

Appendix H Integrity of Unit Tests in Training Data Verification

Test integrity.

A natural concern with LLM-generated training data is whether the model effectively grades itself with trivial tests. Our pipeline avoids this by reusing the human-authored test logic from the 593 seed programs. The mutation operators alter intermediate algorithmic logic and datatypes for diversity, but they preserve the deterministic input–output mapping of each seed. The Teacher therefore synthesizes new test code only when $\mathcal{M}_{type}$ changes the input or output type; in all other cases the original human-written unit test is reused. The vast majority of the corpus is thus evaluated against tests the LLM did not write. Additionally, we uniformly sampled 100 programs from the 13,820 tasks for manual review, confirming that reused tests matched their seed and that synthesized datatype-adapted tests preserved the original semantics. All sampled programs achieved 100% line coverage, ensuring no parallel logic escaped verification.

We additionally compiled and executed the full corpus under ThreadSanitizer (TSan). TSan flagged approximately 150 candidate races or runtime errors; manual inspection of 50 flagged cases identified no genuine data races. This is consistent with the structural properties of the code we generate: ParlayLib’s functional, lock-free data-parallel primitives, used in place of explicit thread management, make data races and deadlocks structurally unlikely.

Appendix I Leakage and Near-Duplicate Overlap Analysis

To verify that our reported results reflect genuine generalization rather than memorization of evaluation problems, we analyze the relationship between our training corpora and the benchmarks used for evaluation.

Held-out tasks.

The 700 held-out tasks in Parlay-Instruct corpus function exclusively as a validation set during fine-tuning and are not used in any of the final evaluations (PBBSBench, ParEval, RPB, DMOJ). Overlap between the training and validation splits therefore does not affect reported results.

Evaluation benchmarks.

The evaluation benchmarks are disjoint from our training data. The Parlay-Instruct corpus, synthesized from ParlayLib, is used to fine-tune the C++ models, while a Rust version of DMOJ is used to fine-tune the Rust models. All base and fine-tuned models share identical pretraining data to ensure fair comparison.

N-gram overlap analysis.

We performed an exact $N$ -gram overlap analysis ( $N=10,15$ ) using Jaccard similarity, comparing our fine-tuning corpora against each evaluation suite (Parlay-Instruct against ParEval, DMOJ, and PBBSBench; Rust-DMOJ against RPB). Results are summarized below.

•

ParEval and DMOJ (C++): Negligible overlap ( $<0.00002$ for 10-grams). All matches consisted of standard library dependencies (e.g., #include <algorithm>) and generic variable assignments.
•

PBBSBench (C++): Minimal overlap ( $<0.6\%$ for 10-grams). Manual review confirmed these matches are predominantly C++ header usage and structural parlaylib boilerplate required for compilation.
•

Rust Performance Benchmarks (RPB): Near-zero leakage ( $0.0$ for 15-grams; $0.000004$ for 10-grams). The single overlapping 10-gram is standard Rust comparison boilerplate (fn partial_cmp...).
•

DMOJ fine-tune problems vs. DMOJ test problems: $0.002559$ for 10-grams.

Summary.

The maximum structural overlap across all evaluation suites is below $0.6\%$ , and manual inspection confirms these matches consist of syntactic boilerplate rather than solution content. We find no evidence of evaluation problem solutions appearing in the training data.

Appendix J Semantic Alignment Example

Figure 21 is an example of the semantic alignment achieved by finetuning.

Input Prompt:

⬇

#include <parlay/primitives.h>

#include <complex>

/* Sort vector x of complex numbers by magnitude (ascending) using ParlayLib. */

void sortComplexByMagnitude(parlay::sequence<std::complex<double>>& x) {

Base Model (Gemini-2.5-Pro)

⬇

1 {

2 // Build Error: parlay::sort expects

3 // different comparator signature

4 parlay::sort(x, [](const std::complex<double>& a, const std::complex<double>& b) {

5 return std::abs(a) < std::abs(b);

6 });

7 }

Status: Compilation Failed (Build@1=0)

Fine-tuned Model (ParEVO)

⬇

1 {

2 // Correct: uses sort_inplace and

3 // generic auto params for complex type

4 parlay::sort_inplace(x, [](const auto& a, const auto& b) {

5 return std::abs(a) < std::abs(b);

6 });

7 }

Status: Passed ( $17.5\times$ Speedup)

Figure 21: Semantic Alignment Example. The base model (top) fails to compile due to incorrect API usage and strict type definitions in the lambda. The fine-tuned model (bottom) correctly identifies sort_inplace and uses auto to handle the complex number types safely.