Modular and Adaptive Conformal Prediction for Sequential Models via Residual Decomposition

Modern machine learning systems increasingly rely on modular pipelines, where upstream predictors generate intermediate representations for downstream models. Such pipelines appear across diverse applications: macroeconomic forecasting, where supply chain indicators inform market predictions, and medical diagnosis, where imaging features guide treatment decisions [11, 25]. In these settings, uncertainty quantification is essential and should reflect how error propagates across stages. Conformal prediction [28] provides principled prediction intervals with finite-sample coverage guarantees under exchangeability. However, existing conformal prediction methods treat these multi-stage systems as monolithic black boxes, overlooking their modular structure and missing opportunities for targeted error attribution.

Recent work expands the flexibility of conformal prediction through reweighting and adaptive calibration [7, 1, 20, 12, 4, 14, 15], structured model-aware adaptations [31, 32, 29], and risk-control [8, 2], but largely focuses on single-stage or black-box models, rather than taking a modular perspective.

This perspective is especially pertinent under distribution shift, which often affects stages asymmetrically—e.g., upstream sensors may drift while downstream mappings remain stable. Standard conformal methods cannot disentangle such effects, which may force practitioners to retrain the entire pipeline when targeted interventions would suffice. To address this gap, we propose a stage-wise abstraction that isolates these effects, embedding stage-wise uncertainty into conformal prediction, yielding robust, interpretable intervals under distribution shift.

By decomposing residuals into stage-wise components, our method constructs prediction intervals through a linear combination of stage-specific quantiles, weighted by scaling parameters selected via family-wise error rate (FWER) control over a calibration set. This yields valid, interpretable intervals without requiring internal model access—only structural knowledge of the pipeline. Our approach offers two key advantages over existing conformal methods: (i) it provides diagnostic transparency, identifying which stages contribute most to uncertainty, and (ii) it maintains coverage under distribution shifts affecting individual components where standard approaches degrade.

For intuition, we briefly introduce the two-stage setting with triplets $(w,x,y)$ and latent structure $x=\mu_{1}(w)+\varepsilon_{1}$, $y=\mu_{2}(x)+\varepsilon_{2}$, where $\varepsilon_{1},\varepsilon_{2}$ denote additive noise, and the model learns estimators $\hat{\mu}_{1}$, $\hat{\mu}_{2}$. For example, in automobile supply chains, $w$ could denote semiconductor prices, with $\hat{\mu}_{1}(w)$ estimating new vehicle demand ($x$), and $\hat{\mu}_{2}(x)$ predicting used vehicle prices ($y$). Our method decomposes the residual $R=|y-\hat{\mu}_{2}(\hat{\mu}_{1}(w))|$ into upstream ($\hat{\mu}_{1}$) error component $\Delta R_{1}$ and downstream ($\hat{\mu}_{2}$) residual $R_{2}$, capturing the uncertainty of each stage.

We also extend our framework to adaptive settings for which we update scaling parameters based on component-wise empirical coverage, improving responsiveness to (i) upstream, (ii) downstream, and (iii) end-to-end distribution shifts. We illustrate our method on synthetic shifts and real-world supply chain and financial data, showing improved robustness and interpretability over adaptive conformal baselines [14, 15, 1, 4]. For simplicity, we focus on two-stage models, but our methodology can be extended to multi-stage models (Appendix C).

We consider a sequential two-stage prediction problem. Each data point is characterized by a triplet $z=(w,x,y)$ where $w\in\mathcal{W}$ is the input to the first-stage model (upstream features), $x\in\mathcal{X}$ is an intermediate representation, and $y\in\mathcal{Y}$ is the final prediction target. We learn predictors $\hat{\mu}_{1}:\mathcal{W}\to\mathcal{X}$ and $\hat{\mu}_{2}:\mathcal{X}\to\mathcal{Y}$ which are composed to form end-to-end predictor $\hat{\mu}_{2}(\hat{\mu}_{1}(w))=\hat{\mu}_{2}(\hat{x})$, where $\hat{x}=\hat{\mu}_{1}(w)$.

To learn these models and perform conformal prediction, we assume access to three disjoint subsets: (i) a training set $S^{\text{train}}=\{(w_{i},x_{i},y_{i})\}_{i=1}^{n}$ used to fit both stages of the model via $(w,x)$ and $(x,y)$ pairs for each stage respectively; (ii) a conformal set $S^{\text{conf}}=\{(w_{i},x_{i},y_{i})\}_{i=n+1}^{n+m}$ for computing nonconformity scores; and (iii) a calibration set $S^{\text{cal}}=\{(w_{i},x_{i},y_{i})\}_{i=n+m+1}^{n+m+l}$ for parameter selection. At prediction time, only the upstream input $w_{\text{test}}$ is observed, while the intermediate value $x_{\text{test}}$ and target $y_{\text{test}}$ are unobserved and must be predicted.

We list some assumptions. Exchangeability. Unless stated otherwise, we assume the data $(w,x,y)$ in $S^{\text{train}},S^{\text{conf}},S^{\text{cal}}$ to be exchangeable, as well as the test point $z_{\text{test}}=(w_{\text{test}},x_{\text{test}},y_{\text{test}})$. Learning algorithms. We assume that the algorithms that learn $\hat{\mu}_{1},\hat{\mu}_{2}$ are deterministic, i.e. given the same training data, they produce identical predictors; and we also assume they are symmetric, i.e. invariant to permutations of the data. We also assume that the intermediate variable $x$ is observable for the given datasets, enabling residual decomposition. Distribution shifts. Distribution shifts violate exchangeability and can impact each prediction stage differently. Let $P$ denote the distribution of a data point in the training set and $P^{\prime}$ denote the distribution of the test point. We consider three types of shift: Upstream covariate shift: $P(w)\neq P^{\prime}(w)$; Upstream concept shift: $P(x|w)\neq P^{\prime}(x|w)$; and Downstream concept shift: $P(y|x)\neq P^{\prime}(y|x)$.

To address the aforementioned challenge of attribution, we partition the total prediction residual $R(w,y)=|y-\hat{\mu}_{2}(\hat{\mu}_{1}(w))|$ into upstream and downstream components. This decomposition enables stage-wise attribution of error, in contrast to standard black-box conformal methods. We provide a visualization in Figure˜1.

For notational purposes, we drop the dependence on $(w,x,y)$, and refer to these terms as $R,\Delta R_{1}$, and $R_{2}$. Intuitively, $\Delta R_{1}$ reflects the error of the upstream predictor, while $R_{2}$ isolates downstream error without upstream influence. This decomposition satisfies a fundamental upper-bound property.

By definition, $\Delta R_{1}=||y-\hat{\mu}_{2}(x)|-|y-\hat{\mu}_{2}(\hat{x})||$. Let $A=|y-\hat{\mu}_{2}(x)|=R_{2}$ and $B=|y-\hat{\mu}_{2}(\hat{x})|=R$. By the reverse triangle inequality, $B\leq A+|A-B|=R_{2}+\Delta R_{1}$. This upper bound property ensures that the sum of components provides a conservative estimate of the total error, which is crucial for the coverage guarantees developed subsequently.

This decomposition provides a clean interpretation: $R_{2}$ measures the inherent uncertainty of the downstream model, while $\Delta R_{1}$ measures how upstream prediction errors affect the final prediction error magnitude. When $\Delta R_{1}$ is small relative to $R_{2}$, the upstream predictor performs well and downstream uncertainty dominates. Conversely, when $\Delta R_{1}$ is large relative to $R_{2}$, upstream errors drive prediction uncertainty. This attribution enables practitioners to identify which stage requires improvement and guide retraining decisions. Furthermore, these components provide insights for handling distribution shifts: under upstream shifts, $\Delta R_{1}$ typically increases as the first-stage predictor encounters out-of-distribution inputs, while under downstream shifts, $R_{2}$ increases. These changes can be experimentally visualized in Appendix Figure˜12(b). Thus, the varied responses enable targeted adaptive strategies for different distribution shifts.

These components exhibit intuitive relationships with the total residual: when $R_{2}$ is small, $\Delta R_{1}$ closely approximates $R$, while when the upstream model is accurate and $\hat{\mu}_{2}$ is smooth, $R_{2}$ closely approximates $R$ (see Appendix A.2). Importantly, at least one component must represent a majority of the error, ensuring meaningful stage-wise attribution. Next, we describe two complementary approaches to combining these residual components into prediction intervals.

We describe two approaches that incorporate $\Delta R_{1},R_{2}$, utilizing the conformal set $S^{\text{conf}}$, to compute component-wise quantiles, with data-driven parameter selection using $S^{\text{cal}}$ addressed in Section˜5. Note that there exists a rich space of heuristics for combining these residual components beyond those listed below—see Appendix C.1. For proofs of the following results, see Appendix A.1.

Since finding $(a^{*},b^{*})$ is impossible in practice, we reframe the problem as selecting $(a,b)$ coefficient pairs that satisfy the nominal coverage level $1-\alpha$. We approach this from the perspective of multiple hypothesis testing; for each $(a,b)$ we test the null hypothesis that the miscoverage rate exceeds $\alpha$.

We consider an adaptive variant of our method for nonstationary data by updating the prediction intervals over time. The sets $S^{\text{conf}}$ and $S^{\text{cal}}$ are defined using a sliding window over the most recent observations, with a user-specified window-length $k$. We construct intervals combining Definition˜4 and Definition˜3: $\hat{C}_{\alpha,a,b,c,d}(w)=\hat{\mu}_{2}(\hat{\mu}_{1}(w))\pm(a\,Q_{1-c}(\{\Delta R_{1}\})+b\,Q_{1-d}(\{R_{2}\})),$ parameterized by scaling coefficients $a,b$, quantile levels $c,d$, and target coverage level $\alpha$, where quantiles are taken over $S^{\text{conf}}$, and $\Lambda_{\text{val}}^{t}$ is recalculated at each time step. With each new point, we dynamically update the $(a_{t},b_{t},c_{t},d_{t},\alpha_{t})$ using adaptive rules based on recent performance: when coverage drops below target, we decrease $\alpha_{t}$ and vice versa; when specific components show persistent errors we adjust their corresponding scaling parameters; and when $\Lambda_{\text{val}}^{t}$ is too restrictive we adjust the quantile levels to expand future options. The algorithmic details are provided in Appendix Section˜B.5.

Crucially, the SelectLambda algorithm implements the adaptive adjustments to $(a_{t},b_{t},c_{t},d_{t})$: it first identifies which component constitutes more of the error by comparing recent averages $\bar{\Delta R_{1}}=\frac{1}{k}\sum_{i=t-k}^{t-1}\Delta R_{1}^{(i)}$ and $\bar{R_{2}}=\frac{1}{k}\sum_{i=t-k}^{t-1}R_{2}^{(i)}$ over the sliding window. If coverage fails and upstream errors dominate ($\Delta\bar{R}_{1}>\bar{R}_{2}$), it seeks to increase $a_{t}$ within the validated set $\Lambda_{\text{val}}^{t}$. Conversely, when downstream errors dominate ($\bar{R}_{2}>\Delta\bar{R}_{1}$), it prioritizes $b_{t}$. If the desired scaling adjustment is unavailable in $\Lambda_{\text{val}}^{t}$, the algorithm returns signals $\Delta c_{t},\Delta d_{t}\in\{-1,0,1\}$ to adjust $c_{t+1},d_{t+1}$, effectively “adjusting" the constraints to allow more suitable scaling options.

Even with additional parameters, this approach preserves the long-run coverage guarantee from prior work: $\lim_{T\to\infty}\frac{1}{T}\sum_{t=1}^{T}\text{cov}_{t}\xrightarrow{\text{a.s.}}1-\alpha$, where $\text{cov}_{t}$ denotes coverage at time $t$. This is because the core $\alpha_{t}$ update rule remains identical to the adaptive conformal method of Gibbs and Candes [14], resulting in the convergence guarantee formally stated in Appendix A.1.1.

We evaluate our method on synthetic and real-world forecasting tasks to assess its ability to (i) maintain coverage, (ii) attribute predictive error to specific model stages, and (iii) adapt to distribution shifts in modular pipelines. Our experiments are designed to isolate upstream, downstream, and full-pipeline shifts, highlighting how stage-aware intervals improve transparency and robustness over existing conformal baselines. Additional experiments and details are in Appendix D.

We proposed a conformal prediction framework for sequential models that decomposes prediction residuals into interpretable stage-specific components. Our method combines this decomposition with risk-controlled parameter selection to construct prediction intervals that provide both coverage guarantees and uncertainty attribution. The approach allows practitioners to identify which pipeline stage contributes to prediction uncertainty and provides diagnostic tools for targeted model improvement. Empirical evaluation on synthetic distribution shifts and real-world economic forecasting shows that our method maintains coverage under shifts that degrade standard conformal approaches. While this robustness comes at the cost of wider intervals and occasional abstention when shifts are severe, these trade-offs reflect the method’s conservative design that prioritizes reliable coverage over optimistic predictions. The stage-wise decomposition proves particularly valuable for asymmetric shifts affecting different pipeline components, enabling targeted adaptive responses that standard methods cannot provide.