Back
#270
April 6, 2026

EP270 The Convenience Tax: Why We Keep Failing at Supply Chain Security

Guest:

Topics:

Supply Chain Security
29:29

Subscribe at YouTube

Subscribe at Spotify

Subscribe at Apple Podcasts

Topics covered:

  • We just saw a security tool (Trivy) get used to pop an AI infrastructure tool (LiteLLM) to eventually pop end users. Have we reached the point where our security tooling is actually our largest unmanaged attack surface?
  • Why now? Software supply chain security had the perennial vibe of “not top concern” for most organizations, right?
  • TeamPCP pushed malicious code to existing GitHub tags. We’ve been screaming about pinning versions to SHAs for years, but clearly, nobody is listening. Is it time to admit that 'convenience' is the primary enemy of supply chain security?
  • The Axios incident showed a victim compromised in under two minutes. In a world of auto-updating dependencies, is the concept of a human-in-the-loop for software updates officially dead, or do we need to look very hard at version pinning and such?
  • With XZ Utils case, we saw a long-game social engineering attack. Beyond just 'watching npm closely,' what are the realistic architectural safeguards for an org that knows they can't audit every line of an update?
  • We’ve spent the last three years talking about SBOMs (Software Bill of Materials) like they were a pill for supply chain health. But if the scanner producing the SBOM is the one that's compromised, isn't the SBOM just a signed receipt for your own house being on fire?
  • What is the one practical thing they can do to ensure their CI/CD isn't a credential-exfiltration-as-a-service platform?

Do you have something cool to share? Some questions? Let us know:

Transcript

This session explored the escalating complexity and frequency of software supply chain attacks, colloquially referred to as a "security turducken of Doom." The discussion centered on recent high-profile breaches where security tools and AI infrastructure wrappers were leveraged to compromise end users. Dan Lorenc provided a sobering analysis of why these attacks are becoming the primary unmanaged attack surface for modern enterprises.

The conversation moved beyond the theoretical "Reflections on Trusting Trust" to the practical, painful reality of current exploits. Lorenc argued that the industry's success in hardening traditional vectors (like HTTPS ubiquity and MFA) has forced sophisticated attackers toward the supply chain—a shift from "spear fishing" specific companies to "net fishing" across the ecosystem. Key technical failures in GitHub Actions, the limitations of Software Bills of Materials (SBOMs), and the "messy middle" of dependency management were dissected. The overarching conclusion is that while the industry is moving toward "secure by design" principles, most organizations are currently ill-equipped to handle the "fractal" nature of these compromises.

Detailed Conversation Overview

1. The Anatomy of Modern Supply Chain Exploits

The discussion opened with a review of recent cascading breaches involving security scanners and AI tools. Lorenc noted that attackers are increasingly patient and cyclical. He highlighted a sequence where a breach on a tool like Trivy was not the end goal, but a means to steal credentials from CI/CD systems of popular open-source projects. This allows attackers to exfiltrate terabytes of data, sifting through it for follow-on targets. The "fractal" nature of these attacks means one breach leads to several more, creating an environment where the industry is constantly waiting for the "next drop."

2. The Shift in Attacker ROI

Anton Chuvakin questioned why supply chain security has historically been a secondary priority compared to credential theft or phishing. Lorenc explained this through the lens of a multiplayer game: as defenders harden traditional perimeters, attackers move to the least-guarded entry points. He posited that supply chain attacks change the risk profile for smaller organizations; even if you aren't a "crown jewel" target like Google or Microsoft, you are now at risk simply by being within the "blast radius" of a larger tool's compromise.

3. Technical Debt in CI/CD Infrastructure (The GitHub Actions Critique)

A significant portion of the debate focused on the architectural flaws of GitHub Actions. Lorenc compared it unfavorably to the Go ecosystem, which uses transparency logs and checksums (go.sum) to pin dependencies by default. In contrast, GitHub Actions relies on mutable tags. Even when users "pin by digest," they often face a false sense of security because the underlying action file may point to an unpinned Docker image. Lorenc also detailed "Imposter Commits," where GitHub’s deduplication of git objects across forks allows an attacker to pull a malicious commit from a repo they control into the context of a trusted repo, provided they have the commit hash.

4. The Dependency Management Paradox: To Update or Not to Update?

The hosts and guest explored the tension between "move fast" culture and security stability. Lorenc pushed back against the "wait a week" strategy for updates, noting that attackers can easily program "time-sleep" logic into malware to bypass cooling-off periods. Furthermore, delaying updates leaves organizations vulnerable to known CVEs and zero-days. The consensus was that organizations must reach a state of agility: the ability to update everything at any time. The psychological barrier—dealing with the "small pain" of daily updates versus the "catastrophic pain" of a massive breach—remains the biggest hurdle for most CISOs.

5. The "Social Engineering" of Open Source (XZ Utils Case)

The XZ Utils attack served as a case study for the "long game." Lorenc warned that nation-states don't just take over existing projects; they can create useful new projects, build trust and "GitHub Stars" over years, and only then introduce malicious code. This renders human or robotic line-by-line auditing nearly impossible for the average organization.

6. The "Condom" Problem: The Limitations of SBOMs

Lorenc offered a scathing critique of the current state of SBOMs (Software Bill of Materials). He compared them to a South Park trope: something people carry for protection without understanding how to use them. Current SBOMs are often:

Inaccurate: Generated by the same tools that might be compromised.

Incomplete: Many scanners fail to identify core components (e.g., failing to find WordPress inside a WordPress Docker image).

Static: Often just a "piece of paper" stapled to a pipeline that no one actually reviews. For SBOMs to be useful, they must be generated by the compiler during the build process, not reconstructed after the fact by a third-party scanner.

7. Practical Architectural Safeguards

In lieu of "buying another scanner," Lorenc recommended:

Secrets Minimization: Removing AWS keys and GitHub secrets from CI/CD pipelines in favor of OIDC and short-lived tokens.

Assume Breach in CI/CD: Treating build infrastructure as production systems that are likely already compromised.

Dependency Pinning + Automation: Pinning everything by digest but using automated tools to manage the update lifecycle.

Timeline of Key Topics

The "Turducken of Doom": Analysis of the recent cascading attacks involving AI infrastructure and security tooling.

Supply Chain as the Primary Vector: Discussion on why attackers have shifted from spear-fishing to "net fishing" via the supply chain.

The Legacy of "Trusting Trust": Connecting modern exploits back to Ken Thompson’s foundational 1984 paper on compiler security.

GitHub Actions Vulnerabilities: A deep dive into mutable tags, unpinnable actions, and the "Imposter Commit" flaw.

The Failures of "Cooling-off" Periods: Why waiting a week to update dependencies is an ineffective and dangerous strategy.

The "Long Game" of Social Engineering: How nation-states build trust in open-source projects over years to facilitate future attacks (Reference to XZ Utils).

The SBOM Reality Check: Why current regulatory pushes for SBOMs are failing to provide actual security.

Zero Trust for Developers: The necessity of treating developer laptops and CI systems as "untrusted" by design.

Final Recommendations: The critical balance of pinning dependencies while maintaining the practice of automated updates.

View more episodes