Skip to content

kernel: require a valid marker before skipping a KernelInstall rebuild - #17701

Draft
bboozzoo wants to merge 3 commits into
canonical:masterfrom
bboozzoo:bboozzoo/kernel-tree-assert-ok
Draft

bboozzoo wants to merge 3 commits into
canonical:masterfrom
bboozzoo:bboozzoo/kernel-tree-assert-ok

Conversation

@bboozzoo

@bboozzoo bboozzoo commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

The KernelInstall idempotency guard only checked directory existence,
not completion. Since this path builds directly into destDir (no
scratch dir), an interrupted build or a marker-less tree from an older
snapd would be wrongly treated as already done. Require
DriversTreeNeedsCheck to confirm a current marker before skipping the
rebuild, and stop discarding DirExists/DriversTreeNeedsCheck errors.
Rebuilding in place on a stale/missing marker is safe: destDir is never
live-mounted at either KernelInstall call site.

Cherry picked form #17680

Related: SNAPDENG-37501

…version

Introduce versioning of the organization of the kernel modules &
firmware trees. The patch adds a helper to record the version of the
layout and another helper to check and compare indicating whether the
tree was set up by older version of the code.

Both are essential for being able to determine if the tree needs to be
rebuilt/updated when updating snapd.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
The KernelInstall idempotency guard only checked directory existence,
not completion. Since this path builds directly into destDir (no
scratch dir), an interrupted build or a marker-less tree from an older
snapd would be wrongly treated as already done. Require
DriversTreeNeedsCheck to confirm a current marker before skipping the
rebuild, and stop discarding DirExists/DriversTreeNeedsCheck errors.
Rebuilding in place on a stale/missing marker is safe: destDir is never
live-mounted at either KernelInstall call site.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
@bboozzoo
bboozzoo requested a balanced review from Copilot September 23, 2026 13:09
@bboozzoo bboozzoo added the Run nested The PR also runs tests inluded in nested suite label Sep 23, 2026
@github-actions github-actions Bot added the Run only one system Only runs spread tests on one system label Sep 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

A failed cleanup can leave stale tree content that is subsequently marked current.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Adds completion/version markers to kernel driver trees so interrupted or legacy builds are rebuilt safely.

Changes:

  • Records and validates kernel tree generator metadata.
  • Rebuilds stale or markerless installation trees.
  • Adds unit and spread coverage.
File Description
kernel/​kernel_drivers.go Implements metadata and guarded rebuilding.
kernel/​kernel_drivers_test.go Tests marker and rebuild behavior.
kernel/​export_test.go Exposes test helpers and mocks.
tests/​core/​basic20plus/​task.yaml Verifies metadata on Ubuntu Core.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread kernel/kernel_drivers.go
Comment on lines 455 to 458
if rmErr := RemoveKernelDriversTree(targetDir); rmErr != nil &&
!errors.Is(err, fs.ErrNotExist) {
!errors.Is(rmErr, fs.ErrNotExist) {
logger.Noticef("while removing old kernel tree: %v", rmErr)
}
Comment on lines +1055 to +1057
restore := kernel.MockKernelDriversTreeGeneratorVersion(100)
c.Assert(kernel.WriteDriversTreeMeta(destDir), IsNil)
restore()
@codecov

codecov Bot commented Sep 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.13115% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.39%. Comparing base (2cbd01b) to head (10c3b38).
⚠️ Report is 23 commits behind head on master.

Files with missing lines Patch % Lines
kernel/kernel_drivers.go 72.13% 10 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #17701      +/-   ##
==========================================
+ Coverage   78.35%   78.39%   +0.04%     
==========================================
  Files        1416     1414       -2     
  Lines      200464   200666     +202     
  Branches     2502     2503       +1     
==========================================
+ Hits       157070   157320     +250     
+ Misses      33972    33911      -61     
- Partials     9422     9435      +13     
Flag Coverage Δ
unittests 78.39% <72.13%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

Wed Sep 23 15:34:44 UTC 2026
The following results are from: https://github.com/canonical/snapd/actions/runs/35865185934

Test Predictor Analysis

Preparing

Test Success %
openstack-ext:ubuntu-18.04-64:tests/nested/classic/activate-state 🔴 20.5%
openstack-ext:ubuntu-18.04-64:tests/nested/classic/snapshots-with-core-refresh-revert 🔴 21.2%
openstack-ext:ubuntu-20.04-64:tests/nested/core/interfaces-custom-devices 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core-factory-reset-new-secboot:tpm 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core20-fault-inject-on-remodel:kernel_reboot_remodel_boot_assets 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core20-remodel 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/devmode-snap-seeded-dangerous 🟢 74.1%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/remodel-uc-to-next-version-fakestore:notencrypted 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/core20-set-efi-boot-vars:BOOTDIR_NOSEC 🟡 62.3%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/fde-auth-support-on-hybrid:no_auth 🔴 1.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/hybrid-fde-recovery-keys 🟢 99.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/hybrid-remodel 🟡 67.7%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/kernel-modules-components:encrypted 🟢 76.4%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/recovery-system:tested 🟢 77.1%

Executing

Test Success %
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core20-custom-kernel-commandline 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/remodel-uc-to-next-version-fakestore:hook 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/seed-refresh-component:optional_in_seed_snap_component_exclusive 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/seed-refresh-eviction 🟢 99.0%
openstack-ext:ubuntu-22.04-64:tests/nested/manual/muinstaller-real:encrypted 🟡 68.0%
openstack-ext:ubuntu-22.04-64:tests/nested/manual/muinstaller-real:seeded 🟡 63.7%
openstack-ext:ubuntu-22.04-64:tests/nested/manual/seed-refresh-eviction 🟢 99.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/core20-fde-dbx 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/muinstaller-core:install_optional_snap 🟢 73.7%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/remodel-to-installed-kernel 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/seed-refresh-back-to-revision:refresh_kernel 🟡 59.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/seed-refresh-component:optional_in_seed_snap_component_exclusive 🟢 99.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/uc-grub-boot-chains 🟢 99.0%
openstack-ext:ubuntu-26.04-64:tests/nested/core/core20-factory-reset:fde 🔴 2.7%
openstack-ext:ubuntu-26.04-64:tests/nested/core/core20-gadget-reseal 🔴 3.2%
openstack-ext:ubuntu-26.04-64:tests/nested/manual/core-recover-from-recovery:clear_tpm 🟡 57.2%

Restoring

Test Success %
openstack:ubuntu-24.04-64:tests/main/snapd-reexec:core 🟢 73.7%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/ 10 times 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core-factory-reset-new-secboot:tpm 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core20-custom-kernel-commandline 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core20-fault-inject-on-remodel:kernel_reboot_remodel_boot_assets 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/core20-remodel 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/devmode-snap-seeded-dangerous 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/remodel-uc-to-next-version-fakestore:encrypted 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/remodel-uc-to-next-version-fakestore:hook 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/remodel-uc-to-next-version-fakestore:notencrypted 🟢 90.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/seed-refresh-component:optional_in_seed_snap_component_exclusive 🟢 99.0%
openstack-ext:ubuntu-20.04-64:tests/nested/manual/seed-refresh-eviction 🟢 99.0%
openstack-ext:ubuntu-22.04-64:tests/nested/manual/ 2 times 🟢 77.4%
openstack-ext:ubuntu-22.04-64:tests/nested/manual/muinstaller-real:seeded 🟢 77.1%
openstack-ext:ubuntu-22.04-64:tests/nested/manual/seed-refresh-eviction 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/ 12 times 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/core20-fde-dbx 🟢 73.7%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/core20-set-efi-boot-vars:BOOTDIR_NOSEC 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/fde-auth-support-on-hybrid:no_auth 🟡 59.9%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/hybrid-fde-recovery-keys 🟢 99.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/hybrid-remodel 🟢 77.1%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/kernel-modules-components:encrypted 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/muinstaller-core:install_optional_snap 🟢 90.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/recovery-system:tested 🟢 77.1%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/remodel-to-installed-kernel 🔴 6.1%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/seed-refresh-back-to-revision:refresh_kernel 🟡 67.7%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/seed-refresh-component:optional_in_seed_snap_component_exclusive 🟢 99.0%
openstack-ext:ubuntu-24.04-64:tests/nested/manual/uc-grub-boot-chains 🟢 99.0%

Skipped tests from snapd-testing-skip

If you wish to have any of the below tests run in your PR, in your PR description, add 'unskip:' followed by a copy-and-pasted list of the below tests you wish to run (unskip plus test list must be valid yaml)

  • openstack:ubuntu-24.04-64:tests/main/auto-refresh-retry
  • openstack:ubuntu-24.04-64:tests/main/i18n
  • openstack:ubuntu-24.04-64:tests/main/interfaces-location-control
  • openstack:ubuntu-24.04-64:tests/main/network-retry

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Run nested The PR also runs tests inluded in nested suite Run only one system Only runs spread tests on one system

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants