fix(wc):GNU wc-cpu.sh #9144

mattsu2020 · 2025-11-04T11:40:49Z

Overview

Disable SIMD-accelerated paths when GLIBC_TUNABLES removes AVX/AVX512 so wc falls back to the naive counters.
Add hidden --debug flag output that reports whether hardware acceleration is active, disabled by tunables, or unavailable at runtime.
Cache SIMD policy decisions and reuse them within the fast path code to avoid repeated environment parsing.

Testing

cargo test -p uu_wc
cargo clippy -p uu_wc -- -D warnings
Spot-check wc -l with and without GLIBC_TUNABLES='glibc.cpu.hwcaps=-AVX2,-AVX512F'

Add CPU feature detection to conditionally use SIMD-accelerated bytecount functions for chars and lines, falling back to naive methods when SIMD is disabled by environment or unavailable on CPU. Introduce a hidden --debug flag to output SIMD status information, aiding in troubleshooting performance issues.

…lity Swap the if-else blocks in the wc function's debug output to check for SIMD allowance first, improving code flow without changing functionality.

- Add asimd, ASIMD, hwcaps, tunables, TUNABLES to .vscode/cspell.dictionaries/jargon.wordlist.txt - Prevents spellchecker from flagging valid technical terms used in the codebase

codspeed-hq · 2025-11-04T11:53:33Z

CodSpeed Performance Report

Merging #9144 will not alter performance

_{Comparing mattsu2020:wc_compatibility (1a25344) with main (b2feb82)}

Summary

✅ 126 untouched
⏩ 6 skipped¹

6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2025-11-04T12:01:36Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

src/uu/wc/src/cpu_features.rs

Co-authored-by: Dorian Péron <[email protected]>

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR #9088 (cksum --debug) and PR #9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: #9088, #9144

- Add "hardware" feature to uucore dependency in Cargo.toml - Replace local cpu_features module with uucore::hardware::simd_policy - Update debug output to use enabled_features() method for consistency

Reorder the imports within the `use uucore::{ ... };` block for consistency and better readability. The previous order had `hardware::simd_policy` misplaced; it is now sorted alphabetically (e.g., error, format_usage, hardware, parser, etc.). No functional changes were made.

github-actions · 2025-11-24T12:35:13Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Reorder [dependencies] and [target.'cfg(unix)'.dependencies] sections in src/uu/wc/Cargo.toml to follow alphabetical order, improving file organization and consistency without changing functionality.

github-actions · 2025-11-24T12:56:39Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

RenjiSann · 2025-11-25T13:21:37Z

Can you add a test to avoid regressions in the future ?

Add a new Unix-specific test to ensure that SIMD optimizations in `wc` correctly respect GLIBC_TUNABLES, disabling SIMD paths for AVX2/AVX512 when specified. The test verifies both debug output accuracy and functional equivalence of line count results with and without these tunables, addressing potential correctness issues when hardware acceleration is restricted.

github-actions · 2025-11-25T14:40:09Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Use `content.clone()` for the first two `pipe_in` calls to avoid moving the string, and `content` directly for the third to consume it, ensuring proper Rust ownership handling in the test. This prevents borrow checker errors and allows the test to compile correctly.

sylvestre · 2025-11-25T14:56:56Z

job fails with:
ERROR taplo:format_files: the file is not properly formatted path="/home/runner/work/coreutils/coreutils/src/uu/wc/Cargo.toml"

github-actions · 2025-11-25T15:09:39Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

Reformat the uucore dependency features in src/uu/wc/Cargo.toml from an inline array to a multiline array with proper indentation for improved readability and consistency. No functional changes; this is solely a styling update.

…e allocations Use std::fmt::Write and fold with writeln! to build content string efficiently, reducing memory allocations in test_simd_respects_glibc_tunables for better performance.

github-actions · 2025-12-01T14:44:05Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

sylvestre · 2025-12-01T15:34:57Z

src/uu/wc/src/wc.rs

+        if policy.allows_simd() {
+            let enabled = policy.enabled_features();
+            if enabled.is_empty() {
+                eprintln!("wc: debug: hardware support unavailable on this CPU");


please use the translate! macro

Add debug messages for hardware acceleration in English and French locales, and update Rust code to use translate! macro instead of hardcoded strings. This ensures debug output is language-agnostic and translatable.

github-actions · 2025-12-01T23:43:53Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

mattsu2020 added 3 commits November 4, 2025 19:43

refactor(wc): reorder SIMD debug output conditions for better readabi…

4752e2a

…lity Swap the if-else blocks in the wc function's debug output to check for SIMD allowance first, improving code flow without changing functionality.

chore: add jargon words to spellcheck dictionary

ead1881

- Add asimd, ASIMD, hwcaps, tunables, TUNABLES to .vscode/cspell.dictionaries/jargon.wordlist.txt - Prevents spellchecker from flagging valid technical terms used in the codebase

mattsu2020 changed the title ~~fix(wc):GNUwc-cpu.sh~~ fix(wc):GNU wc-cpu.sh Nov 4, 2025

RenjiSann reviewed Nov 4, 2025

View reviewed changes

src/uu/wc/src/cpu_features.rs Outdated Show resolved Hide resolved

RenjiSann reviewed Nov 4, 2025

View reviewed changes

src/uu/wc/src/cpu_features.rs Outdated Show resolved Hide resolved

RenjiSann mentioned this pull request Nov 4, 2025

Add --debug flag to cksum #9088

Merged

Update src/uu/wc/src/cpu_features.rs

6ea2596

Co-authored-by: Dorian Péron <[email protected]>

naoNao89 mentioned this pull request Nov 14, 2025

feat(uucore): add shared hardware detection module #9279

Merged

mattsu2020 and others added 3 commits November 24, 2025 21:06

Merge branch 'uutils:main' into wc_compatibility

cf4d860

refactor(wc): use uucore's SIMD policy instead of local cpu_features

a7400a6

- Add "hardware" feature to uucore dependency in Cargo.toml - Replace local cpu_features module with uucore::hardware::simd_policy - Update debug output to use enabled_features() method for consistency

refactor(wc): sort dependencies in Cargo.toml alphabetically

b2f7ead

Reorder [dependencies] and [target.'cfg(unix)'.dependencies] sections in src/uu/wc/Cargo.toml to follow alphabetical order, improving file organization and consistency without changing functionality.

mattsu2020 added 2 commits December 1, 2025 23:23

chore(wc): reformat uucore features to multiline array

d1935f3

Reformat the uucore dependency features in src/uu/wc/Cargo.toml from an inline array to a multiline array with proper indentation for improved readability and consistency. No functional changes; this is solely a styling update.

refactor(test_wc): optimize test input generation by avoiding per-lin…

d23fc8b

…e allocations Use std::fmt::Write and fold with writeln! to build content string efficiently, reducing memory allocations in test_simd_respects_glibc_tunables for better performance.

sylvestre reviewed Dec 1, 2025

View reviewed changes

feat(wc): internationalize hardware support debug messages

1a25344

Add debug messages for hardware acceleration in English and French locales, and update Rust code to use translate! macro instead of hardcoded strings. This ensures debug output is language-agnostic and translatable.

Uh oh!

fix(wc):GNU wc-cpu.sh #9144

Are you sure you want to change the base?

fix(wc):GNU wc-cpu.sh #9144

Uh oh!

Conversation

mattsu2020 commented Nov 4, 2025

Overview

Testing

Uh oh!

codspeed-hq bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #9144 will not alter performance

Summary

Footnotes

Uh oh!

github-actions bot commented Nov 4, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 24, 2025

Uh oh!

github-actions bot commented Nov 24, 2025

Uh oh!

RenjiSann commented Nov 25, 2025

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

sylvestre commented Nov 25, 2025

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

sylvestre Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codspeed-hq bot commented Nov 4, 2025 •

edited

Loading