Skip to content

rz-test: assemble/disassemble/lift asm tests in-process#6442

Draft
notxvilka wants to merge 1 commit into
devfrom
rztest-blazingly-fast
Draft

rz-test: assemble/disassemble/lift asm tests in-process#6442
notxvilka wants to merge 1 commit into
devfrom
rztest-blazingly-fast

Conversation

@notxvilka

Copy link
Copy Markdown
Contributor

Your checklist for this pull request

  • I've read the guidelines for contributing to this repository.
  • I made sure to follow the project's coding style.
  • I've documented every RZ_API function and struct this PR changes.
  • I've added tests that prove my changes are effective (required for changes to RZ_API).
  • I've updated the Rizin book with the relevant information (if needed).
  • I've used AI tools to generate fully or partially these code changes and I'm sure the changes are not copyrighted by somebody else.

Detailed description

The asm test suite (db/asm, ~20k instruction lines) was by far the most process-heavy part of rz-test. For every instruction line rz-test spawned up to three separate rz-asm processes -- one to assemble, one to disassemble and one to lift+validate the IL -- so a full run launched on the order of ~30k short-lived rz-asm processes. Each launch dynamically loads the entire librz plugin stack, and that fixed per-process startup cost is what dominates the asm tests, especially under ASAN (large mappings make fork/exec and the loader much more expensive) and on Windows (CreateProcess is intrinsically costly).

Instead of shelling out, drive RzAsm/RzAnalysis directly from the test runner. rz_test_run_asm_test() now creates an RzAsm and an RzAnalysis, configures them from the test's arch/cpu/bits/endianness exactly like the rz-asm CLI does, and:

  • assembles via rz_asm_rasm_assemble() and compares the raw bytes,
  • disassembles via rz_asm_mdisassemble() (trim + '\n'->';' as before),
  • lifts each instruction with RZ_ANALYSIS_OP_MASK_IL and stringifies + validates the IL, mirroring print_and_check_il() in librz/main/rz-asm.c.

The post-processing of the disassembly and IL strings is identical to the old subprocess path, so the recorded EXPECT values keep matching. This links rz-test against rz_arch (RzAsm/RzAnalysis) and rz_il.

A simple stdin-batching scheme (feeding many instructions to one rz-asm) was considered and rejected: rz-asm's file/stdin paths run the whole input through rz_asm_massemble()/stream disassembly, which merges instruction boundaries and cannot recover the per-line results each asm test asserts. Driving the library in-process is the only way to both remove the spawns and keep exact per-instruction results.

Tradeoffs:

  • The per-test timeout (config->timeout_ms) is no longer applied to asm tests. These are bounded computations over a single, already-parsed instruction; a hang would be a real bug, better surfaced than hidden.
  • Crash isolation is reduced: a crashing assembler/lifter now takes down the worker instead of a child process. The BROKEN mechanism still covers known failures, and asm ops over a single instruction are far simpler than the full-binary analysis cmd tests already run.

The RzAsm/RzAnalysis pair is created per test for now; hoisting it to one instance per worker thread (reconfiguring per test) would remove the remaining per-test setup cost and is a natural follow-up.

Test plan

  • CI is green
  • CI times are significantly lower

The asm test suite (db/asm, ~20k instruction lines) was by far the most
process-heavy part of rz-test. For every instruction line rz-test spawned
up to three separate rz-asm processes -- one to assemble, one to
disassemble and one to lift+validate the IL -- so a full run launched on
the order of ~30k short-lived rz-asm processes. Each launch dynamically
loads the entire librz plugin stack, and that fixed per-process startup
cost is what dominates the asm tests, especially under ASAN (large
mappings make fork/exec and the loader much more expensive) and on
Windows (CreateProcess is intrinsically costly).

Instead of shelling out, drive RzAsm/RzAnalysis directly from the test
runner. rz_test_run_asm_test() now creates an RzAsm and an RzAnalysis,
configures them from the test's arch/cpu/bits/endianness exactly like the
rz-asm CLI does, and:

  - assembles via rz_asm_rasm_assemble() and compares the raw bytes,
  - disassembles via rz_asm_mdisassemble() (trim + '\n'->';' as before),
  - lifts each instruction with RZ_ANALYSIS_OP_MASK_IL and stringifies +
    validates the IL, mirroring print_and_check_il() in librz/main/rz-asm.c.

The post-processing of the disassembly and IL strings is identical to the
old subprocess path, so the recorded EXPECT values keep matching. This
links rz-test against rz_arch (RzAsm/RzAnalysis) and rz_il.

A simple stdin-batching scheme (feeding many instructions to one rz-asm)
was considered and rejected: rz-asm's file/stdin paths run the whole input
through rz_asm_massemble()/stream disassembly, which merges instruction
boundaries and cannot recover the per-line results each asm test asserts.
Driving the library in-process is the only way to both remove the spawns
and keep exact per-instruction results.

Tradeoffs:
  - The per-test timeout (config->timeout_ms) is no longer applied to asm
    tests. These are bounded computations over a single, already-parsed
    instruction; a hang would be a real bug, better surfaced than hidden.
  - Crash isolation is reduced: a crashing assembler/lifter now takes down
    the worker instead of a child process. The BROKEN mechanism still
    covers known failures, and asm ops over a single instruction are far
    simpler than the full-binary analysis cmd tests already run.

The RzAsm/RzAnalysis pair is created per test for now; hoisting it to one
instance per worker thread (reconfiguring per test) would remove the
remaining per-test setup cost and is a natural follow-up.
@notxvilka notxvilka added optimization performance A performance problem/enhancement labels May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimization performance A performance problem/enhancement rz-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants