feature: benchmark cli

[WangzJi]

• I have read the CONTRIBUTING.md guidelines.
• I have registered the PR changes.

Ⅰ. Describe what this PR did

Closes: #7646
Add a new seata-benchmark-cli module under test-suite for stress testing Seata transaction modes.

Features:

• Support for AT, TCC, and SAGA transaction modes
• Dual execution modes:
  • Empty mode (--branches 0): Measures pure Seata protocol overhead without database operations
  • Real mode (--branches N): Executes actual distributed transactions with database operations
• Configurable TPS (Transactions Per Second) rate limiting
• Multi-threaded workload generation with configurable thread pool
• Fault injection with configurable rollback percentage
• Window-based progress reporting (every 10 seconds)
• Performance metrics collection: latency percentiles (P50/P95/P99), success rate, TPS
• CSV export for post-analysis
• Warmup support: exclude initial ramp-up period from final statistics
• YAML configuration file support with priority: CLI args > env var > system property > classpath

Module Structure:

test-suite/seata-benchmark-cli/
├── src/main/java/org/apache/seata/benchmark/
│   ├── BenchmarkApplication.java       # Main entry with picocli CLI
│   ├── BenchmarkConstants.java         # Constants definition
│   ├── config/                         # Configuration classes
│   ├── executor/                       # Transaction executors (AT/TCC/SAGA)
│   ├── model/                          # Data models (metrics, records)
│   ├── monitor/                        # Metrics collection
│   ├── saga/                           # SAGA mode services
│   └── util/                           # Utility classes
└── src/main/resources/
    ├── seata/saga/statelang/           # SAGA state machine definitions
    └── *.conf                          # Seata configuration

Ⅱ. Does this pull request fix one issue?

No, this is a new feature.

Ⅲ. Why don't you add test cases (unit test/integration test)?

This is a benchmark/stress testing tool. The primary validation is manual testing against a running Seata Server. Integration tests would require a full Seata Server setup which is better suited for the existing integration test infrastructure.

Ⅳ. Describe how to verify it

1. Build the module:

   cd test-suite/seata-benchmark-cli
   ../../mvnw clean package

2. Start Seata Server (ensure it's running on 127.0.0.1:8091)

3. Run benchmark (empty mode - no database required):

   java -jar target/seata-benchmark-cli.jar \
     --server 127.0.0.1:8091 \
     --mode AT \
     --tps 100 \
     --duration 60

4. Run benchmark (real mode - requires Docker):

   java -jar target/seata-benchmark-cli.jar \
     --server 127.0.0.1:8091 \
     --mode AT \
     --tps 100 \
     --duration 60 \
     --branches 3

5. Verify the output shows progress every 10 seconds and final report with metrics.

Ⅴ. Special notes for reviews

1. Latency Sampling: To prevent OOM on large-scale tests, latencies are sampled (max 500K samples) - inspired by Kafka ProducerPerformance.

2. Empty vs Real Mode: Empty mode (--branches 0) is useful for measuring pure Seata Server capacity without database overhead.

3. SAGA Implementation: Real SAGA mode uses Seata's state machine engine with predefined state machine definitions in src/main/resources/seata/saga/statelang/.

4. Dependencies: The module uses:

   • picocli for CLI argument parsing
   • snakeyaml for YAML configuration
   • testcontainers for MySQL in real mode
   • Seata's existing infrastructure (TM, RM, Saga engine)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.58%. Comparing base (e2bd89d) to head (3c046e3).
⚠️ Report is 1 commits behind head on 2.x.

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x    #7865      +/-   ##
============================================
- Coverage     71.60%   71.58%   -0.03%     
+ Complexity      884      872      -12     
============================================
  Files          1300     1300              
  Lines         49701    49701              
  Branches       5895     5895              
============================================
- Hits          35589    35576      -13     
- Misses        11192    11210      +18     
+ Partials       2920     2915       -5     

see 6 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This pull request introduces a new seata-benchmark-cli module - a command-line tool for stress testing Seata transaction modes (AT, TCC, and SAGA). The tool supports both "empty mode" (protocol overhead testing) and "real mode" (actual transaction execution) to measure Seata's performance characteristics.

Key Changes:

• CLI-based benchmark tool with picocli framework for argument parsing
• Support for AT, TCC, and SAGA transaction modes with configurable TPS rate limiting
• Dual execution modes: empty transactions for pure protocol overhead testing, and real transactions with database/state machine operations
• Comprehensive metrics collection including latency percentiles (P50/P95/P99), success rates, and TPS measurements with CSV export capability

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
test-suite/seata-benchmark-cli/pom.xml	Maven configuration with dependencies for CLI framework, testcontainers, SAGA engine, and benchmark utilities
test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/BenchmarkApplication.java	Main entry point with CLI argument handling and benchmark orchestration
test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/config/*	Configuration classes for loading/merging benchmark parameters from CLI, YAML, and environment
test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/executor/*	Transaction executors for AT, TCC, and SAGA modes with workload generation
test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/model/*	Data models for metrics collection and transaction records
test-suite/seata-benchmark-cli/src/main/java/org/apache/seata/benchmark/saga/*	SAGA mode service implementations with state machine support
test-suite/seata-benchmark-cli/src/main/resources/*	Configuration files for Seata client and SAGA state machine definitions
test-suite/seata-benchmark-cli/README.md	Comprehensive documentation with usage examples and implementation details
pom.xml	Added new benchmark module to test-suite

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 17 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[funky-eyes]

LGTM