Tags: distributed-system-analysis/pbench
Tags
Fix deadlock and DELETE/UPDATE unindexed datasets (#3620) * Fix deadlock and DELETE/UPDATE unindexed datasets PBENCH-1328 A bug revealed another bug: 1. `POST`/`DELETE` `/datasets/{id}` fails when the dataset isn't indexed; with the ability to disable server indexing entirely and to remove dataset indexed data, this is undesirable. As long as the dataset doesn't have a `WORKING` ops status, we should be able to update or delete. 2. When the operation fails, the status is already set to `WORKING`, and this was not corrected on exit, leaving the dataset locked perpetually.
Increase the TM SIGNAL_RESPONSE_TIMEOUT the 100s timeout is too short for some systems (reported on slightly outdated aarch64). Signed-off-by: Lukáš Doktor <[email protected]>
Ops fixes (#3616) * Ops fixes Correct accounting and reporting of cache unpacking statistics. Avoid reflecting SQLAlchemy `init_db` errors into conmon. I realized that even though I'd shortened the traceback, it would still be displayed by Python on termination with an unhandled exception, and that would still be relayed through conmon one line at a time. To avoid that, I added an `except Exception` clause to print a simple message and exit. In the process, I realized that while I'd written a new Click-based `pbench-reindex`, the old `/opt/pbench-server/bin/pbench-reindex` script was still there, and referenced in several config files. I removed these.
Add upload statistics to report (#3615) * Add upload statistics to report Present dataset statistics in various buckets (by create or upload date), including "this year", "this month", "this week", "today" along with per year/month/day-of-month/hour just for fun.
A "re-imagining" of the old `pbench-reindex` (#3614) * A "re-imagining" of the old `pbench-reindex` This supports deleting a datasets's indexed data and triggering re-indexing. It also provides a mechanism to globally enable or disable indexing. In the short term, this provides a workaround for our sharding issues, which are preventing us from generating useful indexing anyway, by turning it off entirely for new datasets. We can also delete indexed data (and the SQL index maps) for existing datasets and, on demand, trigger re-indexing for selected datasets.
v0.72.0: Fix handling of labeled hosts by pbench-postprocess-tools (#… …3456) * v0.72: Fix handling of labeled hosts by pbench-postprocess-tools Fixes #3454 pbench-postprocess-tools mishandles hosts with labels (added by tool registration commands): it ignores the label and complains that it cannot find the tool output directory. The tool output directory path contains '<label>:<host>' as one element in the path but pbench-postprocess-tools looks for a '<host>' element. pbench-postprocess-tools parses the output of pbench-list-tools to get the tool info it needs, but pbench-list-tools omits the label from its output. This PR fixes pbench-list-tools to add the label to its output and pbench-postprocess-tools to parse that output, derive the label and use it to construct the path of the tool output directory. It also adds a "functional" test for pbench-list-tools to verify the output when labeled hosts have registered tools and fixes an incorrect legacy test (test-07) for pbench-postprocess-tools: the test was using a labeled host, but it was testing the wrong tool output directory (without the label). It adds a similar legacy test (test-07.1) to test the unlabeled host case. Bump the version to 0.72.1 PBENCH-1178
Add a constraints.txt file to limit python-daemon to versions < 3.0.0 (… …#3448) Add it to the agent Makefile and to the pip invocation in the agent spec file. Tested by installing the resulting RPM and running an end-to-end smoke test (register tools, run pbench-user-benchmark, push results) on RHEL7.9, RHEL8.7, RHEL8.8, RHEL9.2, CentOS Stream 8, CentOS Stream 9, Fedora 37 and Fedora 38.
PreviousNext