Skip to content

perf/perf_lock: reduce benchmark workload and remove raw trace dump - #3360

Open
vrbagalkot wants to merge 1 commit into
avocado-framework-tests:masterfrom
vrbagalkot:perf-lock-optimize-workload
Open

vrbagalkot wants to merge 1 commit into
avocado-framework-tests:masterfrom
vrbagalkot:perf-lock-optimize-workload

Conversation

@vrbagalkot

Copy link
Copy Markdown
Contributor

The perf_lock test suite was experiencing severe execution time issues and hangs in automated test environments / CI:

  1. locks_with_entries variant took ~4 hours (~14,704s):

    • The -D (--dump-raw-trace) flag in record_report.yaml instructed perf lock report to dump raw event traces to stdout for millions of events instead of just reporting the top 5 entries.
    • Removed -D from the locks_with_entries variant option (-c -E 5).
  2. script variant hung / timed out (~50+ minutes):

    • The default perf bench sched messaging recording workload generated hundreds of thousands of lock contention tracepoints across 320 processes.
    • Running perf lock script -f -i resolved symbols and printed full callstacks for every single event, resulting in >1,000,000 lines of output that flooded test runners (OpTest/Avocado) and triggered recovery/hangs.
    • Scaled the record workload to perf bench sched messaging -g 1 -l 1 (1 group, 1 loop), which provides sufficient lock events to validate all variants while allowing tests to execute in seconds.

Test Results

Before:

  • locks_with_entries: PASS (14,704.52 s)
  • script/input_data: INTERRUPTED / TIMEOUT (2,970+ s)

After:

 (01/15) perf_lock.py:perf_lock.test_lock;run-subsystem-record-variants-all_cpus-cb98: PASS (6.78 s)
 (02/15) perf_lock.py:perf_lock.test_lock;run-subsystem-report-variants-wait_field-4daf: PASS (2.23 s)
 (03/15) perf_lock.py:perf_lock.test_lock;run-subsystem-report-variants-all_fields-83fe: PASS (2.18 s)
 (04/15) perf_lock.py:perf_lock.test_lock;run-subsystem-report-variants-locks_with_entries-40fa: PASS (2.21 s)
 (05/15) perf_lock.py:perf_lock.test_lock;run-subsystem-report-variants-vmlinux_full-63ad: PASS (2.93 s)
 (06/15) perf_lock.py:perf_lock.test_lock;run-subsystem-info-variants-threads_list-955b: PASS (2.30 s)
 (07/15) perf_lock.py:perf_lock.test_lock;run-subsystem-info-variants-map_instances-46db: PASS (2.18 s)
 (08/15) perf_lock.py:perf_lock.test_lock;run-subsystem-script-variants-input_data-3e1b: PASS (2.50 s)
 (09/15) perf_lock.py:perf_lock.test_lock;run-subsystem-contention-variants-contention_fields-e93e: PASS (2.24 s)
 (10/15) perf_lock.py:perf_lock.test_lock;run-subsystem-contention-variants-all_cpus_entires-48d1: PASS (2.19 s)
 (11/15) perf_lock.py:perf_lock.test_lock;run-subsystem-contention-variants-avg_wait_key-8f59: PASS (2.28 s)
 (12/15) perf_lock.py:perf_lock.test_lock;run-subsystem-contention-variants-trace_PID-fb94: PASS (2.15 s)
 (13/15) perf_lock.py:perf_lock.test_lock;run-subsystem-contention-variants-input_full-bd3e: PASS (2.13 s)
 (14/15) perf_lock.py:perf_lock.test_lock;run-subsystem-contention-variants-vmlinux_full-1f9a: PASS (2.13 s)
 (15/15) perf_lock.py:perf_lock.test_lock;run-subsystem-normal_user_test-b26a: PASS (0.52 s)
RESULTS    : PASS 15 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

The test cases in perf_lock were taking excessively long in CI:
- locks_with_entries took ~4 hours because '-D' dumped full raw traces
  to stdout for millions of events instead of limiting to the top 5 entries.
- script hung / timed out because default 'perf bench sched messaging'
  recorded hundreds of thousands of events across 320 processes, producing
  over a million lines of callstack traces in 'perf lock script' and flooding
  the test logger.

Fix both issues by:
1. Scaling down 'perf bench sched messaging' to '-g 1 -l 1' so the
   trace size remains manageable while providing sufficient lock samples.
2. Removing '-D' from the locks_with_entries variant in record_report.yaml.

Signed-off-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant