Skip to content

Mbaluda test perf#1150

Draft
mbaluda wants to merge 2 commits into
mbaluda-nextfrom
mbaluda-test-perf
Draft

Mbaluda test perf#1150
mbaluda wants to merge 2 commits into
mbaluda-nextfrom
mbaluda-test-perf

Conversation

@mbaluda

@mbaluda mbaluda commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Description

please enter the description of your change here

Change request type

  • Release or process automation (GitHub workflows, internal scripts)
  • Internal documentation
  • External documentation
  • Query files (.ql, .qll, .qls or unit tests)
  • External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

  • No rules added
  • Queries have been added for the following rules:
    • rule number here
  • Queries have been modified for the following rules:
    • rule number here

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

  • The structure or layout of the release artifacts.
  • The evaluation performance (memory, execution time) of an existing query.
  • The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

  • Yes
  • No

🚨🚨🚨
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

  • Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

  • Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Reviewer

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

GitHub Advanced Security started work on behalf of mbaluda June 26, 2026 19:36 View session
GitHub Advanced Security finished work on behalf of mbaluda June 26, 2026 19:37
GitHub Advanced Security started work on behalf of mbaluda June 26, 2026 19:39 View session
@mbaluda mbaluda changed the base branch from main to mbaluda-next June 26, 2026 19:40
GitHub Advanced Security finished work on behalf of mbaluda June 26, 2026 19:40
@codeql-coding-standards-automation

Copy link
Copy Markdown

🏁 Beep Boop! Performance testing complete! See below for performance of the last 3 runs vs your PR. Times are based on predicate performance. You can find full graphs and stats in the PR that was created for this test in the release engineering repo.


Release                            : v2.60.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3326599
Mean_Predicate_Execution_Time_Ms   : 64.96502363004335
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 512.262910554225
Total_Serialized_Execution_Time_s  : 3326.599
Mean_Query_Execution_Time_s        : 0.0649650236300433
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 185.0
Number_of_Predicates               : 51206

Release                            : v2.60.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4817689
Mean_Predicate_Execution_Time_Ms   : 67.30966119455117
Median_Predicate_Execution_Time_Ms : 1.0
Standard_Deviation_Ms              : 458.1131446697552
Total_Serialized_Execution_Time_s  : 4817.689
Mean_Query_Execution_Time_s        : 0.0673096611945511
Median_Predicate_Execution_Time_s  : 0.001
Percentile95_Ms                    : 227.0
Number_of_Predicates               : 71575

Release                            : v2.60.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2651790
Mean_Predicate_Execution_Time_Ms   : 51.91241533221096
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 542.373048121457
Total_Serialized_Execution_Time_s  : 2651.79
Mean_Query_Execution_Time_s        : 0.0519124153322109
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 139.0
Number_of_Predicates               : 51082

Release                            : v2.60.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4440651
Mean_Predicate_Execution_Time_Ms   : 62.29170407361688
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 569.4574578020323
Total_Serialized_Execution_Time_s  : 4440.651
Mean_Query_Execution_Time_s        : 0.0622917040736168
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 159.0
Number_of_Predicates               : 71288

Release                            : v2.61.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3266370
Mean_Predicate_Execution_Time_Ms   : 63.90362718628947
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 503.1579843913974
Total_Serialized_Execution_Time_s  : 3266.37
Mean_Query_Execution_Time_s        : 0.0639036271862894
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 191.0
Number_of_Predicates               : 51114

Release                            : v2.61.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4756186
Mean_Predicate_Execution_Time_Ms   : 66.62911337433282
Median_Predicate_Execution_Time_Ms : 1.0
Standard_Deviation_Ms              : 451.0173047967091
Total_Serialized_Execution_Time_s  : 4756.186
Mean_Query_Execution_Time_s        : 0.0666291133743328
Median_Predicate_Execution_Time_s  : 0.001
Percentile95_Ms                    : 208.0
Number_of_Predicates               : 71383

Release                            : v2.61.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2784651
Mean_Predicate_Execution_Time_Ms   : 54.388777124553215
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 610.0917903271135
Total_Serialized_Execution_Time_s  : 2784.651
Mean_Query_Execution_Time_s        : 0.0543887771245532
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 142.0
Number_of_Predicates               : 51199

Release                            : v2.61.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4615626
Mean_Predicate_Execution_Time_Ms   : 64.84988900441172
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 574.4148904892992
Total_Serialized_Execution_Time_s  : 4615.626
Mean_Query_Execution_Time_s        : 0.0648498890044117
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 165.0
Number_of_Predicates               : 71174

Release                            : 1150
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 5484082
Mean_Predicate_Execution_Time_Ms   : 72.34173174335163
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 758.599656524416
Total_Serialized_Execution_Time_s  : 5484.082
Mean_Query_Execution_Time_s        : 0.0723417317433516
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 171.0
Number_of_Predicates               : 75808

Release                            : 1150
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2932578
Mean_Predicate_Execution_Time_Ms   : 53.62575430640384
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 632.3847601149118
Total_Serialized_Execution_Time_s  : 2932.578
Mean_Query_Execution_Time_s        : 0.0536257543064038
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 137.0
Number_of_Predicates               : 54686

🏁 Below are the slowest predicates for the last 2 releases vs this PR.


Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 33098

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : c
Suite             : cert-default
Predicate         : IncompatibleFunctionDeclaration::interestedInFunctions/4#95575433
Execution_Time_Ms : 28367

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#6de8614f#cpe#1236
Execution_Time_Ms : 30727

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Class::Class.getALinkTarget/0#dispred#29b2b38a#bf_Class::Class.getALinkTarget/0#dispred#29b2b38a#bf__#shared
Execution_Time_Ms : 27762

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 30647

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTable/5#79217c12
Execution_Time_Ms : 38785

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : _Class::Class.getALinkTarget/0#dispred#29b2b38a#bf_Class::Class.getALinkTarget/0#dispred#29b2b38a#bf__#shared
Execution_Time_Ms : 23350

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : _Macro::Macro.getName/0#dispred#e28b3699_Preprocessor::PreprocessorBranchDirective#bcd2bde4#b_Prepro__#antijoin_rhs
Execution_Time_Ms : 32365

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#6de8614f#cpe#1236
Execution_Time_Ms : 43006

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Macro::Macro.getName/0#dispred#e28b3699_Preprocessor::PreprocessorBranchDirective#bcd2bde4#b_Prepro__#antijoin_rhs
Execution_Time_Ms : 49239

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 31182

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : cpp
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 38755

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : CheckedException::CheckedException#b0aa5ec8
Execution_Time_Ms : 28801

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : CharacterOutsideTheLanguageStandardBasicSourceCharacterSetUsedInTheSourceCode::getUniversalCharacterName/1#36dbaa42
Execution_Time_Ms : 29041

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : VirtualDispatchPrototype::VirtualDispatch::cannotInheritHelper/4#7c75bd87
Execution_Time_Ms : 27715

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-linux
Language          : cpp
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 30704

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency::dependsOnTransitive/2#cbda84a0
Execution_Time_Ms : 18772

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : cpp
Suite             : misra-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 17638

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : FunctionEquivalence::typeSig/1#194ac728
Execution_Time_Ms : 25220

Release           : v2.61.0
Run               : 2026-06-16_14-01-24
Platform          : x86-windows
Language          : cpp
Suite             : misra-default
Predicate         : IncompatibleFunctionDeclaration::interestedInFunctions/4#95575433
Execution_Time_Ms : 20639

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Class::Class.getALinkTarget/0#dispred#29b2b38a#bf_Class::Class.getALinkTarget/0#dispred#29b2b38a#bf__#shared
Execution_Time_Ms : 33076

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 75472

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : HoldsForAllCopies::HoldsForAllCopies<DeadCode::DeadOperationInstance,Expr::Expr>::RelevantElement#acaf8a9e
Execution_Time_Ms : 33113

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Macro::Macro.getName/0#dispred#e28b3699_Preprocessor::PreprocessorBranchDirective#bcd2bde4#b_Prepro__#antijoin_rhs
Execution_Time_Ms : 32149

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : CharacterOutsideTheLanguageStandardBasicSourceCharacterSetUsedInTheSourceCode::getUniversalCharacterName/1#36dbaa42
Execution_Time_Ms : 34806

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : cpp
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 88746

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#6de8614f#cpe#1236
Execution_Time_Ms : 35920

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : _Call::Call.getArgument/1#dispred#ada436ba_Call::Call.getArgument/1#dispred#ada436ba_120#join_rhs_El__#shared
Execution_Time_Ms : 32139

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : FunctionEquivalence::typeSig/1#194ac728
Execution_Time_Ms : 32047

Release           : 1150
Run               : 2026-06-26_19-37-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : SmartPointers::localExprFlow/2#e8a2b02b
Execution_Time_Ms : 50163

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant