PR Custody

feat: add dependent parameter combination testing to extensive_testing mode

@venkat82checks n/achecks…feat/extensive-testing-dependent-param-combinations → next11 files · +1122 −52updated 2mo ago

▸Description

Why

When extensive_testing is enabled, the agent generates many test cases but doesn't systematically test combinations of dependent parameters (e.g., enum selectors that determine which other fields are required, boolean flags that enable parameter groups, co-required parameter pairs like lat+lon). This leads to gaps in coverage for actions with interdependent parameters.

What

Added mandatory step 5D ("Test Dependent Parameter Combinations") to both test_and_fix_agent and test_and_fix_agent_curl extensive testing prompts
- Identifies dependent parameter patterns (enum selectors, boolean flags, co-required groups, constraining parameters, format/encoding pairs)
- Generates a combination matrix focusing on meaningful combinations
- Executes combinations as additional test cases on top of existing 5C tests
Added Dependent Parameter Combination Check to the step 7 critique in both agents
- Validates the agent identified all dependent parameter groups
- Checks controlling parameter values were tested with appropriate dependents
- Forces the agent to go back if combinations weren't tested

How to Test

Trigger test_and_fix workflow with extensive_testing=true on actions that have dependent parameters (e.g., actions with enum type selectors or boolean flags)
Verify the agent identifies dependent parameter groups and tests their combinations

Pre-Review Checklist

I have self-reviewed this PR

Notes

Only affects the extensive_testing flow — standard testing mode is unchanged
Changes are in prompt text only, no logic changes

🤖 Generated with Claude Code

loading diff…