Codeship5 min readintermediate
CloudBees Smart Tests: How AI selects the right tests to run
Summary
CloudBees Smart Tests uses GenAI-based semantic analysis to select only relevant tests for code changes, reducing CI runtime by 35-70% and providing immediate feedback. It avoids statistical models, requiring no historical data, and also groups failures and deprioritizes flaky tests.
- Uses GenAI semantic analysis to match code changes to relevant tests, not historical pass/fail patterns.
- Requires no training period or pipeline data, enabling immediate test selection from the first run.
- Reduces CI runtime by 35-70% by running only a subset of tests relevant to the change.
- Includes a "confidence curve" to balance test coverage and runtime based on pipeline stage.
Engineering teams struggling with slow CI, high compute costs, and unreliable test signals due to large, slow, or flaky test suites should consider this approach for faster feedback and improved developer experience.
6/10