Python Regression Automation GUI Tool
Python GUI tool that cut regression setup time by 70% and boosted coverage by 30%. See how PyQt automation transformed our UVM verification workflow.
The Challenge: Manual Regression Configuration Nightmare
In any verification team, managing regression runs efficiently is crucial for maintaining productivity and achieving coverage goals. In our SoC verification environment at Microchip, we faced a significant challenge with regression configuration that was costing the team valuable time and resources.
The Three-File Problem
Our regression setup relied on three separate plaintext configuration files:
- Include file — Listed all test names
- Runs file — Specified test names with the number of runs and RTL/GLS selection
- Ignore file — Override mechanism to exclude specific tests from runs
The manual process of keeping these files aligned was error-prone and time-consuming. Engineers had to edit multiple files for any test configuration change, ensure perfect synchronization between files, and hope that misalignments wouldn't cause silent failures.
The worst part? Misconfigurations often resulted in failed overnight regressions, discovered only the next morning — losing an entire night of coverage collection and simulation time.
The Solution: PyQt-Based Regression GUI
I developed a Python GUI tool using PyQt that transformed our regression setup process from a manual, error-prone task into an intuitive, visual experience.
Why It Worked
Automatic Environment Detection. The tool reads project environment variables set by our standard initialization scripts, making it project-agnostic and user-independent. It automatically adapts to any project structure and is duplicated automatically for every project.
Intelligent Test Discovery. The tool scans test directories using pattern matching, identifies test files based on naming conventions, and filters out non-test files and directories.
Visual Test Configuration. The GUI presents tests as interactive "cubes" in a grid layout. Each cube displays the test name, a number box for run count configuration, an RTL/GLS checkbox for simulation mode selection, and visual indicators for current configuration state.
File Synchronization Engine. The core parsing engine uses regular expressions combined with a custom parser to read the three configuration files, parse their current state, generate properly formatted output using f-strings, and ensure perfect alignment between all files.
State Management. Save overwrites configuration files directly. Save As creates named configuration sets. Load imports previous configurations through file dialogs. User configuration state is auto-saved per-user.
Safety Features. The tool prompts on unsaved changes when closing, validates for zero-run tests (a test marked to run but with 0 runs), shows error dialogs for environment variable or path issues, and handles file access exceptions gracefully.
Maintenance Tools. A "Clean" button removes duplicate files created by the simulator during test runs, keeping the test directories organized.
File Parsing Strategy
The tool uses a combination of regular expressions and custom parsing logic to handle the plaintext configuration files. The f-string formatting ensures that the output maintains the exact format expected by our regression infrastructure.
Results and Impact
Quantitative Improvements
- 70% reduction in regression setup time — what previously took 10–15 minutes of careful file editing now takes 2–3 minutes of visual configuration
- 30% improvement in coverage closure — engineers could now confidently run more comprehensive test combinations
- Zero overnight regression failures due to configuration errors
Qualitative Benefits
Engineers could start regressions and go home, knowing the configuration was correct. New team members could configure regressions immediately without needing to understand the file formats. Less time debugging configuration issues meant more time for actual verification work.
Future Enhancements
Several enhancements were planned or implemented based on team feedback:
Version 2.0 Features included plusarg management (enable/disable specific plusargs per test by modifying makefiles), test categorization by functionality or block ownership, user tagging so engineers can filter tests they own or are currently debugging, and auto-documentation that extracts and displays test descriptions from .sv file comments.
Proposed Features included test dependency visualization (showing agents and blocks active for each test) and coverage trend integration showing the last coverage a single test reached.
Lessons Learned
Good UI design matters — the visual representation of tests as "cubes" made the configuration immediately intuitive. Reading environment variables made the tool portable across projects without any per-project setup. And early adoption by team members drove the most useful feature development, shaping the tool into something the whole team actually wanted to use.