I've been building an agent that debugs long-running verification jobs. The idea is unglamorous: a regression runs for hours, things fail overnight, and the first stretch of the next morning goes into working out which of the failures are actually the same failure. That part is mechanical. It should not need a person.
To build it I needed a simulator the agent could actually drive. Not a GUI I automate by pretending to be a mouse, but something I can invoke headless, script, inspect, and rebuild when it does the wrong thing. Commercial simulators are none of those. They are excellent, and they are closed, licence-counted, and not mine to fix.
So I went looking for something open and found xezim, a SystemVerilog simulator written in Rust by Bondan Rufen. It parses IEEE 1800, runs real Accellera UVM, and has a readable codebase. That was promising. It was also, at that point, a claim I hadn't tested.
A simulator you haven't loaded is just a README
You learn nothing about a simulator by running its examples. The examples pass. That is what examples are for.
What tells you something is a testbench with real structure in it: a factory, TLM connections, virtual interfaces through uvm_config_db, parameterised types, generate loops, packed structs. All the ordinary machinery that a production UVM environment leans on and that a young simulator has probably not had pushed through it yet.
I didn't have one lying around that I could publish, so I built one.
The bench
FlooNOC-verif is a ground-up UVM environment around PULP's FlooNoC floo_router, pinned at v0.8.4 and never edited. tb_top generates clock and reset, instantiates one router (5 ports, XY routing at coordinate (1,1), one virtual channel, generic 64-bit flit built from FlooNoC's own typedef.svh macros) behind a purpose-built interface, and publishes the virtual interface through uvm_config_db before run_test().
A single active agent drives flit items into the input ports and randomly backpressures the outputs. The monitor publishes every accepted and delivered handshake through analysis ports to a scoreboard and to a covergroup subscriber. The scoreboard checks four properties:
- routing follows the documented XY rule
- payload integrity end to end
- no loss and no duplication
- per input to output ordering
25 flits per run, 10 directed and 15 constrained-random, all constrained to routes that are legal under the router's NoLoopback and XYRouteOpt defaults. Coverage is source, output, backpressure, and a source by output cross. Interface-level SVA mirrors the DUT's own valid-stability contracts.
Now the part I want to say before anyone else says it: this is one router. One instance, single-flit packets, one VC, no illegal-destination testing, no AXI chimneys. It is a feasibility milestone. It proves an RTL, toolchain and methodology stack end to end on the smallest DUT that means anything. It does not verify the NoC, and the README says so up front.
Three tests that are supposed to fail
make run-negative runs three fault-injection self-tests, and all three must fail with the expected diagnostic. If any of them passes, the build is broken.
This is the part I would want to see if I were reading someone else's bench. A testbench that reports PASS tells you almost nothing on its own. It tells you the checkers ran and didn't object, which is also what a bench with a dead scoreboard does, and a bench with a covergroup that never samples, and a bench whose monitor is watching a signal that stopped toggling in week two. Vacuous pass is the default failure mode of verification, not an exotic one.
So each negative test injects a fault the checkers are supposed to catch, and the run only counts as green if the expected diagnostic comes out. They were cheap to write, and they are the only reason I believe the other 25 flits.
What broke underneath
Bringing the bench up surfaced real bugs in xezim. Not gaps in exotic corners of the LRM, but things any UVM environment hits on the way to its first run_phase. Six of my fixes are merged upstream:
| PR | What it was | Shipped in |
|---|---|---|
| #48 | genfor localparam substitution and type-param default resolution | 0.9.7 |
| #49 | typedef alias not resolving in type_id::create | 0.9.8 |
| #50 | virtual-interface write forms not propagating to the bound instance | 0.9.8 |
| #52 | packed-struct assignment patterns and packed-2D element continuous assign | 0.9.8 |
| #53 | TLM connect/write intercept for statement-position method calls | 0.9.8 |
| #114 | instance_assoc_member resolving through the lexical stack instead of the runtime class | 0.10.1 |
Two of them sit inside the UVM library's own machinery rather than the language. #49 is factory behaviour: the type lookup in type_id::create when the type was registered through a typedef alias. #53 is the TLM path: connect and write calls in statement position were not being intercepted, which is not a corner case in a bench where every monitor publishes through an analysis port. Neither is an exotic scheduling question. Both are the kind of thing you find only by putting a full environment on top and watching it not work.
I also filed two issues rather than patching around them locally, which is the more useful thing to do when the problem is somebody's build rather than their semantics.
Two suspects, or three
Here is the thing I actually want to say.
When a test fails, verification engineers run a two-suspect investigation. Either the bench is wrong or the RTL is wrong. Everything in our training, our tooling and our review culture assumes those are the only two options, because the third one is a binary we did not compile and cannot read. The simulator is an axiom.
On an open stack it stops being an axiom. My scoreboard reported a mismatch and I had to genuinely hold three hypotheses: my checker, the router, or the thing executing both of them. Six times the answer was the third one.
That changes the debug method more than I expected. You stop trusting the printout as ground truth and start asking what the printout is evidence of. You write the smallest SystemVerilog file that reproduces the behaviour, because that file is now either a bug report or a compliance test. You read a Rust backtrace and a UVM report in the same sitting. It is slower per bug and it is much better for understanding, and it is the reason I now think an agent that debugs verification jobs has to be able to see below the simulator line, not just above it.
Where this is going
The agent is not finished, and I'm not going to claim results I don't have. What I have is the substrate: a simulator I can drive headless and modify, and a bench small enough to be a fast, honest test case for anything I automate on top of it. The next step is the boring one, which is teaching it to reduce a failing regression to a minimal reproducer without me.
Everything above is public and pinned. make setup fetches FlooNoC at a fixed revision, Bender.lock is tracked, make lint runs with zero unwaived warnings and every waiver is in a register, and make doctor prints the tool and pin status so you can tell where your environment differs from mine. Verilator 5.050 exactly, Accellera UVM 1800.2-2017.
If you run it and it does something different on your machine, I'd like to know.