Architecture for Massively Parallel HDL Simulations
- 1. Architecture for Massively Parallel HDL Simulations Rich Porter Art of Silicon 1
- 2. Art of Silicon ● Founded in 2005 ● Bristol based ● Multimedia centric Silicon IP ● Bespoke IP creation ● Consultancy 2
- 3. It's all about MONEY ● Engineer productivity is key to success – Employment cost is $$$s – Tooling cost – Computers ● Makes good business sense to improve productivity – Less time per chip – More chips per man year ● Productive engineers are happy 3 engineers!
- 4. Coding Time ● Engineers do NOT spend all their time writing code – 5% writing code – 95% debugging code ● Thinking & waiting for waves, regressions ● Highly desirable to quantify the effect of any design delta – As quickly & as easily as possible – For all engineers & management 4
- 5. Minimizing Idle Time ● Scalable test engine – 40+ instances per engineer – Up to as many as they can use ● Large compute farm of dense elements – 8, 12, 16, 24 core boxes ● Reduce iterations during the day ● Provide full regression bandwidth at night, weekends ● So how do I do this? 5
- 6. Verilator ● Used to generate standalone license-free test executable ● Execute as many as your compute infrastructure can support ● Deliver to other teams – Architecture – Systems – Toolchain – Customer 6
- 7. Signoff ● Event driven simulators are gold standard for signoff – Gate level simulations ● HAVE to run in this environment too ● Spending engineer time to create 2 environments costs $s – Gratuitous creation of extra work 7
- 8. Architecture ● Single testbench architecture – Ease of maintanence – Issue replication – Portable to silicon ● It is essential that test stimulus and cycle level behaviour is identical across platforms – Even though one platform is event driven and the other is cycle based 8
- 9. What AoS did ● Stimulus & checker DUV – in C++ checker – Identical code ● 'Gaskets' used model – Provides uniform I/F – interface>signal = value; – value = interface>signal>signed(); ● All design code in regular verilog 9
- 10. Example – verilog initial begin `ifdef verilator $c("aos_simctrl_init(&", rst_cnt, ", &", cycles, DUV ", &", std_simctrl_finish_r, ");" ); `else $aos_simctrl_init( checker rst_cnt, cycles, std_simctrl_finish_r ); `endif model end always @(posedge clk1) `ifdef verilator $c("aos_simctrl_clk();"); `else $aos_simctrl_clk; `endif 10
- 11. Autogeneration ● Single sourced from SPIRIT XML description – Verilog ● module declarations ● grey box instantiation ● verilog side gasket instantiation – C++ ● headers ● port descriptions (name, size, direction) ● PLI wrappers 11
- 12. Test Configuration ● Tests were configured at execution using plusargs – +argument=value ● Scanning code was identical for both – To ensure consistency ● Regression script – Created these configurations – Passed them to batch scheduler – Collated results for web presentation 12
- 13. Logging ● Unified logging interface – Consistent output – Captured in markup to preserve semantics – No cludgy parsing of text post mortem that varies from platform to platform ● Test fail produces identical logs across platforms – No problems with spaces, split lines, mixed streams, output ordering 13
- 14. Triage ● Group messages by content, code, file, line – Placing most numerous first ● Order by cycles within – Increasing cycle count to failure ● Failure at the top is debug candidate ● Web I/F can provide rich set of filters/collation options 14
- 15. Performance ● Performance has steadily increased E5405 @ 2000MHz E5345 @ 2333MHz kHz X5472 @ 3000MHz X3430 @ 2400MHz 0 50 100 150 200 ● Memory footprint remains low – No simulation kernel 15
- 16. Key Features ● Single testbench providing cycle accurate stimulus ● Autogeneration from single source ● Single test configuration ● Unified logging 16
- 17. Conclusion / AoS Experience ● Little spent waiting for simulations – 60 wide queue gave 6MHz effective – Engineers spent time really debugging ● Small amount of time spend bringing up each simulator – Not much, but was very early on ● Verilator proved robust ● See no reason why cannot scale to 1000s of simulations 17
- 18. Recommendations ● Get a compute farm ● Use a queue ● Replace CPUs regularly ● Prioritize high value jobs – $ licenses – interactive jobs ● Evaluate what verilator can do for you ● Profile, record and report 18
- 19. Questions ● Questions Rich Porter [email protected] 19