By Steve Miller | Sep 27, 2013
Recently Senior Design Engineer Steve Miller was working on a client project that required taking multiple video inputs and processing them in real-time through an FPGA. Some of the necessary computations involved comparing images in two parallel video streams. Steve wanted to independently verify the FPGAs blocks prior to integration into the final product, but typical FPGA design verification methods using test cases (written in tools such as Aldec) would require writing tedious video stream test inputs and expected outputs for each test case.
In addition, the client wanted a simulation of the entire system to an accuracy that reflected the FPGA output to the least significant bit. Traditional image processing through tools like Matlab process the pixel data in floating point variables and could not guarantee this precision. In addition, the impact of rounding in truncation within the FPGA and special edge condition processing needed to modeled.
To solve both of these requirements, Steve developed a simulation tool called the Computational Model (CM). Written in C++, it allows the user to write equivalent block objects in software and wrap a generic multi-input/output block interface around them. The simulation is platform-independent and can be compiled and executed in either Windows or Linux OSs. The simulation collects groups of blocks (based on a configuration input) and executes them on one or more separate threads on the host computer.
The CM can accept any arbitrary PNG image as an input and automatically provide the configuration of the input image to all connected blocks. The output of the CM provides both the configuration of the block and the resulting input and expected output data streams. This meant that a boilerplate test case could be developed for the verification of the FPGAs. A tester simply had to apply the same block configuration to his test case and make sure that the output matched the expected output. Several test case scenarios could be run from the same test case by providing different CM simulation runs of different test images. The CM could automatically execute images of a couple lines or full 4k images without extra configuration changes between runs.
The simulation allowed multiple threads of groups of blocks to be defined and executed. Using core processing functionality, Steve was able to create a simulation that took one or several of the block functions in the FPGA and provided an expected output based on the applied input. Then the results for any image could be verified against the FPGA and the simulation, and Nuvation could deliver a known-good product to a satisfied client.
Check out the video below to learn more about the Computational Method, or read the transcript at the end of this post.
Nuvation specializes in product development for the video industry, including complex video processing, image sensor interfacing and image processing, and 4K/high definition video designs. Contact Nuvationto learn about how our experienced engineering teams can get your product to market faster.
George Reimer: Today we’re talking to Steve Miller, who’s developed a
simulation program to help us understand and develop FPGA applications in
other areas. So Just going to hand it over to you.
Steve Miller: Thanks George. What I did is, my background, give you a
little background, is I’ve had a lot of experience writing
simulations. I’ve done a master’s in robotics, and I’ve
worked on history of military sensor systems and
simulations, and even have worked on vision systems, like
operating in space.
But the challenge I was facing coming here was that we had this very
complicated project, where we’re taking multiple video
inputs and we were processing them in real time through the
FPGA. There’s some of the computations had to compare the
images on two parallel video streams together. The question
was: How do we verify all of our FPGAs prior to actually
testing on the final product?
Typically you use test cases or test benches to verify functionality.
One of the difficulties with that is that when you write a
test bench, you also have to write the test inputs on what
you want to test. So trying to write a test input, which is
giving you two video streams in, and verify and then also
try and decide what the data content is going to be for
that video was seen as too much of a burden.
So what I came up with was what we call a computation model, but it’s
really just a simulation tool, which was written in C++. It
allows anyone to come in and write data objects. You could
write your own, or you could take third-party libraries or
what have you. You can wrap this block interface around it.
Blocks are just a general term to provide multiple inputs
and multiple outputs, however you wish to map them, and
then sort of a core processing functionality.
So what I was able to do was create a simulation which took all of
our block functions in the FPGA, and I could write a
simulation that could just run one, or I could run a set
that can concatenate one after the other. I can provide it
a video input.
It was designed in such a way so that it could give you a lot of
detail. It could give you a lot of internal state on what
the simulation did. But it also, more importantly, would
provide you an output, an expected output. It would also
generate the input that drove that simulation, and you
could take those two, and we created some simple interfaces
in our FPGA test kit, test benches. It will take that video
stream or multiple video streams and feed them into the
block and test it and then have a comparable output to
So I could literally take any image I wanted, or set of images, put
them through the test bench, and verify the results against
the FPGA and the simulation. When we were going the process
of test bench, I could easily change the data by just
literally going into Matlab, creating all sorts of
different data, data steps or data functions, and write a
PNG out, and feed that PNG to the simulation, get the
outputs, and run those through the test bench.
We actually found some very good and interesting timing issues before
we even had to put this onto the actual. So it was a very
easy way to validate the overall FPGAs.
George Reimer: Well, that’s awesome Steve. Thanks for telling us about
that. We’ll be looking forward to new developments on that
Steve Miller: Thanks.