using files for testbench

[Al]

Hi everyone,
I found the need to access a text based file to create my vector
stimuli. Basically I use procedures which will send hexadecimal values
in the correct protocol, but so far I've always added "manually" the
values in the testbench in the following way:

start_simulation;
send_data ("abcd");
send_data ("1234");
send_data ("a0a0");
send_data ("ffff");
send_data ("5aa5");
...
end_simulation;

With this approach if I want to add other commands I need to add new
lines and compile it again.
I thought that using a file to set the value contents will save me from
compiling again every time, is this correct?
In this case the simulation will just stop when file is finished and
there are no more data available, is my assumption correct?
Could anyone post some examples on how this approach can be sketched?
Thanks a lot

Al

 

[KJ]

Hi everyone,
I found the need to access a text based file to create my vector stimuli.
Basically I use procedures which will send hexadecimal values in the
correct protocol, but so far I've always added "manually" the values in
the testbench in the following way:

start_simulation;
send_data ("abcd");
send_data ("1234");
send_data ("a0a0");
send_data ("ffff");
send_data ("5aa5");
..
end_simulation;

With this approach if I want to add other commands I need to add new lines
and compile it again.
I thought that using a file to set the value contents will save me from
compiling again every time, is this correct?

Yes, because now the testbench code consists of a routine to read stuff from
a file until the file is empty, interpretting each line that it reads in.
The code has no dependencies on the contents of that file so you can change
the file at will.

In this case the simulation will just stop when file is finished and there
are no more data available, is my assumption correct?

Not 'automatically'. Your testbench code will read the file until it has
reached the end. Then it will close the file and then stop the clock or
whatever else is needed to finish the simulation.

Could anyone post some examples on how this approach can be sketched?

First look up how to use VHDL file i/o routines to open, close, read and
write files. They're really not that much different than how you would do
it in most other languages.

Try writing something to read and write a simple file. That will give you
the basic feel for how it works and is totally independent of any project
that you're planning on using this for (but at the same time be directly
applicable once you've got the hang of it).

Although you're only asking about reading files, you'll probably quickly
find that writing files is useful too to write out logging information of
various sorts. For example let's say you're writing code that does JPEG
image compression (just as an example). A good testbench would be able to
read in real image files (like .JPG, .TIF, .BMP etc.) and then write out a
..JPG file based on what comes out of the unit you're testing. The list of
files that would be compressed would be read in as input from another file
that had one line which contains the name of each file. Just an example of
ways to use file i/o effectively.

On a different tangent, one other approach to writing the testbench would be
to recognize that the FPGA will not exist in a vacuum. It will be on a
printed circuit board with other parts around it. Instead of writing code
to simulate the interface(s) to the FPGA, consider getting (or writing) the
models for these parts. It will generally give you a more accurate view of
what will really be going on with the board to the extent that you model
those other parts correctly.

I find this approach more useful than simply reading stimulus from a file
since it models the actual system. Also, with the 'read stimulus from a
file' approach that you were asking about, even though you don't need to
recompile anything you do have to restart and re-run the simulation when you
change the file. If you consider that you probably are only recompiling one
file right now as you add/change things then the only time you're saving by
going down this path is the time it takes to recompile one file....a second
or two at best in Modelsim. Yes it adds up....but not very quickly. Not
trying to necessarily discourage that approach since it does have merit,
just consider what time you're actually saving if you do it, and keep in
mind that now this external file is also part of your simulation testbench
and needs to be archived and will probably be in a format useful only to you
but probably poorly documented unless you take the additional time to
document what exactly are the list of commands you'll support in this file
and the proper syntax, etc.....rather like specifying a language....which it
is.

KJ

 

[Ralf Hildebrandt]

I found the need to access a text based file to create my vector
stimuli. ....
I thought that using a file to set the value contents will save me from
compiling again every time, is this correct?

Yes. Textfile-based stimulus generation is quite powerful

What you need is a simulation component that reads your textfile. In
other words: you have to write a parser and invent a syntax for your
textfile, that is easy to parse but powerful enough to provide stimuli
for all test cases.

I use textfile stimuli quite often and the ASNI-C like file I/O routines

In this case the simulation will just stop when file is finished and
there are no more data available, is my assumption correct?

This depends on the behavior of the parser, that you have to write. A
lot of stuff is possible.

Could anyone post some examples on how this approach can be sketched?

For a communication component a very basic set of stimuli would be the
following:

You have to control transmission and reception of data packets.
Therefore you need one command to send and one to receive data - e.g.:

send 0xabcd
receive 0x1234

Note that the "receive" command has a data value attached. I use this if
I expect this data word during reception. So this means: The simulation
component will send 0xabcd and afterwards it will receive a data packet
containing 0x1234.

This ideal can be easily extended: Let's assume you have to provide an
address, some configuration bits and a data value for the transmission.
One option would be:

send 0xab 0xcd 0xef

while 0xab would be the address, 0xcd the configuration bits and 0xef
the data. Looks quite bad? No problem, just redefine your syntax:

address 0xab
config 0xcd
data 0xef
send

The concept can be extended. You need some time to wait? Just define a
command for waiting. You need a reset? Also no problem. Even checking
the value of a signal and a lot more stuff is possible. All you need is
a parser, that reads this textfile, parses your commands and does
whatever you want.

To make it easy to write a parser, make the syntax easy. Try to add only
commands, that are really necessary. Don't put supersets of smaller
commands into it, except you have a good reason. Even if the ANSI-C
routines for VHDL are a great help, writing a parser in VHDL is not too
easy.

If your parser becomes too huge think about the following:
If you can define a very basic set of commands, that can be read by your
VHDL parser, but want to define more complex commands that use the basic
commands, then write a "translator" for it. The translator can be
written in any language you like. (I use ANSI C, because I am familiar
with it.) This translator is also a parser, that maps a complex syntax
to the basic syntax for the VHDL parser. Example: If you want to
transmit a specific "frame", then this frame consists of several basic
transmissions - e.g.:

transmit myframe

could be translated by the translator to

start_of_frame
address 0xab
config 0xcd
data 0xef
send
address 0x12
config 0x34
data 0x45
send
end_of_frame


As you can see: "Nothing" is impossible. You only have to think about a
suitable syntax for your text-stimulus file and a VHDL-parser that reads
the file and applies the stimuli to your VHDL simulation.


Ralf

 

[NigelE]

Hi everyone,
I found the need to access a text based file to create my vector
stimuli. Basically I use procedures which will send hexadecimal values
in the correct protocol, but so far I've always added "manually" the
values in the testbench in the following way:

start_simulation;
send_data ("abcd");
send_data ("1234");
send_data ("a0a0");
send_data ("ffff");
send_data ("5aa5");
..
end_simulation;

With this approach if I want to add other commands I need to add new
lines and compile it again.
I thought that using a file to set the value contents will save me from
compiling again every time, is this correct?
In this case the simulation will just stop when file is finished and
there are no more data available, is my assumption correct?
Could anyone post some examples on how this approach can be sketched?
Thanks a lot

Al

If you're using ModelSim, then included with the software is a
precompiled library called std_developerskit.

In this library are a number of packages, one of which is std_iopak,
which has numerous string manipulation/conversion and file IO
functions.

These provide an easy way to manipulate file data into values that can
be appiled to standard vhdl types.

Hope this helps

- Nigel

 

[Mike Treseler]

I found the need to access a text based file to create my vector
stimuli.
start_simulation;
send_data ("abcd");

With this approach if I want to add other commands I need to add new
lines and compile it again.
I thought that using a file to set the value contents will save me from
compiling again every time, is this correct?

That's correct, but running a sim compile
is very quick and does a syntax check as well.
With a plain text file, I have to verify
my custom syntax manually and debugging may be challenging.

Consider using a vhdl constant array to hold this
sort of data. You have to edit some file in any case.

-- Mike Treseler

 

[Jim Lewis]

Hi everyone,
I found the need to access a text based file to create my vector
stimuli. Basically I use procedures which will send hexadecimal values
in the correct protocol, but so far I've always added "manually" the
values in the testbench in the following way:

start_simulation;
send_data ("abcd");
send_data ("1234");
send_data ("a0a0");
send_data ("ffff");
send_data ("5aa5");
..
end_simulation;

With this approach if I want to add other commands I need to add new
lines and compile it again.

For large data sets, such as packets of data or images, I put the data
in a file. Then handle the file using something of the form:
send_file(already_open_file_handle); -- in your style
send_file(Rec, already_open_file_handle); -- alternately style, using records and BFMs

Generally though, I do all of the command and synchronization in
VHDL and not in a file. That way I don't have to write a parser -
which results in a slower running interpreted language

While files do allow you to run a different simulation just
by changing the files (either input name to the testbench or
changing the file name with a script in the OS), I generally
put all of the stimulus source in one VHDL file with multiple processes
(one process per interface). A test is an architecture of the stimulus
source file. A test suite has multiple test architectures. Using
default binding rules, one would compile a test architecture and
run a simulation. These steps are so quick, if you scripted
it, it would not add any significant amount of time to your
simulation run.

Cheers,
Jim
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jim Lewis
Director of Training mailto:[email protected]
SynthWorks Design Inc. http://www.SynthWorks.com
1-503-590-4787

Expert VHDL Training for Hardware Design and Verification
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Paul Uiterlinden]

Yes. Textfile-based stimulus generation is quite powerful

But never as powerful as VHDL!

What you need is a simulation component that reads your textfile. In
other words: you have to write a parser and invent a syntax for your
textfile, that is easy to parse but powerful enough to provide
stimuli for all test cases.

Been there, done that. ;-)
Loops? Yes.
Nesting loops? Sure.
Subroutines? You bet.

I've implemented them all. But in the end you're implementing your own
language and yet that nagging feeling of missing features remains.
And the requests of users to implement more stuff: variables, logic
operators, arithmetic operators, conditionals.

VHDL has all this readily implemented, so why not make use of it.

My prefered way of working: for each interface of your DUT, write a
BFM (Bus Functional Model). Write these BFMs in a client/server
fashion. The BFM is the server, waiting for the testbench (the
client) to send commands. Such a method is described in the book
"Writing Testbenches" (http://janick.bergeron.com/).

The advantage is that the BFM's are re-usable for other DUT's using
the same interfaces. Another big advantage is that if your testbench
contains more BFM's (which is usually the case) they all run
concurrently. This gives more real life behavior than models reading
from the same command text file. The alternative of using one command
file per model will make synchronizing the actions of the models
(where needed) quite cumbersome (been there and done that as well).

I use textfile stimuli quite often and the ASNI-C like file I/O
routines from <http://bear.ces.cwru.edu/vhdl/> are a great help.

Thanks for the link. Some people like it, I get C-sick quite easily. I
will have a look with the door shut, not to be caught. As one
colleague says: the last good thing written in C is Schubert's 9th
symphony.

 

[Al]

Not 'automatically'. Your testbench code will read the file until it has
reached the end. Then it will close the file and then stop the clock or
whatever else is needed to finish the simulation.

Yes, this is what a basically intended.

First look up how to use VHDL file i/o routines to open, close, read and
write files. They're really not that much different than how you would do
it in most other languages.

Try writing something to read and write a simple file. That will give you
the basic feel for how it works and is totally independent of any project
that you're planning on using this for (but at the same time be directly
applicable once you've got the hang of it).

Although you're only asking about reading files, you'll probably quickly
find that writing files is useful too to write out logging information of
various sorts. For example let's say you're writing code that does JPEG
image compression (just as an example). A good testbench would be able to
read in real image files (like .JPG, .TIF, .BMP etc.) and then write out a
.JPG file based on what comes out of the unit you're testing. The list of
files that would be compressed would be read in as input from another file
that had one line which contains the name of each file. Just an example of
ways to use file i/o effectively.
....

I find this approach more useful than simply reading stimulus from a file
since it models the actual system.

Unfortunately this approach always failed in my case, most likely
because documentations about the "environment" to model you're talking
about is always lacking. In all my experience (that I admit is very
little) most of the problems come out just when you build the system and
finally you realized that the board produced in shanghai had a 50 MHz
clk instead of 33!
In physics experiments, as far as I can say, groups are very wide and
information is very spread, that's why I'm abandoning the concept of a
"full modeled testbench" because is practically impossible to do it. I
rather prefer to quickly move from testbench to real chip (even if it
costs a lot, since they are antifuse) because there is something in the
real system that I am symply not able to know.

Not
trying to necessarily discourage that approach since it does have merit,
just consider what time you're actually saving if you do it, and keep in
mind that now this external file is also part of your simulation testbench
and needs to be archived and will probably be in a format useful only to you
but probably poorly documented unless you take the additional time to
document what exactly are the list of commands you'll support in this file
and the proper syntax, etc.....rather like specifying a language....which it
is.

I understand your point and I really think that self-explained source,
whatever it will be a C or VHDL or whatever, is always the best way to
proceed. Still I think I will try, as you said, to write some basic
functions to handle this problem and see by experience what I can get.

Thanks for your thorough explanation.

 

[Al]

I've implemented them all. But in the end you're implementing your own
language and yet that nagging feeling of missing features remains.
And the requests of users to implement more stuff: variables, logic
operators, arithmetic operators, conditionals.

Yes, that's why my concept is just to change "data" content, not
"control" content. Most likely all the projects has an interface,
standard or not, on which runs data to exchange with outer world. Is
this point that I want to speed up, not to build up a C-environment, or
my own language environment to test my device.y not make use of it.

My prefered way of working: for each interface of your DUT, write a
BFM (Bus Functional Model). Write these BFMs in a client/server
fashion. The BFM is the server, waiting for the testbench (the
client) to send commands. Such a method is described in the book
"Writing Testbenches" (http://janick.bergeron.com/).

I will check it, thanks for the reference.

Cheers
Al

 

[Al]

That's correct, but running a sim compile
is very quick and does a syntax check as well.
With a plain text file, I have to verify
my custom syntax manually and debugging may be challenging.

Consider using a vhdl constant array to hold this
sort of data. You have to edit some file in any case.

this is how I use to do it now, to have a constant array of "data" and
then just run through it, is just that every time I need to recompile,
so if the simulation is very quick then compilation time starts to
weight a bit.
In case simulations need to go on for hours I understand that this is
not really the point, but if they take only minutes then is quite
annoying this approach, at least to me.
Anyway I think is always worthwhile to have different approaches in
order to have a wider background and run the project faster next times ;-)

 

[Al]

If you're using ModelSim, then included with the software is a
precompiled library called std_developerskit.

In this library are a number of packages, one of which is std_iopak,
which has numerous string manipulation/conversion and file IO
functions.

These provide an easy way to manipulate file data into values that can
be appiled to standard vhdl types.

Hope this helps

I will check and let you know. Thanks a lot!

 

[Ralf Hildebrandt]

Paul Uiterlinden schrieb:

But never as powerful as VHDL! ....
I've implemented them all. But in the end you're implementing your own
language and yet that nagging feeling of missing features remains.

Good point! I agree.

Maybe one "advantage" is, that the not so experienced user if _forced_
to separate the stimuli from the device model. Therefore the chance,
that a clear modelled testbench is the result is maybe a little bit higher.

My prefered way of working: for each interface of your DUT, write a
BFM (Bus Functional Model). Write these BFMs in a client/server
fashion. The BFM is the server, waiting for the testbench (the
client) to send commands.

I think the basic idea of this concept can be easily adopted to textfile
stimulus generation. (Having the disadvantages you have mentioned.) A
textfile forces the newbie to use such a concept while in VHDL it is
most likely that the newbie does something stupid and mixes DUT, BFM and
testbench.


Ralf

 

[Paul Uiterlinden]

I think the basic idea of this concept can be easily adopted to
textfile stimulus generation. (Having the disadvantages you have
mentioned.) A textfile forces the newbie to use such a concept while
in VHDL it is most likely that the newbie does something stupid and
mixes DUT, BFM and testbench.

The way I model my BFMs is that at one side they interface with the
DUT and at the other side there is a single command port. This is a
single inout with a complex record structure. The user however does
not see this record. All I present him is a package with commands he
can use. These commands are procedures that communicate through the
command signal with the BFM, transparent to the user.

So in the end, the testcase looks just like a command file, calling
the commands from the package. With the added benefit that you're
still in VHDL. So if the user wants to do something clever, he can do
that.

The feedback that I receive is that this way of working is far more
flexible and easier than using textfiles (which how it used to be).
Self-checking testcase are very hard to create using textfiles, due
to the lack of interactivity between the testcase and the BFMs.

Regarding mixing DUT, BFM and testbench: the way to avoid that is to
have a single centrally managed testbench. The user instantiates this
testbench in his testcases (which may add up to 100 for a reasonably
sized project). This gives a clear divison between them.

 

[jtw]

Just a little perspective: I have had designs which I stimulated with
vector files that came from logic analyzer dumps. These can be very useful
when trying to pinpoint behavior problems, and for predicting 'real-world'
behavior. (In other words: sometimes the real world doesn't match up to
what you expected.)

Another perspective: I typically run a single test bench with many
different input file sets (command files and/or data input files). The
command files tend to evolve as the design matures (and I'm checking more
stuff.) My preference is for the command files to match the syntax that the
user will use in controlling the system.

Jason

 

[KJ]

Just a little perspective: I have had designs which I stimulated with
vector files that came from logic analyzer dumps. These can be very useful
when trying to pinpoint behavior problems, and for predicting 'real-world'
behavior. (In other words: sometimes the real world doesn't match up to
what you expected.)

I agree, it doesn't come up often but when it does this can be useful
when trying to figure out why sim doesn't match reality.

Another perspective: I typically run a single test bench with many
different input file sets (command files and/or data input files). The
command files tend to evolve as the design matures (and I'm checking more
stuff.) My preference is for the command files to match the syntax that the
user will use in controlling the system.

If you instead model more of the entire system in which the design that
you're trying to test with VHDL models for those other parts you can
accomplish the same goal of "match the syntax that the user will use in
controlling the system.". Doing so will give you a more robust and
accurate model with far more detailed assertion checking than by using
command files to generate stimulus and check responses.

When the 'sim doesn't match reality' bug bites then you have a complete
(or as complete as you've made it) VHDL model of what you were
expecting to compare with the real world to find out where your model
is inaccurate. Then when you update your model to reflect how the real
world is really working you should also be able to see now where your
design is not properly handling the situation as well. Once you've
come up with a fix, re-running the system model should now show
everything working, no asserts getting flagged, etc.

Granted you can do the same thing with command files, the problem is
that your command file essentially defines a new 'language' that likely
will not have all of the features of VHDL (or Verilog or any other) and
you'll have to spend time writing a parser for your 'language'....then
ask yourself again 'why did you spend time writing that parser?'

KJ