[PATCH 00/33] Test framework improvements
sebastian.huber at embedded-brains.de
Thu Jul 23 05:00:05 UTC 2020
On 23/07/2020 00:25, Chris Johns wrote:
> On 22/7/20 7:16 pm, Sebastian Huber wrote:
>> On 22/07/2020 07:04, Chris Johns wrote:
>>> On 22/7/20 1:04 am, Sebastian Huber wrote:
>>> b) As previously discussed there are a few more states a test result can be
>>> other than pass and fail and I thikn the wording here may need tightening. I am
>>> not sure what is needed. Also how do resource leaks effect the result?
>> Currently, the test framework supports only passed and failed at the level of a
>> test check. I am not sure if adding more states at this level is really helpful.
> The terms pass and failed are overloaded and relative to a specific context and
> I am wondering if this needs to be clearer.
Yes they are overloaded. For the test framework it is defined what the
terms mean for a test check, a test case, and a test suite. If it is not
clear from the context, you can say for example "the test check failed".
googletest uses these terms:
"When using googletest, you start by writing /assertions/, which are
statements that check whether a condition is true. An assertion's result
can be /success/, /nonfatal failure/, or /fatal failure/. If a fatal
failure occurs, it aborts the current function; otherwise the program
ISTQB uses pass/fail:
I think we just have to clearly define what the terms mean in our scope
and this is what I tried to do.
>>> c) Does the framework provide a standard way to export a dynamic test
>>> environment so the test and the results can be reviewed as a complete set of
>> Sorrry, I don't understand this question.
> If dynamic configuration effects the test and the test fails would reproducing
> the result be more difficult if you do not know what the dynamic state was? For
> example using this for a network test and the dynamic configuration can select
> between DHCP and a static set up.
You could write a log message which tells you the configuration used.
The tests can be written in C, so you can do a lot of things. It is up
to the writer of the tests to do it properly.
>>> a) What is the picture drawn in?
>> Libreoffice Draw.
>>> I will comment on the patches as well.
>> Thanks for the review.
> Sorry, another question, is libtest installed and available to other parts of
> RTEMS like libbsd and also available to users?
Yes, it is installed:
project_lib_LIBRARIES += librtemstest.a
More information about the devel