crypt01 execution time

[Joel Sherrill]

On 11/26/2014 1:12 AM, Sebastian Huber wrote:
> On 25/11/14 23:25, Joel Sherrill wrote:
>> How long is this test supposed to run?
>>
>> It takes 4:42 using sis on my computer which is a 2.9 Ghz i7 .
> SIS is a slow simulator.  On Qemu it runs much faster.
To repeat a statement by Jiri, SIS is an accurate simulator.  Just
a different goal.
>> Is there anything to do? Split it?
> Splitting it up doesn't reduce the overall test time.  The test cases in 
> this test are standard test cases.  I am not so fond of removing them, 
> but in case the test time is too long for you then you can drop the ones 
> with the many rounds.
>
I took the statement "doesn't reduce overall test time" earlier at face
value
and didn't push back. It is a true statement to the extent that splitting
it results in N tests which run for the same combined time. But what
was missed is that the run-time of this test is much longer than any
other test. This results in having to set a timeout value that accommodates
this test while others could be timed out in the equivalent of 180
seconds of CPU time.

Thus it isn't the test time itself that is the problem. It is the impact
it has on setting the overall timeout value since this single test
runs for 309.51s of simulated CPU time according to sis. Every test
which times out is bound by the maximum allowed execution time
of the slowest tests.

RTEMS suffers from having many unwritten rules. Originally, all the
sptests generally did not run much longer than ticker. That test takes
35 seconds of target time plus boot.  Of course, this was never a hard
rule and many exceptions crept in. 

sis (and tsim) can be programmed to timeout after N seconds of simulated
CPU time. Based on sim-scripts, 180 seconds is sufficient as a general
timeout if you ignore crypt01.  Splitting the test into multiple tests
would
bring it back in line with the maximum execution time of any other test.
 
When testing with a simulator, we can run multiple instances in parallel.
It is just a matter of managing the maximum run-time of any test so
they cluster rather than having a single outlier driving the selection.

This test should be easy to split. The code looks like this:

    static void Init(rtems_task_argument arg)
    {
      TEST_BEGIN();

      test_formats();
      test_md5();
      test_sha256();
      test_sha512();
      test_generic();

      TEST_END();
      rtems_test_exit(0);
    }

The test has five sub-parts and it is just a matter of dividing the test
into those subparts. The simplest way would be conditional compilation
driven from the Makefile.am to build part 1, 2, 3, ... as needed.

FWIW it took 24.61 seconds to run on a 2.4 Ghz older CPU. I added a
print of uptime to get a feel of how it would need to be split.

    *** BEGIN OF TEST CRYPT 1 ***
    UPTIME: 0: 2479000
    UPTIME: 0: 3537000
    test crypt_md5_r()
    UPTIME: 0: 404662000
    test crypt_sha256_r()
    UPTIME: 196: 135394000
    test crypt_sha512_r()
    UPTIME: 644: 629217000
    test crypt_r()
    UPTIME: 663: 426214000
    *** END OF TEST CRYPT 1 ***

Unfortunately, it looks like the big "chunks" are sha256 and
sha512 which are both over 180 seconds. So the list of
test input/output list for those would have to be split
into different runs.

-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel.sherrill at OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
Support Available                (256) 722-9985

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20150414/08e1a04f/attachment.html>