[tools] tester: Normalize JSON and YAML reports

[Sebastian Huber]

On 29/06/2022 11:20, Chris Johns wrote:
> 
>> On 29 Jun 2022, at 4:42 pm, Sebastian Huber <sebastian.huber at embedded-brains.de> wrote:
>>
>> On 29/06/2022 08:40, Sebastian Huber wrote:
>>> Report the same data in JSON and YAML reports.  Do not report redundant
>>> information.
>>> Update 4671.
>>
>> This patch changes the JSON reports. Are there already consumers for the JSON reports so that we have to be backward compatible?
> 
> Could compatibility be added back in if this proves to be an issue?
> 
> I am wondering if that could be considered if comparability becomes an issue.

The JSON report was added by:

commit 5fcaf843bdb491e2da14dc1af61806ac6d20bae5
Author: Kinsey Moore <kinsey.moore at oarcorp.com>
Date:   Wed Aug 21 16:34:12 2019 +0000

     Add JSON log generation

     Add log formatter hooks and JSON log formatter to the test 
infrastructure
     for consumption by automated processes or report generators.


I am not sure if these automated processes or report generators already 
exist.

The existing attribute names are quite inconsistent, for example 
"Command Line", "passed_count", "wrong-version_count". I would use lower 
case only with "-" as a separator. The JSON report should contain all 
information of a test run.

The new report looks like this:

{
     "command-line": [
         "/opt/rtems/6/bin/rtems-test",
         "--rtems-bsp=xilinx_zynq_a9_qemu",
         "--report-format=json",
         "--report-path=report",
         "build/arm/xilinx_zynq_a9_qemu/testsuites/samples/"
     ],
     "end-time": "2022-06-28T14:08:47.595131",
     "host": "Linux-5.3.18-150300.59.76-default-x86_64-with-glibc2.3.4 
(Linux lupus 5.3.18-150300.59.76-default #1 SMP Thu Jun 16 04:23:47 UTC 
2022 (2cc2ade) x86_64 x86_64)",
     "python": "3.6.15 (default, Sep 23 2021, 15:41:43) [GCC]",
     "reports": [
         {
             "arch": "arm",
             "bsp": "xilinx_zynq_a9_qemu",
             "command-line": "qemu-system-arm -no-reboot -nographic -net 
none -serial null -serial mon:stdio -M xilinx-zynq-a9 -m 256M -kernel 
build/arm/xilinx_zynq_a9_qemu/testsuites/samples/cdtest.exe",
             "end-time": "2022-06-28T12:08:48.161691+00:00",
             "executable": 
"build/arm/xilinx_zynq_a9_qemu/testsuites/samples/cdtest.exe",
             "executable-sha512": 
"413bc8a506053c95a0e1d93942cce9fd0d653f7b3cde42623085ce147aea324f4baeff7ce0102008322fb1341bc2b38defd3925a64144e47e13cc5f32185bf78",
             "output": [
                 "qemu-system-arm: warning: nic cadence_gem.0 has no peer",
                 "qemu-system-arm: warning: nic cadence_gem.1 has no peer",
                 "",
                 "",
                 "*** BEGIN OF TEST CONSTRUCTOR/DESTRUCTOR ***",
                 "*** TEST VERSION: 
6.0.0.3302b72754df5f37214e86dd68522189857772c7",
                 "*** TEST STATE: EXPECTED_PASS",
                 "*** TEST BUILD: RTEMS_DEBUG RTEMS_POSIX_API",
                 "*** TEST TOOLS: 12.1.1 20220622 (RTEMS 6, RSB 
bf3f66c29a046ec1c5029e4f919b525947df6245, Newlib e7ffbdb)",
                 "GLOBAL: Hey I'm in base class constructor number 1 for 
0x214474.",
                 "GLOBAL: Hey I'm in base class constructor number 2 for 
0x214480.",
                 "GLOBAL: Hey I'm in derived class constructor number 3 
for 0x214480.",
                 "LOCAL: Hey I'm in base class constructor number 4 for 
0x228864.",
		...

 
"WRoZqtoO3A8AAAABDAAAAJkUSmRGNOBaaHW1UwAAoQGQ////AAAAAQwAAABck8Bx+K/znDWWTEcA",
 
"AKEB2P///wAAAAEMAAAAVP6TRqQZY+4+srvAAAChAfD///8AAAABDAAAAMXOxS0Rhzqx6Old2wAA",
 
"oQH4////AAAAAQwAAADJyx1ZCnZ4l+m/XkMAAKEB0P///wAAAAEMAAAAN3+9YAwMW8gTHIoPAACh",
 
"Adj///8AAAABDAAAABtxsj3zXZF/+UqzfAAAoQHw////AAAAAQwAAAAwMDMeE7mphT6yu8AAAKEB",
                 "8P///wAAAAEMAAAAr2rLCcwzVnf5SrN8AAChAfD///8AAAAA",
                 "*** END OF GCOV INFO BASE64 ***",
                 ""
             ],
             "result": "passed",
             "start-time": "2022-06-28T12:08:47.721822+00:00"
         }
     ],
     "start-time": "2022-06-28T14:08:47.595131"
}

-- 
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.huber at embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/