[PATCH 1/2] rtl-obj.c: Added an early return if rtl lock fails

[Ryan Long]

-----Original Message-----
From: Chris Johns <chrisj at rtems.org> 
Sent: Monday, April 5, 2021 6:46 PM
To: Ryan Long <ryan.long at oarcorp.com>; Gedare Bloom <gedare at rtems.org>
Cc: devel at rtems.org
Subject: Re: [PATCH 1/2] rtl-obj.c: Added an early return if rtl lock fails

On 5/4/21 11:11 pm, Ryan Long wrote:
> When you say "allocating the main data structure", are you referring to the Run-Time Link Editor (rtl)? If so, rtems_rtl_data_init() is allocating memory for it. If it fails then rtems_rtl_data_init() returns false. Then, this causes rtems_rtl_lock() to return NULL.

I would need to look at this in much more detail to completely resolve what happens at an architectural level. That is something else to this patch.

> 
> -----Original Message-----
> From: Chris Johns <chrisj at rtems.org>
> Sent: Tuesday, March 30, 2021 11:31 PM
> To: Gedare Bloom <gedare at rtems.org>; Ryan Long <ryan.long at oarcorp.com>
> Cc: devel at rtems.org
> Subject: Re: [PATCH 1/2] rtl-obj.c: Added an early return if rtl lock 
> fails
> 
> On 31/3/21 4:08 am, Gedare Bloom wrote:
>> This looks ok. I'm wondering however if these rtems_rtl_lock () ever 
>> should fail in practice, should it be an assert? Just wondering.
> 
> Good question, I cannot remember. Maybe a better solution is to move to a statically allocated lock using the newer interface Sebastian has added and removing the check?
> 
> However is the lock also allocating the main data structure and also returning the status of that? I do this as a way of not uses resources until the service is used. If that is the case the change maybe OK but the name of the call is a bit average.
> 
> Ryan, are you able to take have a closer look.
> 
> Thanks
> Chris
> 
>>
>> On Tue, Mar 30, 2021 at 10:58 AM Ryan Long <ryan.long at oarcorp.com> wrote:
>>>
>>> ping
>>>
>>> -----Original Message-----
>>> From: Ryan Long <ryan.long at oarcorp.com>
>>> Sent: Monday, March 22, 2021 12:08 PM
>>> To: devel at rtems.org
>>> Cc: Ryan Long <ryan.long at oarcorp.com>
>>> Subject: [PATCH 1/2] rtl-obj.c: Added an early return if rtl lock 
>>> fails
>>>
>>> CID 1444138: Dereference null return value in rtems_rtl_obj_find_file().
>>>
>>> Closes #4332
>>> ---
>>>  cpukit/libdl/rtl-obj.c | 4 ++++
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/cpukit/libdl/rtl-obj.c b/cpukit/libdl/rtl-obj.c index 
>>> a7dd740..d5a867e 100644
>>> --- a/cpukit/libdl/rtl-obj.c
>>> +++ b/cpukit/libdl/rtl-obj.c
>>> @@ -409,6 +409,10 @@ rtems_rtl_obj_find_file (rtems_rtl_obj* obj, 
>>> const char* name)
>>>
>>>    rtl = rtems_rtl_lock ();
>>>
>>> +  if (rtl == NULL) {
>>> +    return false;
>>> +  }

An error should be set here before returning or the caller will not know whhy that got an error.

>>> +
>>>    if (!rtems_rtl_find_file (pname, rtl->paths, &obj->fname, &obj->fsize))
>>>    {
>>>      rtems_rtl_set_error (ENOENT, "file not found");

Here is an example.
[Ryan Long] Would this patch be good to merge with this addition? If not, what is the preferred solution path?

Chris