[RTEMS Project] #4565: medit malloc problem of RTEMS5.1

[RTEMS trac]

#4565: medit malloc problem of RTEMS5.1
----------------------------+--------------------
  Reporter:  chenjin_zhong  |      Owner:  (none)
      Type:  enhancement    |     Status:  new
  Priority:  normal         |  Milestone:  5.1
 Component:  shell          |    Version:  5
  Severity:  normal         |   Keywords:
Blocked By:                 |   Blocking:
----------------------------+--------------------
 I find malloc function is called by move_gap function in medit.c. The
 returned value does not check. At least 32KB of memory is allocated at
 each time, maybe more than. The returned value "start" should be check to
 avoid malloc failure. The move_gap function should return immediatelty
 when malloc failure. the code frament is listed as follows.

  static void move_gap(struct editor *ed, int pos, int minsize) {
    int gapsize = ed->rest - ed->gap;
    unsigned char *p = text_ptr(ed, pos);
    if (minsize < 0) minsize = 0;
    if (minsize <= gapsize) {
      if (p != ed->rest) {
        if (p < ed->gap) {
          memmove(p + gapsize, p, ed->gap - p);
        } else {
          memmove(ed->gap, ed->rest, p - ed->rest);
        }
        ed->gap = ed->start + pos;
        ed->rest = ed->gap + gapsize;
      }
    } else {
      int newsize;
      unsigned char *start;
      unsigned char *gap;
      unsigned char *rest;
      unsigned char *end;

      if (gapsize + MINEXTEND > minsize) minsize = gapsize + MINEXTEND;
      newsize = (ed->end - ed->start) - gapsize + minsize;
      ** start = (unsigned char *) malloc(newsize);** // TODO check for out
 of memory
      gap = start + pos;
      rest = gap + minsize;
      end = start + newsize;

      if (p < ed->gap) {
       memcpy(start, ed->start, pos);
       memcpy(rest, p, ed->gap - p);
       memcpy(end - (ed->end - ed->rest), ed->rest, ed->end - ed->rest);
      } else {
       memcpy(start, ed->start, ed->gap - ed->start);
       memcpy(start + (ed->gap - ed->start), ed->rest, p - ed->rest);
       memcpy(rest, p, ed->end - p);
      }

      free(ed->start);
      ed->start = start;
      ed->gap = gap;
      ed->rest = rest;
      ed->end = end;
    }

   #ifdef DEBUG
     memset(ed->gap, 0, ed->rest - ed->gap);
   #endif
  }

--
Ticket URL: <http://devel.rtems.org/ticket/4565>
RTEMS Project <http://www.rtems.org/>
RTEMS Project