POSIX Async IO (GSoC)

[Wei Shen]

Hi,

Just to correct some errors of the previous mail, and extended it with a few
more thoughts.



> On 4/3/08, Wei Shen <cquark at gmail.com> wrote:
>

I studied the implementation of Glibc. Below I list some elementary
implementation thoughts. Comments are solicited.

Seems that AIO is not so simple a task as I have thought, especially in
that:
(1) should support priority (defined by aiocb.aio_reqprio) based AIO request
scheduling
This can be realized by: a) using one task queue for each priority level or
linking the tasks having the same priority; b) inserting a new task to the
right position in the queue according to its priority (O(lgN) time
complexity required).

(2) aio_cancel should support cancel all the AIO tasks on a FD.
This implies a task queue for each FD or another linklist to link the
tasks on the same FD, otherwise this call would be time-consuming.

(3) Do we need to consider optimization at task selection from a task queue.
For example if there is a task on one FD still in processing, then we
consider first tasks of other FDs. Glibc seems to use one worker thread for
each FD.

We also need to support two kinds of notification - signal (SIGEV_SIGNAL)
and callback (SIGEV_THREAD), immediate stutus query (aio_error), and
blocking on an AIO task (aio_suspend).

It would be useful to abstract a general task pool that supports various
async tasks besides AIO. Such a task pool should include:

(1) a number of task queues, and an interface for users to create and
locates a task queue - based on a specific attribute or link tasks according
to their attributes (e.g. FD attribute of AIO tasks).

(2) user defined task processing handlers (one hander per pool, per task
queue, or per task?), and completion notification per task (maybe better to
leave this to the handler)?
For the case of AIO, sharing one handler in the whole pool is enough.

(3) priority based task scheduling.

(4) task enqueuing, canceling (matching by specific attributes), status
query, blocked waiting.

In summary, an efficient algorithm and data-structures to support task
enqueue/dequeue, scheduling, search and sort, assignment and concurrent
processing.

(5) resource management/configuration/accounting
Including:
a) max number of worker threads, thread stack size, thread priority;
b) max number of active task queues in the pool;
c) max number of tasks in the pool or in each queue;
d) dynamic adjustment of worker threads.

So, quite a little work to build a good solution ...

Thanks,
Wei Shen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20080403/3e693048/attachment-0001.html>