Console problem update
Thomas Doerfler
Thomas.Doerfler at imd-systems.de
Fri Nov 5 18:40:50 UTC 2004
Etienne,
> The sender of a file in YMODEM waits for an ACK to be received before
> sending the next packet. So, I can wait forever (at least a little less
> than the timeout value of YMODEM which is, if I'm not mystaken, about 30
> seconds) between each packet.
>
> And I don't understand why you say hardware flow control would not help.
> If RTS is negated during transfer because the buffer is about to get
> filled completely, then the transmitter (sender) will stop and wait for
> RTS (its CTS) to get asserted again before continuing. This would ensure
> that the input buffer never overflow. Am I understanding correctly or is
> there something I miss in my comprehension?
Sorry, this was my fault. For protocols like YModem, HW Flow
Control might work. I once had problems with HW flow control and
different protocols.
Nonetheless I would expect HW flow control to make things even
more difficult. I guess you are on a ColdFire derivate, and the
YModem is currently the only application really consuming
processor load? Then you should definitively have enough
processing power for a 9.6kBaud line.
Hm another guess: What do you do with the data received? If you
start storing it on a Flash Disk (or a real harddisk), things
might be slow there... So, whenever you perform a "write" call
to disk, it might wait until the data has been actually written.
And this might make your communication task stop for several
milliseconds. Hmpf. Let me look at it a bit closer. This would
not make your input buffer overrun. So forget this guess. Hmpf.
But maybe it would nevertheless make sense to try to write to
RAMDISK (IMFS) instead. Just to make sure.
Let me try another run: Maybe it you perform sort of a timed log
of your protocol implementation, putting additional, self-built
log calls to specific parts of your protocol implementation
("Now I am calling read, now I am back with xx bytes read").
Maybe you can find out, why your software seems to hang from
time to time.
I really don't think it is the termios/console mechanism which
generates your headache...
wkr,
Thomas.
>
> Etienne
>
>
>
> -----Message d'origine-----
> De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> Envoyé : 5 novembre, 2004 12:19
> À : Etienne Fortin
> Objet : RE : RE : RE : RE : RE : Console problem update
>
>
> Etienne,
>
> > In what situation does the rawInDropped value can increase? I only see
>
> > buffer overrun, but I really don'T have that much experience with the
> > console especially on RTEMS. So, do you think that putting hardware
> > flow control to work on my faulty board would fix the problem?
>
> rawInDropped will be increased whenever a character has been
> received but there is no more position available in the
> rawInBuffer.
>
> I don't think that any kind of flow control (XON/XOFF, hardware
> etc) would help you, because in general the protocol itself
> should have some kind of flow control.
>
> When your input buffer has an overflow, obviously you get too
> much data.
>
> filling 1K of input buffer takes about 1 second, so I would
> think you look too seldomly, whether input data is available...
>
> What do you test currently? file stransfers from RTEMS to host
> or from host to RTEMS?
>
> Do you send big chunks of data (let's say a 1K Buffer) in
> situations when the output buffer is already filled? Then your
> "write" call will block until the whole new data block can be
> moved into the rawOutBuffer, and this may take too much time for
> your input processing...
>
> wkr,
> Thomas.
>
>
> >
> > Etienne
> >
> >
> >
> >
> > -----Message d'origine-----
> > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > Envoyé : 5 novembre, 2004 11:24
> > À : Etienne Fortin
> > Cc : rtems-users at rtems.com
> > Objet : RE : RE : RE : RE : Console problem update
> >
> >
> > Etienne,
> >
> > head is always "ahead" of tail in the ring buffer, so when
> > head==tail+1, there is exactly one char in the buffer. This is
> > ok, I would say.
> >
> > In the last snapshot, you have a bunch of Debug info in the
> > output buffer, (like "C3D 1.0.0 File Tranfer Test\r\nNeed a
> > valid FAT12 file system on /dev/nvram0 ") I hope this does not
> > happen during YMOdem transfer?
> >
> > Sorry for asking, but do you possibly have a flaw in your
> > protocol engine design? When there are more characters in the
> > input than expected, do you consume them properly?
> >
> > Seems to get a weekend with headaches for you ?
> > wkr,
> > Thomas.
> >
> >
> > > Some more info.
> > >
> > > My rawInBuf is now 1024 bytes. And it gives me the same problem:
> > > Head
> > > go beyond Tail in rawInBuf and rawInBufDropped starts to go up.
> > >
> > > $2 = {forw = 0x0, back = 0x0, refcount = 3, major = 1, minor = 0,
> > > isem
> >
> > > = 436273159, osem = 436273160, cbuf = 0x80d53be4 "ò", ccount = 1,
> > > cindex = 1, column = 0, read_start_column = 0, termios = {c_iflag =
> > > 8450, c_oflag = 4294967230, c_cflag = 2237, c_lflag = 32768, c_line
> =
> > > 0 '\0', c_cc =
> > >
> > "\003\034\177\025\004\n\001\000\021\023\032\000\022\017\027\026\000\00
> > 0"
> > > }, vtimeTicks = 100, rawInBuf = {theBuf = 0x80d5407c "", Head = 197,
>
> > > Tail = 196, Size = 1024, Semaphore = 436273162},
> > > rawInBufSemaphoreOptions = 0, rawInBufSemaphoreTimeout = 100,
> > > rawInBufSemaphoreFirstTimeout = 0, rawInBufDropped = 3, rawOutBuf =
> > > {theBuf = 0x80d53c70 "\r\n\r\nC3D 1.0.0 File Tranfer Test\r\nNeed a
> > > valid FAT12 file system on /dev/nvram0 (run fs_create).\r\n\r\nMount
>
> > > DOSFS file system on /dev/nvram0 (run fs_create before)\r\nfsmount:
> > > mounting of \"/dev/nvram0\" to "..., Head = 413, Tail = 413, Size =
> > > 1024, Semaphore = 436273161}, t_dqlen = 0, rawOutBufState =
> > > rob_idle, device = {firstOpen = 0x17a8 <console_first_open>,
> > > lastClose = 0x1804 <console_last_close>, pollRead = 0, write =
> > > 0x16de <console_interrupt_write>, setAttributes = 0x172e
> > > <console_set_attributes>, stopRemoteTx = 0x1750
> > > <console_stop_remote_tx>, startRemoteTx = 0x177c
> > > <console_start_remote_tx>, outputUsesInterrupts = 1}, flow_ctrl = 1,
>
> > > lowwater = 512, highwater = 768, rxTaskId = 0, txTaskId = 0, t_line
> > > =
> > 0,
> > > t_sc = 0x0, tty_snd = {sw_pfn = 0, sw_arg = 0x0}, tty_rcv = {sw_pfn
> > > =
> > 0,
> > > sw_arg = 0x0}, tty_rcvwakeup = 0}
> > >
> > >
> > > Something to do with the absence of hardware flow control?
> > >
> > > And now no matter what file I send, the problem appears at 12k for a
> > > 1024 buffers (sooner for a 128 bytes buffer).
> > >
> > > Etienne
> > >
> > >
> > >
> > >
> > >
> > > -----Message d'origine-----
> > > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > > Envoyé : 5 novembre, 2004 09:56
> > > À : Etienne Fortin; rtems-users at rtems.com
> > > Objet : RE : RE : RE : Console problem update
> > >
> > >
> > > Etienne,
> > >
> > > a baud rate difference of 2% is still tolerable, as
> > > fas as I know. And you would be well below.
> > >
> > > I just had another look into the data structure
> > > snapshot you sent this morning (last night...). To
> > > my suprise, the "flow_ctrl" fiels is non-zero. The
> > > value 512 means, that the "input is controlled with
> > > XON/XOFF protocol". Maybe you should try to clear
> > > the "IXON" and "IXOFF" flags in the c_iflag field of
> > > termios settings?
> > >
> > > wkr,
> > > Thomas.
> > >
> > >
> > > > Hi Thomas,
> > > > Yes, the buffer is empty. But, it shows that at least 1161
> > > > characters
> > > > were received. After that, termios thinks there's no more data
> > > > received or it lost sync or something else.
> > > >
> > > > I said before that my baudrate is really 9630 and not 9600. The
> > > > computer is 9600. Do you think that can be the problem?
> > > >
> > > > Etienne
> > > >
> > > >
> > > >
> > > > -----Message d'origine-----
> > > > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > > > Envoyé : 5 novembre, 2004 02:23
> > > > À : Etienne Fortin
> > > > Objet : RE : RE : Console problem update
> > > >
> > > >
> > > > Etienne,
> > > >
> > > > from the data you sent, the input and output buffer really is
> > > > "empty", (for rawInBuf and rawOutBuf, "Head" and "Tail" have the
> > > > same values) but I guess you already saw this.
> > > >
> > > > Which BSP are you using? Is it possible your console driver
> > > > interrupt gets stuck, when there is a receive and transmit
> interrupt
> >
> > > > pending at the same time?
> > > >
> > > > wkr,
> > > > Thomas.
> > > >
> > > >
> > > > > Hi Thomas,
> > > > > Yes a VERY nasty problem... It must have something to do with my
> > > > > comprehension and the way I implemented the console driver...
> But,
> >
> > > > > I
> > >
> > > > > was able to make a lot of other things work before (I mean, the
> > > > > balance of
> > > > > RTEMS) and I got nasty problem with RTEMS before. But that one
> > > > > is special. Its so stupid! And I hate to work with transfer
> > > > > protocol.
> > > You
> > > > > know you'll be able to make it work, you just don't know when...
> > > > >
> > > > > Anyway, here's the snapshot of the rtems_termios_tty structure
> > > > > when the semaphore get stuck.
> > > > >
> > > > > $2 = {forw = 0x0, back = 0x0, refcount = 3, major = 1, minor =
> > > > > 0, isem
> > > >
> > > > > = 436273159, osem = 436273160, cbuf = 0x80d542a4 "}", ccount =
> > > > > 0,
> > > > > cindex = 0, column = 0, read_start_column = 0, termios =
> {c_iflag
> > > > > = 9474, c_oflag = 4294967230, c_cflag = 2237, c_lflag = 32768,
> > > > > c_line
> > > =
> > > > > 0 '\0', c_cc =
> > > > >
> > > > "\003\034\177\025\004\n\001\000\021\023\032\000\022\017\027\026\00
> > > > 0\
> > > > 00
> > > > 0"
> > > > > }, vtimeTicks = 100, rawInBuf = {theBuf = 0x80d53a98 "", Head =
> > > > > 1161, Tail = 1161, Size = 2048, Semaphore = 436273162},
> > > > > rawInBufSemaphoreOptions = 0, rawInBufSemaphoreTimeout = 100,
> > > > > rawInBufSemaphoreFirstTimeout = 0, rawInBufDropped = 0,
> > > > > rawOutBuf
> > =
> > > > > {theBuf = 0x80d543b0 "ransfer...\r\n\r\nC\006C.\r\nrawInBuf.Size
>
> > > > > =
> >
> > > > > 2048\r\nPress a key to begin t", Head = 17, Tail = 17, Size =
> > > > > 64,
> > > > > Semaphore = 436273161}, t_dqlen = 0, rawOutBufState = rob_idle,
> > > > > device
> > > > =
> > > > > {firstOpen = 0x17a2 <console_first_open>, lastClose = 0x17ea
> > > > > <console_last_close>, pollRead = 0, write = 0x16dc
> > > > > <console_interrupt_write>, setAttributes = 0x172c
> > > > > <console_set_attributes>, stopRemoteTx = 0x174e
> > > > > <console_stop_remote_tx>, startRemoteTx = 0x1778
> > > > > <console_start_remote_tx>, outputUsesInterrupts = 1}, flow_ctrl
> > > > > = 512, lowwater = 64, highwater = 96, rxTaskId = 0, txTaskId =
> > > > > 0, t_line = 0, t_sc = 0x0, tty_snd = {sw_pfn = 0, sw_arg = 0x0},
>
> > > > > tty_rcv = {sw_pfn =
> > > > 0,
> > > > > sw_arg = 0x0}, tty_rcvwakeup = 0}
> > > > >
> > > > > As you can see, I "hardwired" the rawInBuf to be 2048 characters
> > > > > in size. And surprisingly enough, the Head is at offset 1161.
> This
> >
> > > > > mean
> > >
> > > > > that I received the first 130 bytes packet (init packet) and
> > > > > some
> > > part
> > > >
> > > > > of the first data packet. But for some reason, termios lost sync
> > > > > and lost all the next characters, stucking the semaphore and
> > > > > giving me headaches :)
> > > > >
> > > > > What do you think???
> > > > >
> > > > > And by the way, you can't imagine how glad I am to have help
> > > > > from all of you on RTEMS forum.
> > > > >
> > > > > Etienne Fortin
> > > > > Sensio
> > > > >
> > > > >
> > > > >
> > > > > -----Message d'origine-----
> > > > > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > > > > Envoyé : 4 novembre, 2004 16:09
> > > > > À : Etienne Fortin; rtems-users at rtems.com
> > > > > Cc : rtems-users at rtems.com
> > > > > Objet : RE : Console problem update
> > > > >
> > > > >
> > > > > Etienne,
> > > > >
> > > > > can you provide a snapshot of the termios data structures when
> > > > > your
> > > > > console input gets stuck? What values do the buffer pointers
> have
> > > > > for the raw input buffer?
> > > > >
> > > > > What baud rate are you using currently?
> > > > >
> > > > > What happens, if you write a simple application with two tasks,
> > > > > one
> > > > > sending a 1K buffer to the console, one receiving it, and
> putting
> > > > > the console into loopback mode (with a nice little wire between
> > pin
> > > > > 2 and 3 of your serial connector)? Does it also hand in that
> > > > > case?
> > > > >
> > > > > You really seem to have a nasty problem there...
> > > > >
> > > > > I have been using termios code for machine interfaces back with
> > > > > RTEMS version 4.0.0, and I never had similar problems...
> > > > >
> > > > > wkr,
> > > > > Thomas.
> > > > >
> > > > >
> > > > > > I don'T know why I tought that even after releasing the
> > > > > > sempahore,
> > >
> > > > > > I was still stuck in the idle thread. To the contrary! Forcing
> > > > > > the
> > >
> > > > > > release of the semaphore make the task continue to execute
> > > > > > code and
> > > > I
> > > > > > eventually get out of the code with an error (that's ok since
> > > > > > the semaphore was forced to be released so no valid data is
> > > > > > there waiting)...
> > > > > >
> > > > > > The question now is why the code is not notified of new
> > > > > > characters. Is
> > > > >
> > > > > > it possible that the termios code, being designed for "human"
> > > > > > interraction (at least I presume), can't cope with sustained
> > > > > > binary data transfer and so I lost data and eventually the
> > > > > > semaphore is struck waiting for characters that will never
> > > > > > come?
> >
> > > > > > Is it possible?
> > > > > >
> > > > > > Etienne Fortin
> > > > > > Sensio
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > -----Message d'origine-----
> > > > > > De : Joel Sherrill <joel at OARcorp.com>
> > > > > > [mailto:joel.sherrill at OARcorp.com]
> > > > > >
> > > > > > Envoyé : 4 novembre, 2004 15:21
> > > > > > À : Etienne Fortin
> > > > > > Cc : rtems-users at rtems.com
> > > > > > Objet : Re: Console problem update
> > > > > >
> > > > > >
> > > > > > Etienne Fortin wrote:
> > > > > > > Hello all, its me again, the console guy :)
> > > > > > >
> > > > > > > Little update for the ones that have an interest, or wish to
>
> > > > > > > help me on that matter.
> > > > > >
> > > > > > Is your interrupt an RTEMS ISR or a raw one directly connected
> > > > > > to hardware? If you do not go through the RTEMS ISR wrapper,
> > > > > > you
> > > will
> > > > > > NEVER schedule out of an interrupt.
> > > > > >
> > > > > > Also are you sure you are geting the serial port interrupts?
> > > > > >
> > > > > > > I tricked RTEMS to release the semaphore at each clock tick
> > > > > > > to
> >
> > > > > > > make sure the semaphore was the problem. And surprise
> > > > > > > surprise!
> > > It
> > > >
> > > > > > > still hang in the idle thread!!!!!!!!!
> > > > > > >
> > > > > > > Theory: My task/scheduler/... config is fu***d up. Anyone
> > > > > > > here thinks it can be possible? For what I know of the
> > > > > > > problem so far, it
> > > > seems
> > > > > > > that as soon as the idle thread is entered, no other thread
> > > > > > > ever
> > > > > gets
> > > > > > > executed... Really seems a config problem to me...
> > > > > > >
> > > > > > > Idea someone?
> > > > > > >
> > > > > > > Etienne Fortin
> > > > > > > Sensio
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Joel Sherrill, Ph.D. Director of Research &
> > > Development
> > > > > > joel at OARcorp.com On-Line Applications Research
> > > > > > Ask me about RTEMS: a free RTOS Huntsville AL 35805
> > > > > > Support Available (256) 722-9985
> > > > > >
> > > > > >
> > > > >
> > > > > --------------------------------------------
> > > > > IMD Ingenieurbuero fuer Microcomputertechnik
> > > > > Thomas Doerfler Herbststrasse 8
> > > > > D-82178 Puchheim Germany
> > > > > email: Thomas.Doerfler at imd-systems.de
> > > > > PGP public key available at: http://www.imd-
> > > > > systems.de/pgp_keys.htm
> > > > >
> > > > >
> > > >
> > > > --------------------------------------------
> > > > IMD Ingenieurbuero fuer Microcomputertechnik
> > > > Thomas Doerfler Herbststrasse 8
> > > > D-82178 Puchheim Germany
> > > > email: Thomas.Doerfler at imd-systems.de
> > > > PGP public key available at: http://www.imd-
> > > > systems.de/pgp_keys.htm
> > > >
> > > >
> > >
> > > --------------------------------------------
> > > IMD Ingenieurbuero fuer Microcomputertechnik
> > > Thomas Doerfler Herbststrasse 8
> > > D-82178 Puchheim Germany
> > > email: Thomas.Doerfler at imd-systems.de
> > > PGP public key available at: http://www.imd- systems.de/pgp_keys.htm
> > >
> > >
> >
> > --------------------------------------------
> > IMD Ingenieurbuero fuer Microcomputertechnik
> > Thomas Doerfler Herbststrasse 8
> > D-82178 Puchheim Germany
> > email: Thomas.Doerfler at imd-systems.de
> > PGP public key available at: http://www.imd- systems.de/pgp_keys.htm
> >
> >
>
> --------------------------------------------
> IMD Ingenieurbuero fuer Microcomputertechnik
> Thomas Doerfler Herbststrasse 8
> D-82178 Puchheim Germany
> email: Thomas.Doerfler at imd-systems.de
> PGP public key available at: http://www.imd- systems.de/pgp_keys.htm
>
>
--------------------------------------------
IMD Ingenieurbuero fuer Microcomputertechnik
Thomas Doerfler Herbststrasse 8
D-82178 Puchheim Germany
email: Thomas.Doerfler at imd-systems.de
PGP public key available at: http://www.imd-
systems.de/pgp_keys.htm
More information about the users
mailing list