Console problem update

Etienne Fortin etienne.fortin at sensio.tv
Fri Nov 5 18:28:52 UTC 2004


Hi Thomas,
The sender of a file in YMODEM waits for an ACK to be received before
sending the next packet. So, I can wait forever (at least a little less
than the timeout value of YMODEM which is, if I'm not mystaken, about 30
seconds) between each packet. 

And I don't understand why you say hardware flow control would not help.
If RTS is negated during transfer because the buffer is about to get
filled completely, then the transmitter (sender) will stop and wait for
RTS (its CTS) to get asserted again before continuing. This would ensure
that the input buffer never overflow. Am I understanding correctly or is
there something I miss in my comprehension?

Etienne



-----Message d'origine-----
De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de] 
Envoyé : 5 novembre, 2004 12:19
À : Etienne Fortin
Objet : RE : RE : RE : RE : RE : Console problem update


Etienne,

> In what situation does the rawInDropped value can increase? I only see

> buffer overrun, but I really don'T have that much experience with the 
> console especially on RTEMS. So, do you think that putting hardware 
> flow control to work on my faulty board would fix the problem?

rawInDropped will be increased whenever a character has been 
received but there is no more position available in the 
rawInBuffer. 

I don't think that any kind of flow control (XON/XOFF, hardware 
etc) would help you, because in general the protocol itself 
should have some kind of flow control. 

When your input buffer has an overflow, obviously you get too 
much data. 

filling 1K of input buffer takes about 1 second, so I would 
think you look too seldomly, whether input data is available... 

What do you test currently? file stransfers from RTEMS to host 
or from host to RTEMS?

Do you send big chunks of data (let's say a 1K Buffer) in 
situations when the output buffer is already filled? Then your 
"write" call will block until the whole new data block can be 
moved into the rawOutBuffer, and this may take too much time for 
your input processing...

wkr,
Thomas.


> 
> Etienne
> 
> 
> 
> 
> -----Message d'origine-----
> De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> Envoyé : 5 novembre, 2004 11:24
> À : Etienne Fortin
> Cc : rtems-users at rtems.com
> Objet : RE : RE : RE : RE : Console problem update
> 
> 
> Etienne,
> 
> head is always "ahead" of tail in the ring buffer, so when
> head==tail+1, there is exactly one char in the buffer. This is 
> ok, I would say.
> 
> In the last snapshot, you have a bunch of Debug info in the
> output buffer, (like "C3D 1.0.0 File Tranfer Test\r\nNeed a 
> valid FAT12 file system on /dev/nvram0 ") I hope this does not 
> happen during YMOdem transfer?
> 
> Sorry for asking, but do you possibly have a flaw in your
> protocol engine design? When there are more characters in the 
> input than expected, do you consume them properly? 
> 
> Seems to get a weekend with headaches for you ?
> wkr,
> Thomas.
> 
> 
> > Some more info.
> > 
> > My rawInBuf is now 1024 bytes. And it gives me the same problem: 
> > Head
> > go beyond Tail in rawInBuf and rawInBufDropped starts to go up.
> > 
> > $2 = {forw = 0x0, back = 0x0, refcount = 3, major = 1, minor = 0, 
> > isem
> 
> > = 436273159, osem = 436273160, cbuf = 0x80d53be4 "ò", ccount = 1,
> > cindex = 1, column = 0, read_start_column = 0, termios = {c_iflag = 
> > 8450, c_oflag = 4294967230, c_cflag = 2237, c_lflag = 32768, c_line
= 
> > 0 '\0', c_cc = 
> >
> "\003\034\177\025\004\n\001\000\021\023\032\000\022\017\027\026\000\00
> 0"
> > }, vtimeTicks = 100, rawInBuf = {theBuf = 0x80d5407c "", Head = 197,

> > Tail = 196, Size = 1024, Semaphore = 436273162}, 
> > rawInBufSemaphoreOptions = 0, rawInBufSemaphoreTimeout = 100, 
> > rawInBufSemaphoreFirstTimeout = 0, rawInBufDropped = 3, rawOutBuf = 
> > {theBuf = 0x80d53c70 "\r\n\r\nC3D 1.0.0 File Tranfer Test\r\nNeed a 
> > valid FAT12 file system on /dev/nvram0 (run fs_create).\r\n\r\nMount

> > DOSFS file system on /dev/nvram0 (run fs_create before)\r\nfsmount: 
> > mounting of \"/dev/nvram0\" to "..., Head = 413, Tail = 413, Size = 
> > 1024, Semaphore = 436273161}, t_dqlen = 0, rawOutBufState = 
> > rob_idle, device = {firstOpen = 0x17a8 <console_first_open>, 
> > lastClose = 0x1804 <console_last_close>, pollRead = 0, write = 
> > 0x16de <console_interrupt_write>, setAttributes = 0x172e 
> > <console_set_attributes>, stopRemoteTx = 0x1750 
> > <console_stop_remote_tx>, startRemoteTx = 0x177c 
> > <console_start_remote_tx>, outputUsesInterrupts = 1}, flow_ctrl = 1,

> > lowwater = 512, highwater = 768, rxTaskId = 0, txTaskId = 0, t_line 
> > =
> 0,
> > t_sc = 0x0, tty_snd = {sw_pfn = 0, sw_arg = 0x0}, tty_rcv = {sw_pfn 
> > =
> 0,
> > sw_arg = 0x0}, tty_rcvwakeup = 0}
> > 
> > 
> > Something to do with the absence of hardware flow control?
> > 
> > And now no matter what file I send, the problem appears at 12k for a
> > 1024 buffers (sooner for a 128 bytes buffer).
> > 
> > Etienne
> > 
> > 
> > 
> > 
> > 
> > -----Message d'origine-----
> > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > Envoyé : 5 novembre, 2004 09:56
> > À : Etienne Fortin; rtems-users at rtems.com
> > Objet : RE : RE : RE : Console problem update
> > 
> > 
> > Etienne,
> > 
> > a baud rate difference of 2% is still tolerable, as
> > fas as I know. And you would be well below.
> > 
> > I just had another look into the data structure
> > snapshot you sent this morning (last night...). To
> > my suprise, the "flow_ctrl" fiels is non-zero. The 
> > value 512 means, that the "input is controlled with 
> > XON/XOFF protocol". Maybe you should try to clear 
> > the "IXON" and "IXOFF" flags in the c_iflag field of 
> > termios settings?
> > 
> > wkr,
> > Thomas.
> > 
> > 
> > > Hi Thomas,
> > > Yes, the buffer is empty. But, it shows that at least 1161
> > > characters
> > > were received. After that, termios thinks there's no more data 
> > > received or it lost sync or something else.
> > > 
> > > I said before that my baudrate is really 9630 and not 9600. The 
> > > computer is 9600. Do you think that can be the problem?
> > > 
> > > Etienne
> > > 
> > > 
> > > 
> > > -----Message d'origine-----
> > > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > > Envoyé : 5 novembre, 2004 02:23
> > > À : Etienne Fortin
> > > Objet : RE : RE : Console problem update
> > > 
> > > 
> > > Etienne,
> > > 
> > > from the data you sent, the input and output buffer really is
> > > "empty", (for rawInBuf and rawOutBuf, "Head" and "Tail" have the 
> > > same values) but I guess you already saw this.
> > > 
> > > Which BSP are you using? Is it possible your console driver
> > > interrupt gets stuck, when there is a receive and transmit
interrupt
> 
> > > pending at the same time?
> > > 
> > > wkr,
> > > Thomas.
> > > 
> > > 
> > > > Hi Thomas,
> > > > Yes a VERY nasty problem... It must have something to do with my
> > > > comprehension and the way I implemented the console driver...
But,
> 
> > > > I
> > 
> > > > was able to make a lot of other things work before (I mean, the 
> > > > balance of
> > > > RTEMS) and I got nasty problem with RTEMS before. But that one 
> > > > is special. Its so stupid! And I hate to work with transfer 
> > > > protocol.
> > You
> > > > know you'll be able to make it work, you just don't know when...
> > > > 
> > > > Anyway, here's the snapshot of the rtems_termios_tty structure
> > > > when the semaphore get stuck.
> > > > 
> > > > $2 = {forw = 0x0, back = 0x0, refcount = 3, major = 1, minor = 
> > > > 0, isem
> > > 
> > > > = 436273159, osem = 436273160, cbuf = 0x80d542a4 "}", ccount = 
> > > > 0,
> > > > cindex = 0, column = 0, read_start_column = 0, termios =
{c_iflag 
> > > > = 9474, c_oflag = 4294967230, c_cflag = 2237, c_lflag = 32768, 
> > > > c_line
> > =
> > > > 0 '\0', c_cc =
> > > >
> > > "\003\034\177\025\004\n\001\000\021\023\032\000\022\017\027\026\00
> > > 0\
> > > 00
> > > 0"
> > > > }, vtimeTicks = 100, rawInBuf = {theBuf = 0x80d53a98 "", Head = 
> > > > 1161, Tail = 1161, Size = 2048, Semaphore = 436273162}, 
> > > > rawInBufSemaphoreOptions = 0, rawInBufSemaphoreTimeout = 100, 
> > > > rawInBufSemaphoreFirstTimeout = 0, rawInBufDropped = 0, 
> > > > rawOutBuf
> =
> > > > {theBuf = 0x80d543b0 "ransfer...\r\n\r\nC\006C.\r\nrawInBuf.Size

> > > > =
> 
> > > > 2048\r\nPress a key to begin t", Head = 17, Tail = 17, Size = 
> > > > 64,
> > > > Semaphore = 436273161}, t_dqlen = 0, rawOutBufState = rob_idle, 
> > > > device
> > > =
> > > > {firstOpen = 0x17a2 <console_first_open>, lastClose = 0x17ea 
> > > > <console_last_close>, pollRead = 0, write = 0x16dc 
> > > > <console_interrupt_write>, setAttributes = 0x172c 
> > > > <console_set_attributes>, stopRemoteTx = 0x174e 
> > > > <console_stop_remote_tx>, startRemoteTx = 0x1778 
> > > > <console_start_remote_tx>, outputUsesInterrupts = 1}, flow_ctrl 
> > > > = 512, lowwater = 64, highwater = 96, rxTaskId = 0, txTaskId = 
> > > > 0, t_line = 0, t_sc = 0x0, tty_snd = {sw_pfn = 0, sw_arg = 0x0},

> > > > tty_rcv = {sw_pfn =
> > > 0,
> > > > sw_arg = 0x0}, tty_rcvwakeup = 0}
> > > > 
> > > > As you can see, I "hardwired" the rawInBuf to be 2048 characters
> > > > in size. And surprisingly enough, the Head is at offset 1161.
This
> 
> > > > mean
> > 
> > > > that I received the first 130 bytes packet (init packet) and 
> > > > some
> > part
> > > 
> > > > of the first data packet. But for some reason, termios lost sync
> > > > and lost all the next characters, stucking the semaphore and 
> > > > giving me headaches :)
> > > > 
> > > > What do you think???
> > > > 
> > > > And by the way, you can't imagine how glad I am to have help 
> > > > from all of you on RTEMS forum.
> > > > 
> > > > Etienne Fortin
> > > > Sensio
> > > > 
> > > > 
> > > > 
> > > > -----Message d'origine-----
> > > > De : Thomas Doerfler [mailto:Thomas.Doerfler at imd-systems.de]
> > > > Envoyé : 4 novembre, 2004 16:09
> > > > À : Etienne Fortin; rtems-users at rtems.com
> > > > Cc : rtems-users at rtems.com
> > > > Objet : RE : Console problem update
> > > > 
> > > > 
> > > > Etienne,
> > > > 
> > > > can you provide a snapshot of the termios data structures when
> > > > your
> > > > console input gets stuck? What values do the buffer pointers
have 
> > > > for the raw input buffer?
> > > > 
> > > > What baud rate are you using currently?
> > > > 
> > > > What happens, if you write a simple application with two tasks,
> > > > one
> > > > sending a 1K buffer to the console, one receiving it, and
putting 
> > > > the console into loopback mode (with a nice little wire between
> pin
> > > > 2 and 3 of your serial connector)? Does it also hand in that 
> > > > case?
> > > > 
> > > > You really seem to have a nasty problem there...
> > > > 
> > > > I have been using termios code for machine interfaces back with 
> > > > RTEMS version 4.0.0, and I never had similar problems...
> > > > 
> > > > wkr,
> > > > Thomas.
> > > > 
> > > > 
> > > > > I don'T know why I tought that even after releasing the
> > > > > sempahore,
> > 
> > > > > I was still stuck in the idle thread. To the contrary! Forcing
> > > > > the
> > 
> > > > > release of the semaphore make the task continue to execute 
> > > > > code and
> > > I
> > > > > eventually get out of the code with an error (that's ok since
> > > > > the semaphore was forced to be released so no valid data is 
> > > > > there waiting)...
> > > > > 
> > > > > The question now is why the code is not notified of new 
> > > > > characters. Is
> > > > 
> > > > > it possible that the termios code, being designed for "human" 
> > > > > interraction (at least I presume), can't cope with sustained 
> > > > > binary data transfer and so I lost data and eventually the 
> > > > > semaphore is struck waiting for characters that will never 
> > > > > come?
> 
> > > > > Is it possible?
> > > > > 
> > > > > Etienne Fortin
> > > > > Sensio
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > -----Message d'origine-----
> > > > > De : Joel Sherrill <joel at OARcorp.com> 
> > > > > [mailto:joel.sherrill at OARcorp.com]
> > > > > 
> > > > > Envoyé : 4 novembre, 2004 15:21
> > > > > À : Etienne Fortin
> > > > > Cc : rtems-users at rtems.com
> > > > > Objet : Re: Console problem update
> > > > > 
> > > > > 
> > > > > Etienne Fortin wrote:
> > > > > > Hello all, its me again, the console guy :)
> > > > > > 
> > > > > > Little update for the ones that have an interest, or wish to

> > > > > > help me on that matter.
> > > > > 
> > > > > Is your interrupt an RTEMS ISR or a raw one directly connected
> > > > > to hardware?  If you do not go through the RTEMS ISR wrapper, 
> > > > > you
> > will
> > > > > NEVER schedule out of an interrupt.
> > > > > 
> > > > > Also are you sure you are geting the serial port interrupts?
> > > > > 
> > > > > > I tricked RTEMS to release the semaphore at each clock tick 
> > > > > > to
> 
> > > > > > make sure the semaphore was the problem. And surprise
> > > > > > surprise!
> > It
> > > 
> > > > > > still hang in the idle thread!!!!!!!!!
> > > > > > 
> > > > > > Theory: My task/scheduler/... config is fu***d up. Anyone 
> > > > > > here thinks it can be possible? For what I know of the 
> > > > > > problem so far, it
> > > seems
> > > > > > that as soon as the idle thread is entered, no other thread
> > > > > > ever
> > > > gets
> > > > > > executed... Really seems a config problem to me...
> > > > > > 
> > > > > > Idea someone?
> > > > > > 
> > > > > > Etienne Fortin
> > > > > > Sensio
> > > > > > 
> > > > > 
> > > > > 
> > > > > -- 
> > > > > Joel Sherrill, Ph.D.             Director of Research &
> > Development
> > > > > joel at OARcorp.com                 On-Line Applications Research
> > > > > Ask me about RTEMS: a free RTOS  Huntsville AL 35805
> > > > >     Support Available             (256) 722-9985
> > > > > 
> > > > > 
> > > > 
> > > > --------------------------------------------
> > > > IMD Ingenieurbuero fuer Microcomputertechnik
> > > > Thomas Doerfler           Herbststrasse 8
> > > > D-82178 Puchheim          Germany
> > > > email:    Thomas.Doerfler at imd-systems.de
> > > > PGP public key available at: http://www.imd-
> > > > systems.de/pgp_keys.htm
> > > > 
> > > > 
> > > 
> > > --------------------------------------------
> > > IMD Ingenieurbuero fuer Microcomputertechnik
> > > Thomas Doerfler           Herbststrasse 8
> > > D-82178 Puchheim          Germany
> > > email:    Thomas.Doerfler at imd-systems.de
> > > PGP public key available at: http://www.imd- 
> > > systems.de/pgp_keys.htm
> > > 
> > > 
> > 
> > --------------------------------------------
> > IMD Ingenieurbuero fuer Microcomputertechnik
> > Thomas Doerfler           Herbststrasse 8
> > D-82178 Puchheim          Germany
> > email:    Thomas.Doerfler at imd-systems.de
> > PGP public key available at: http://www.imd- systems.de/pgp_keys.htm
> > 
> > 
> 
> --------------------------------------------
> IMD Ingenieurbuero fuer Microcomputertechnik
> Thomas Doerfler           Herbststrasse 8
> D-82178 Puchheim          Germany
> email:    Thomas.Doerfler at imd-systems.de
> PGP public key available at: http://www.imd- systems.de/pgp_keys.htm
> 
> 

--------------------------------------------
IMD Ingenieurbuero fuer Microcomputertechnik
Thomas Doerfler           Herbststrasse 8
D-82178 Puchheim          Germany
email:    Thomas.Doerfler at imd-systems.de
PGP public key available at: http://www.imd- systems.de/pgp_keys.htm





More information about the users mailing list