RING BUFFER 18LF4620 Oshonsoft

Hi,
I'm learning INTERRUPTS on 18LF4620 PIC and Oshonsoft.

So far I've been using linear buffers, but a couple of friends suggested ring buffers, and one of them has been helping me write CODE for Oshonsoft.

What are the pros and cons of ring buffers over linea ones? On the PIC there is a limited amount of memory, and wonder if ring buffers, use more?

A search found this from 'E', but it is for a different reason: https://www.electro-tech-online.com...ps-bmp280-hmc5983.150378/page-25#post-1316563 #484

Here's another:https://www.electro-tech-online.com/threads/pic16f877a-prevent-queuing-interrupts.153654/page-3 This one is written in C, so I can't quite follow it!

Cheers, Camerart.

Ring buffers are far simpler to use than linear, once you get your head around the concept.

I always use power-of-two sizes, which means you can just AND the pointers with an appropriate mask after each increment to make them "wrap" in the buffer space.

You then never have to be bothered with the absolute positions within the buffer.

eg. for a 32 location ring buffer, use a 32 location array, then AND the index pointers with 0x1F after each increment.

rjenkinsgb said:

Ring buffers are far simpler to use than linear, once you get your head around the concept.

I always use power-of-two sizes, which means you can just AND the pointers with an appropriate mask after each increment to make them "wrap" in the buffer space.

You then never have to be bothered with the absolute positions within the buffer.

eg. for a 32 location ring buffer, use a 32 location array, then AND the index pointers with 0x1F after each increment.

Click to expand...

Hi R,
Ok, than I'll try to adopt ring buffers.

Can you read Oshonsoft? If so, my friend has given me a couple of pages in longhand, that I'm slowly putting into Oshonsoft format, which will help me understand the idea, and I'll post once working.
Thanks,
C.

camerart said:

Can you read Oshonsoft?

Click to expand...

I've never used it, but from fragments I've seen it seems straightforward.

From a quick look at the language reference, to make eg. a receive buffer:

Dim rxbuf_in as byte
Dim rxbuf_out as byte
Dim rxbuffer(16) as byte

In your initialisation section:

rxbuf_in = 0
rxbuf_out = 0

Buffer in routine, eg. in your receive interrupt:

rxbuffer(rxbuf_in) = recievedata_value, wherever that comes from
rxbuf_in = rxbuf_in + 1
rxbuf_in = rxbuf_in And %00001111

Optionally, check for overflow at that point
If rxbuf_in = rxbuf_out then
error, do something.....
Endif


In the main programs loop, check for new characters:

if rxbuf_out <> rxbuf_in then
Some data in the buffer do something with it

newdata = rxbuffer(rxbuf_out)
rxbuf_out = rxbuf_out + 1
rxbuf_out = rxbuf_out And %00001111

Do whatever you need with newdata...

Endif


Or you can wait until the recieve character matches a marker (using another If comparison on the received data after storing a copy), a carriage return or whatever, then set another variable to tell the main program to copy the content from the buffer..
There are many ways to handle such things.

rjenkinsgb said:

I've never used it, but from fragments I've seen it seems straightforward.

From a quick look at the language reference, to make eg. a receive buffer:

Dim rxbuf_in as byte
Dim rxbuf_out as byte
Dim rxbuffer(16) as byte

In your initialisation section:

rxbuf_in = 0
rxbuf_out = 0

Buffer in routine, eg. in your receive interrupt:

rxbuffer(rxbuf_in) = recievedata_value, wherever that comes from
rxbuf_in = rxbuf_in + 1
rxbuf_in = rxbuf_in And %00001111

Optionally, check for overflow at that point
If rxbuf_in = rxbuf_out then
error, do something.....
Endif


In the main programs loop, check for new characters:

if rxbuf_out <> rxbuf_in then
Some data in the buffer do something with it

newdata = rxbuffer(rxbuf_out)
rxbuf_out = rxbuf_out + 1
rxbuf_out = rxbuf_out And %00001111

Do whatever you need with newdata...

Endif


Or you can wait until the recieve character matches a marker (using another If comparison on the received data after storing a copy), a carriage return or whatever, then set another variable to tell the main program to copy the content from the buffer..
There are many ways to handle such things.

Click to expand...

Hi R,
If you don't mind, I won't read your program, as it will confuse me.

So far, my program makes the 1st buffer, inside the INTERRUPT, and now I'm looking at the MAIN LOOP where the second buffer/STRING will be.
I'll post it when I get farther down the program.
C.

camerart said:

my program makes the 1st buffer, inside the INTERRUPT, and now I'm looking at the MAIN LOOP where the second buffer/STRING will be.

Click to expand...

There should only be one buffer, if you have two then something is horribly wrong.

The code above (rjenkingsgb) is short and very easy to follow.

Mike.

Pommie said:

There should only be one buffer, if you have two then something is horribly wrong.

The code above (rjenkingsgb) is short and very easy to follow.

Mike.

Click to expand...

Hi M,
I can see from a quick scan that it isn't complicated, but sadly, I don't find it easy to understand, I would have to slowly feed it through the Oshonsoft simulator and build it up step by step, which would take me days.
Cheers, C.

It's called a ring buffer because the end is joined to the start - not physically but in code. Think of it as a snake with a head and tail - and kinda eating itself. Initially, the snake has no length - the buffer is empty so head and tail have the same value. When a value arrives we add it to the front of the buffer (the head) and move the head (pointer) along one. In our main code we check the length of the snake (length=head-tail) and if it's not zero we can remove a byte from the tail end and move the tail along one. If we're tied up doing something else then the snake can just grow longer. Maybe not a good analogy but if you can visualize it in a different way then it may help you.

Mike.

Pommie said:

It's called a ring buffer because the end is joined to the start - not physically but in code. Think of it as a snake with a head and tail - and kinda eating itself. Initially, the snake has no length - the buffer is empty so head and tail have the same value. When a value arrives we add it to the front of the buffer (the head) and move the head (pointer) along one. In our main code we check the length of the snake (length=head-tail) and if it's not zero we can remove a byte from the tail end and move the tail along one. If we're tied up doing something else then the snake can just grow longer. Maybe not a good analogy but if you can visualize it in a different way then it may help you.

Mike.

Click to expand...

Hi M,
Each morning I talk to mates on a local radio net, and one of them wrote 'how to do it' long hand.
I've been questioning him for a day or two, and moving forward. This morning after another explanation, I said exactly what you said, 'It's like a snake following it's tail, with the INTERRUPT adding to it's head and the MAIN LOOP eating it's tail' and if the head catches the tail, then the BUFFER isn't long enough. As this does help me visualise it, I'll call the VARIABLES head and tail
C

In my code, the variables are also called head and tail. Visualization can often let you see what is happening so much more.

Good luck with your project.

Mike.

Hi,
It's been pointed out to me that a RING BUFFER would 'eat' too much memory for my project (PIC), so I've reverted back to a LINEAR BUFFER.

As with the RING BUFFER, I am able to WRITE the LINEAR BUFFER until it was over writing the previous digits , as expected, by incrementing the BUF(VARIABLE), but I'm now having difficulties READing the digits OFF the saved BUFFER. How do I increment throught the saved buffer, without incrementing and affecting the new digits being saved?
C.

camerart said:

Hi,
It's been pointed out to me that a RING BUFFER would 'eat' too much memory for my project (PIC), so I've reverted back to a LINEAR BUFFER.

Click to expand...

Why would it be any different?, a ring buffer is no different, it's simply how you access it.

I agree... A ring buffer can be smaller than a linear one.. It's how you service the buffer that counts... Ring buffers work extremly well with software / hardware hand shaking as you can control the input speed to coinside with your processing speed..

I've been racking my brains.. I'm sure I did a ring buffer in Oshonsoft for you sometime since... But I cannot find it.

Ian Rogers said:

I agree... A ring buffer can be smaller than a linear one.. It's how you service the buffer that counts... Ring buffers work extremly well with software / hardware hand shaking as you can control the input speed to coinside with your processing speed..

I've been racking my brains.. I'm sure I did a ring buffer in Oshonsoft for you sometime since... But I cannot find it.

Click to expand...

Hi N and I,
I, I don't think I can control the DATA input speed, (GPS MODULE) (I'll look for your previous CODE)

I'm on two forums, a member from the other one, who has helped me a great deal with the project I'm working on, reminded me that the PIC I'm using may run out of memory if I'm not careful.

Regarding Linear or Ring Buffers, I'm having difficulty with both of them.

I was advised to make the Ring buffer twice as long as the longest sentence, so I complied, but my logic said, that as the MAIN LOOP must be faster that the DATA input, then the MAIN LOOP shold overtake the DATA writing, so a long buffer shouldn't be needed.

With both types, I'm unable to increment through and READ the saved DATA without affecting the incoming DATA being WRITTEN.
C

Ian Rogers said:

I've been racking my brains.. I'm sure I did a ring buffer in Oshonsoft for you sometime since... But I cannot find it.

Click to expand...

Hi I,
I couldn't find it either, I do have a Buffer from a while ago, but it is kind of different, so not really useful for this.

I have a few examples, to look at, but they confuse me, especially the READ WRITE INCREMENT point from the previous post.
C.

This is what MCC generates (in C) as a serial ring buffer interrupt routine:


To read bytes out of the buffer, it generates this:


I know it's in C, but it should be fairly easy to understand.

A buffer - eusart1RxBuffer[]

Head variable - eusart1RxHead

Tail variable - eusart1RxTail

And a count of how many bytes are in the buffer - eusart1RxCount (used to check if there's anything in the buffer or not)

Nigel Goodwin said:

This is what MCC generates (in C) as a serial ring buffer interrupt routine:

C:

else if(PIE3bits.RC1IE == 1 && PIR3bits.RC1IF == 1)
        {
            if(1 == RC1STAbits.OERR)
            {
                // EUSART1 error - restart

                RC1STAbits.CREN = 0;
                RC1STAbits.CREN = 1;
            }
            // buffer overruns are ignored
            eusart1RxBuffer[eusart1RxHead++] = RC1REG;
            if(sizeof(eusart1RxBuffer) <= eusart1RxHead)
            {
                eusart1RxHead = 0;
            }
            eusart1RxCount++;
        }

To read bytes out of the buffer, it generates this:

C:

unsigned char EUSART1_Read(void)
{
    unsigned char readValue  = 0;
  
    while(0 == eusart1RxCount)
    {
    }

    readValue = eusart1RxBuffer[eusart1RxTail++];
    if(sizeof(eusart1RxBuffer) <= eusart1RxTail)
    {
        eusart1RxTail = 0;
    }
    PIE3bits.RC1IE = 0;
    eusart1RxCount--;
    PIE3bits.RC1IE = 1;

    return readValue;
}

I know it's in C, but it should be fairly easy to understand.

A buffer - eusart1RxBuffer[]

Head variable - eusart1RxHead

Tail variable - eusart1RxTail

And a count of how many bytes are in the buffer - eusart1RxCount (used to check if there's anything in the buffer or not)

Click to expand...

Hi N,
Thanks for the CODE, but from experience, I can be pretty sure it would take me weeks to convert to Oshonsoft, and I'm a bit knackered after trying with other examples.

I'll have a break and come back to it later though.
Cheers, C.

camerart said:

Hi N and I,
I, I don't think I can control the DATA input speed, (GPS MODULE) (I'll look for your previous CODE)

I'm on two forums, a member from the other one, who has helped me a great deal with the project I'm working on, reminded me that the PIC I'm using may run out of memory if I'm not careful.

Regarding Linear or Ring Buffers, I'm having difficulty with both of them.

I was advised to make the Ring buffer twice as long as the longest sentence, so I complied, but my logic said, that as the MAIN LOOP must be faster that the DATA input, then the MAIN LOOP shold overtake the DATA writing, so a long buffer shouldn't be needed.

With both types, I'm unable to increment through and READ the saved DATA without affecting the incoming DATA being WRITTEN.
C

Click to expand...

When you have a complete message in the buffer, can't you switch the Usart to another message source and write it's characters to second buffer.
Meanwhile the main program can parse the completed message or do something useful.

jjw said:

When you have a complete message in the buffer, can't you switch the Usart to another message source and write it's characters to second buffer.
Meanwhile the main program can parse the completed message or do something useful.

Click to expand...

Hi J,
Multiple buffers have been suggested before, and we seem to have got by without them, but now considering my puzzle, this may be the answer, so I'll give it a try.

The above would work with linear buffers, but do you prefer LINEAR or RING BUFFERS? Keeping in mind memory usage.
C.

C, it does not matter much what kind of buffer you use, it is how you handle the interrupt and the data it gets. Think in time frames of 1 millisecond (mS)

First, try to get the interrupt RX routine to determine if the received characters are valid, and in the proper position in the string. For example, if you expect "$" as the first character and "A" as the 10th character, then the interrupt routine keeps track of what position your RX buffer is at, and if the expected character is not "correct", you zero the pointers/length and just go onto reading the next character. If length pointer happens to go past the buffer length (maximum message length), there is something wrong, and simply zero your pointers and counter in the interrupt routine and wait for next character. Do so until you get the right data stream and proper terminator. At that point, you set a flag that the UART RX buffer has a valid string and is complete. Nothing else!
All this can be done in the interrupt routine in the order of micro seconds (uS), maybe as much as 100uS for slower clock rates. That is 1/10th the rate at which characters arrive at the UART at 9600 baud (approx 1mS each). Keep it simple, use lots of flags and pointers, and parse the incoming characters as "valid" ONLY, do not interpret the string itself, that is for the main program to do! You basically filter the RX characters for valid data format only within the interrupt routine. Anything invalid, you reset things within the interrupt routine.
You have to think in terms on microseconds, a chip can do hundreds, if not thousands of instructions between each UART character. Your main program can then parse the RX buffer when it can, usually within the time frame before the next incoming string is complete and valid. After that point, the interrupt routine ignores all RX characters until the main program has processed the message and cleared the flags.

For example, I have an RX interrupt routine. looking for a string of characters. First I check OERR, FERR, and such at start of interrupt routine. First character has to be "FE", and so on, 5th character has to be a "03". End of message is "FD" If any of those requirements are not correct, I zero my pointers and wait for the proper message within the interrupt routine. Simple...
Now, I do something different in the main loop. Assume I have a valid RX buffer of 20 characters. Flags are set to show the length of message, and string is complete and correct format by the interrupt routine. Main program now detects valid message via those flags and I save the pointers/length into new variables, and start to copy the RX buffer into another buffer, a "working buffer" Since copying characters from buffer to buffer is so fast (<20uS at 8Mhz clock), I can simply transfer 5 or 10 characters and then reset the interrupt flags/pointers while I copy the rest of the working buffer. This way, I allow more messages to be received by the interrupt routine while I still work on the work buffer in the main program loop. This is perfectly safe, as it will take a whole millisecond (mS) before the interrupt routine will trigger again, and for one character only. Next character will take another whole 1mS. By then, I've saved the entire working buffer, and main program can take its time processing it. No interaction between RX interrupt routine and its RX buffer vs the main program loop and its "working buffer"

I know this can be overwhelming to some. You have to think in terms of microprocessor speeds, and your UART speeds (or other data). The micro can process so much data between UART characters, you just have to think like a microprocessor to get the timing right, and see that 1mS for a UART character is a very, very long time.