Monday, October 23rd 2017, 8:37pm UTC+2

You are not logged in.

  • Login
  • Register

How to manage network buffers for incoming UDP packets?

Dear visitor, welcome to SEGGER Forum. If this is your first visit here, please read the Help. It explains how this page works. You must be registered before you can use all the page's features. Please use the registration form, to register here or read more information about the registration process. If you are already registered, please login here.

stefan.waldschmidt

Beginner

Date of registration: Apr 8th 2015

Posts: 8

Location: Darmstadt, Germany

Occupation: Software Developer

1

Tuesday, March 7th 2017, 10:42am

How to manage network buffers for incoming UDP packets?

Hi,

I observed a deadlock in by application because the IP-stack ran out of network buffers.

My solution is to make send non-blocking (see below)
but my question is if there is a better way to configure allocation strategy of network buffers.

Situation:
Assume a thread needs to receive and, from time to time, send UDP multicasts (for some kind of service announcement and discovery).
Most of the time, this thread is blocked in a select() call waiting for incoming network packets.
This select call is configured with a timeout, which, when fired causes the thread to send a UDP multicast by itself.

I observed a situation, where, between the thread being woken-up from select due to timeout and the thread reaching the socket's send call,
a lot of network packets arrive on the same socket (from other devices).
Those incoming network packets seem to consume all available network buffers.
This causes the send-call to block, for it can not allocate a network buffer to send out its message.
This situation causes not only to deadlock this thread but renders the entire IP-stack unusable, for there is not a single network buffer available anymore.

Could my observation be correct? Can this situation occur?
Is there a way to deal with it?
As a work-around, I configured the send to be non-blocking and skip sending out this single message.
Can I limit number of network packets consumed by incoming UDP (multicasts) packets to ensure packets are available for send-calls?

Thanks a lot
Stefan

SEGGER - Oliver

Super Moderator

Date of registration: Nov 14th 2007

Posts: 41

2

Tuesday, March 7th 2017, 2:59pm

RE: How to manage network buffers for incoming UDP packets?

Dear Stefan,

Quoted from "stefan.waldschmidt"

Could my observation be correct? Can this situation occur?
The short answer here is yes, if the PacketBuffer configuration is insufficient and/or does not match the configuration of the SocketBuffer.

By default an UDP SocketBuffer is configured to hold up to 2048 bytes. This can be changed for a socket via setsockopt(SO_RCVBUF).

The PacketBuffer configuration of the system should be prepared to handle the situation where the complete size of the SocketBuffer
consumes PacketBuffers, for of ALL sockets that can be open at the same time.

For UDP this is hard to calculate as each UDP message consumes one PacketBuffer. This means with 16 PacketBuffers in the System it
would be possible to consume all PacketBuffers by 16 incoming 1 byte UDP messages.
For TCP things work different. As TCP is a stream, new incoming data might get merged into existing packet buffers.

With embOS/IP V3.04 we have added checks that prevent most of the deadlock situations that might occur due to small PacketBuffer
configurations. However for UDP this is still tricky and hard to avoid.

Quoted from "stefan.waldschmidt"

Is there a way to deal with it?
As a work-around, I configured the send to be non-blocking and skip sending out this single message.
This would have been our first suggestion as well. The stack itself is in "blocking state" as no more data can be received (one packet is kept free on Rx, to ensure sending a TCP retransmit or ACK is still possible). In your case I assume the UDP send() call is blocking as the only free PacketBuffer is not big enough for your data to send.

Quoted from "stefan.waldschmidt"

Can I limit number of network packets consumed by incoming UDP (multicasts) packets to ensure packets are available for send-calls?
I will check if we can add this for the next version. As explained above, a good PacketBuffer configuration for UDP is hard to estimate.
Thank you for the suggestion.

As soon as I have something ready for testing I will let you know by mail.

Regards,
Oliver

stefan.waldschmidt

Beginner

Date of registration: Apr 8th 2015

Posts: 8

Location: Darmstadt, Germany

Occupation: Software Developer

3

Tuesday, March 7th 2017, 4:02pm

Hi Oliver,

thank you so much for your fast response.
Your explanation really helps me a lot to better understand what is going on.

And thanks for pointing out the SO_RCVBUF socket option I was not aware of.

So we can either live with the "non-blocking workaround" (which seems to work fine)
or do some math and fine-tune the buffer sizes, based on your explanations.

Thanks again
Stefan

SEGGER - Oliver

Super Moderator

Date of registration: Nov 14th 2007

Posts: 41

4

Tuesday, March 7th 2017, 4:08pm

Hi Stefan,

But please be aware that SO_RCVBUF only means a limitation of the socket buffer in bytes.
This will not prevent you from wasting packet buffers for like 1 byte UDP packets.

We will check what we can do about an optional, additional NumPackets limitation for UDP/RAW Rx socket buffers.
Stay tuned.

Regards,
Oliver

GemsTyn

Beginner

Date of registration: Jul 6th 2017

Posts: 8

5

Friday, July 7th 2017, 2:46pm

RE: RE: How to manage network buffers for incoming UDP packets?

Quoted from "SEGGER - Oliver"

Dear Stefan,

Quoted from "stefan.waldschmidt"

Could my observation be correct? Can this situation occur?
The short answer here is yes, if the PacketBuffer configuration is insufficient and/or does not match the configuration of the SocketBuffer.

By default an UDP SocketBuffer is configured to hold up to 2048 bytes. This can be changed for a socket via setsockopt(SO_RCVBUF).

The PacketBuffer configuration of the system should be prepared to handle the situation where the complete size of the SocketBuffer
consumes PacketBuffers, for of ALL sockets that can be open at the same time.

For UDP this is hard to calculate as each UDP message consumes one PacketBuffer. This means with 16 PacketBuffers in the System it
would be possible to consume all PacketBuffers by 16 incoming 1 byte UDP messages.
For TCP things work different. As TCP is a stream, new incoming data might get merged into existing packet buffers.

With embOS/IP V3.04 we have added checks that prevent most of the deadlock situations that might occur due to small PacketBuffer
configurations. However for UDP this is still tricky and hard to avoid.

Quoted from "stefan.waldschmidt"

Is there a way to deal with it?
As a work-around, I configured the send to be non-blocking and skip sending out this single message.
This would have been our first suggestion as well. The stack itself is in "blocking state" as no more data can be received (one packet is kept free on Rx, to ensure sending a TCP retransmit or ACK is still possible). In your case I assume the UDP send() call is blocking as the only free PacketBuffer is not big enough for your data to send.

Quoted from "stefan.waldschmidt"

Can I limit number of network packets consumed by incoming UDP (multicasts) packets to ensure packets are available for send-calls?
I will check if we can add this for the next version. As explained above, a good PacketBuffer configuration for UDP is hard to estimate.
Thank you for the suggestion.

As soon as I have something ready for testing I will let you know by mail.

Regards,
Oliver

Quoted from "SEGGER - Oliver"

I think this is the best answer for your query

Quoted from "SEGGER - Oliver"


Similar threads