Scenario.

You want to transfer large objects across the DDS by, for example, modifying the ping pong example to send an object of size 25MB.

Sending objects of this size over DDS without reconfiguring the OpenSplice daemon will cause several memory errors, ranging from networking service crashes to out of memory errors and more. In this article we will go through some configuration and application development tips for this scenario.

Shared Memory without Networking.

If both ping and pong are ran locally the example use only shared memory by removing the network service entries from the ospl.xml configuration. The shared memory database must be configured to accommodate the object you will be sending over DDS, if a you want to allocate/write/read samples of 25 MB you first need a Database/Size that is a multiple of 25MB. To handle all objects safely in the database allocate a sensibly high value, such as 250MB. Next the Database/Threshold must be modified.

The Database/Threshold is the minimum available free (shared) memory for any application related memory allocations to succeed, the Threshold should be at least the 25MB, but it is better to specify a multiple of 25MB.

More information on database configurations can be found in the OpenSplice Deployment Manual. The Configurator tool also describes available database parameters .

<Database>
<Size>750MB</Size>
<Threshold>75MB</Threshold>
</Database>

The Memory Statistics Tool is a very useful way to monitor the state of OpenSplice Shared Memory.

 Shared Memory with Networking.

In this scenario the networking service must be configured to handle large objects and application must be slowed down or adapted to relieve stress from the networking service. Failure to do so with respect to pingpong results in sending/receiving network buffers failing to clear before the next ping is sent, this backlog fills up the shared memory and creates out of memory errors in the ospl-error.log file and the messages will be lost.

When large data is sent, the networking service at the sender side divides the large message into smaller fragments. The size of a fragment is configurable via OpenSplice/NetworkService/Channels/Channel/FragmentSize, and is set by default to 1300 bytes. Sending 25MB would therefore require at least 20165 fragments. On the receiver side these fragments are stored in a buffer that can handle, by default for a Best Effort Channel, 5000 fragments. The size of this buffer (called the DefragBufferSize) is expressed in number of fragments and is configurable via OpenSplice/NetworkService/Channels/Channel/Receiving/DefragBufferSize. In order to be on the safe side (or in case you need to send slightly larger files) you can double, or triple that minimum value (i.e set DefragBufferSize to “20165 * 3”).

The pingpong example runs with minimal delay between pings, as it is designed for a small sample. To give the networking service time to clear its buffers it is recommended to manually slow the sending loop with a delay.

To prevent a memory error the Data Writer QoS policy can be modified to add a resource limit. This can be done through the max_samples QoS property:

p->get_default_datawriter_qos (dwQos);
dwQos.reliability.kind = DDS::BEST_EFFORT_RELIABILITY_QOS;
dwQos.history.kind = DDS::KEEP_LAST_HISTORY_QOS;
dwQos.resource_limits.max_samples = 3;

If this limit is reached and the ping application continues to attempt to send samples an out of resources error will be generated in the ospl-error.log file, but the shared memory will not fill up.

Another option is to limit the network queue size, by default set to 400, if the ping application continues to write samples the networking service fills the shared memory. Modify this to a lower value via OpenSplice/NetworkService/Channels/Channel/Sending/QueueSize.

Efficiency over time.

The ping pong applications run more efficiently as time goes on due to the internal mechanisms of OpenSplice adapting to the stresses placed on it. If sparse errors are still appearing running pingpong they should disappear after a few runs and the shared memory usage displayed in mmstat will also stabilise. The ping pong example was not originally designed to be used this way, hence these configuration options as well as the sending mechanism of ping.cpp will need to be tweaked until you achieve a satisfactory result.