Memory Claim Denied Error

Memory claim denied errors may appear in the OpenSplice log file. This is a symptom of running out of shared memory. These are the steps to follow to troubleshoot this problem.

Memory Claim Denied Error in ospl.error log

OpenSplice DDS can runs in two modes, federated or shared memory mode and single process memory mode. When using shared memory mode you may occasionally see unexpected behaviour. The following log messages appear in the opsl.error log file.

========================================================================================
Report : ERROR
Date : Wed Apr 05 18:17:02 UTC 2000
Description : Memory claim denied for small object of 64 bytes: request exceeds available matching resources (approximately 0 bytes). To remedy this situation try to configure a bigger value for the OpenSplice/Domain/Database/Threshold value, most likely a database size is configured that is bigger than the default without adapting the default threshold value accordingly (hint: likely the solution if memory claim is denied while deleting large amounts of entities/data or when using very big topic sizes). Alternatively the value configured for the OpenSplice/Domain/Database/Size value has to be increased (hint: likely the solution if memory claim is denied while receiving a lot of data).
Node : cpu2-2
Process : Process1 <12549>
Thread : watchdog 7f7dfcbae700
Internals : V6.5.0p9/a3b348e/64b93f0/c_mmbase/c_mmbaseAdv.c/1564/0/954958622.466426306

Troubleshooting the Memory Claim Denied Error Message

Increase the amount of shared memory available

Check that OpenSplice has enough shared memory available to deal with the data sizes you are working with.

The size of the shared memory is set in the xml configuration file. Look for the Database element under the Domain section.

<Domain>
<Database>
<Size>10485760</Size>
</Database>
</Domain>

The default value is 10485760 bytes or 10 MB. You can increase this value by changing this setting. The value can be set in bytes. You can use values including K for Kilobytes, M for Megabytes or G for Gigabytes. For example 10M instead of 10485760.

You may need to configure your operating system to support the requested size. On most platforms you need ‘root’ privileges to set large sizes.

If you have enough shared memory available you should look into why the Domain is using more shared memory than expected. This requires some investigation on the application side.

Check your application is dealing with instances correctly

When adding instances into a system you need to remember that at some point these need cleaning up. If you keep adding instances into the system and do not do anything to get rid of the irrelevant ones you will eventually run out of memory. To avoid this you need to explicitly tell the system which instances should remain available and which are no longer relevant.

To ensure that OpenSplice can reclaim memory correctly the readers and writers should do the following:

On the writer side you should unregister all instances which you no longer intend to update.
Writers should also dispose and unregister all transient instances that are no longer relevant to the rest of the system.
The reader needs to take all instance data that is marked DISPOSED or NO_WRITER. Please note that DISPOSED and/or UNREGISTERED instances don’t clean up themselves from your readers automatically. This has to be done manually by taking them out. You can also set the the auto_purge timers on the ReaderDataLifecycleQosPolicy to remove them automatically after some time interval elapses.

Further Troubleshooting for the Memory Claim Denied Error

If you have done above and are still seeing problems with the memory claim denied error message then you are probably allocating resources faster than you are releasing them. This means the total number of instances/samples in the system will be continually increasing. To see if this is the case you can use mmstat. This tool can provided details about the total number of instances and the total number of samples in the system.

If you do see that the number of instances is continually increasing then your writer may be updating samples at a faster rate than the readers can consume them. If you are using a HISTORY QOS of KEEP_LAST this may not be a problem since newer instances will overwrite older instances. If you have a HISTORY QOS of KEEP_ALL then all the instances are kept and the reader will get further and further behind.

There are a couple of things you can try:

Slowing down the writer or increasing the speed that the reader is consuming the data
Set instance bounds on the reader and writer side. This allows you to block a writer from creating additional instances until current resources are reclaimed. On the reader side it will prevent a reader from reading any more until the existing information has been dealt with. By default the reader will buffer as much as it wants. If the writer is writing faster than the reader is reading then you will not hit the writer bounds as the writer will only start buffering after the reader reaches its limit.

Setting a Memory Threshold

In normal use you should not see a Memory Claim Denied error. When they do occur they happen during the middle of an update. It is better to make an memory estimate before an update begins and prevent it happening if there is not enough memory available.

To do this you can set the OpenSplice/Domain/Database/Threshold value in the xml configuration file. This checks the available memory before doing an update. If enough memory is available the modification will take place successfully. If the modification will take more memory than there is available it will not happen The error code RETCODE_OUT_OF_MEMORY will be thrown. This can be caught by the application and the Memory Claim Denied error will not occur.

The Threshold is set in the xml file as follows:

<Database>
<Size>10485760</Size>
<Threshold>1048576</Threshold>
</Database>

It is advisable to use the OpenSplice configuration tool to modify the xml file as this will check the file for errors.

By default the Threshold is set to 1MB. This is about 10% of the default Database size and is a sensible value. However, if you increase your database size, eg to 100MB, without modifying your Threshold, it only covers about 1% of the relative database size. This may be on the low side. As a general rule don’t make the Threshold lower than 1MB and leave it between 5 – 10% of your database size. When using big samples it will need to be higher, when using smaller samples it can be lower.