This article explains how you can configure shared memory in OpenSplice DDS.  There may be occasions when you need to alter the default settings.

OpenSplice DDS can run in two different architectural modes. These are:

• Single process or standalone deployment
• Federated or shared memory mode

Shared Memory Mode

Each DDS application will then interface with the shared memory database.  The data is only present once on any machine. This makes the reading and writing of data more efficient. The OpenSplice  services are able to see all the data on the node. This allows them to make decisions about data delivery.

Starting OpenSplice in Shared Memory Mode

ospl -h
ospl -v
ospl [-e] [-f] start [URI]
ospl [-e] [[-d <domain> | -a] stop [URI]]
ospl [-e] list
ospl [-e] status [URI]

To start OpenSplice use the command:

ospl start

What the different command line options do

The meaning of these commands is as follows:

-h  – lists out the meaning of the different command line options

-v  – shows the ospl version information

start  – Start the identified system

The system is identified and configured by the URI which is defined by the environment variable OSPL_URI. This setting can be overruled with the command line URI definition. When none of the URI definitions is specified, a default system will be started. Upon exit an exit code will be provided to indicated the cause of termination. The following exit codes are supported:

• 0 : normal termination as result of OSPL stop.
• 1 : a recoverable error occurred.  – The system has encountered a runtime error and has terminated. A restart of the system is possible. E.g., the system ran out of resources.
• 2 : an unrecoverable error occurred. – The system has encountered an error that cannot be resolved by a restart of the system. E.g., the configuration file contains errors or does not exist.

When the -f option is specified the ospl command will not return directly, but it will instead block until termination of the OpenSpliceDDS deamon. If the -f option is not specified the ospl command will return immediately after start up.

stop – Stop the identified system

The system to stop is identified by the URI which is defined by the environment variable OSPL_URI. This setting can be overruled by the command line URI definition or the domain name which is associated with the URI and specified via the -d option. When no domain is specified by the URI or by it’s name a default system is assumed. The -a options specifies to stop all running splice systems started by the current user.
Upon exit an exit code will be provided to indicated the cause of termination. The following exit codes are supported:

• 0 : normal termination as result of OSPL stop.
• 1 : Not Applicable
• 2 : an unrecoverable error occurred. – The system has encountered an error that cannot be resolved by a retry of the same command. E.g., A faulty URI was provided.

list – Show all systems started by the current user

status  – Check whether the identified system is currently running The system is identified and configured by the URI which is defined by the environment variable OSPL_URI. This setting can be overruled with the command line URI definition. When none of the URI definitions is specified, a default system will be checked.

-e When the -e option is specified, then the exit codes of ospl will have extended values compared to 0/1/2:

• 0 : The domain is operational or normal/successful exit.
• 1 : The domain is already operational.
• 2 : The domain is present but is initializing.
• 3 : The domain is present but is terminating.
• 16 : The domain is non existent.
• 31 : Recoverable error, which a re-invoke/restart might fix.
• 32 : General tool failure, like invalid command line options or unexpected internal error.
• 33 : Unrecoverable error, which a re-invoke/restart will not fix.
• 34 : A configuration has been specified that is inappropriate for the command issued.
  The domain as specified in the configuration.

Configuring Shared Memory

Configuration of OpenSplice is done with an xml file.  Any changes to the configuration need to be made in the opsl.xml file that is in use in the system. Changes should be made with the OpenSplice Configuration Tool as this will check the xml file to ensure it is valid.  Configuration of shared memory is done in the Database element in the Domain section of the xml file.

Database

For shared memory configuration to work this section must exist in the xml file.

<Domain>
<Database>
<Size>10485760</Size>
</Database>
</Domain>

Size

This section of the xml file specifies the size of the shared memory segment holding the database.

The default value is 10485760 bytes or 10 MB.  You can increase this value by changing this setting.  The value can be set in bytes.  You can use values including K for Kilobytes, M for Megabytes or G for Gigabytes. For example 10M instead of 10485760.

If your system requires more memory than the default you can increase this value.  You may need to configure your operating system to support the requested size. On most platforms you need ‘root’ privileges to set large sizes.

You may need to investigate changing this setting if you are seeing a Memory Claim Denied error or if you run out of memory when sending large topics.

Address

It is useful to use a Memory Mapper to help select the value for the memory address.

• Windows : VMmap (Sysinternals)
• Linux     : pmap –x <pid>

OpenSplice stores absolute pointers in shared memory and not relative pointers, this being better for performance reasons. As a result the shared memory segment needs to be mapped into the same virtual memory address of all processes on that machine that use OpenSplice (otherwise pointers are corrupted). This means the virtual memory address range between the configured address and the configured address + configured shared memory size needs to be available in all processes that use OpenSplice. When (part of) the virtual memory range is already used for different purposes (for instance to map shared libraries) OpenSplice and/or any application(s) will not start as mapping in the shared memory will fail. One other well-known application of shared memory that uses the exact same way as OpenSplice is the Java Virtual Machine which also maps in a shared memory segment at start-up at a fixed virtual memory address.

When the OpenSplice service manager tool starts it obtains an address from the configuration file. This determines the start address in the process’ virtual address space where the shared memory segment is mapped.  Each process that attaches to the current Domain gets mapped into this memory space.

There may be times when you need to change this memory address.  For example you may see OpenSplice not being able to start and errors in the log file similar to this:

ERROR [0x000006e4] opensplice – Failure to open the domain, domain id = 0 and URI=”<NULL>” The most common cause of this error is that OpenSpliceDDS is not running (when using shared memory mode). Please make sure to start OpenSplice before creating a DomainParticipant.
ERROR [0x000006e4] opensplice – os_sharedMemoryAttach failed for domainId (0) and domainName (ospl_shmem_t3k)
ERROR [0x000006e4] opensplice – Can not Map View Of file: Attempt to access invalid address.

These errors are seen because the default address space is already in use on that node. The OpenSplice deamon is not able to use the address to create the shared administration and the database. The error can also occur after OpenSplice has been started. When a new application is started OpenSplice will map this into the database. If there is not enough free memory after the default address then this can fail. The problem can also be seen if a large database size has been specified causing the memory segment boundary to be crossed.

To change the start address of the virtual memory space you need to add the option

<Address>0x200000</Address>

to the configuration xml file into the section

/Name>
<Database>
<Size>10485670</Size>
</Database>

This address must be the same for each process communicating within a domain.  The possible default values are platform dependent.

Threshold

The Threshold value is set to indicate a value of shared memory that must be kept available.  If the Threshold value is set the amount of available memory is checked before an update is done. If there is less free shared memory than the value indicated by the threshold then no new allocation will be allowed.  The update will not occur and the error code RETCODE_OUT_OF_MEMORY will be thrown.

Locking