Resource constraint thresholds

In many systems, it's important to guard against clumsy or malicious applications that attempt to exhaust global resources.

A system likely has a “core” or “critical” part that consists of essential services and has some bounded (but not necessarily constant) resource consumption. The rest of the system could have unbounded resource consumption, either because it's designed to handle problems with inherently unbounded resource usage, or because the processes can't be relied upon to bound their resource usage.

Resource constraints ensure that the core can operate even if the extended system is maximizing its resource usage. The core system might have a monitoring capability that allows it to reset the extended system in some way, but that's not essential. The most obvious such resource is system RAM, but the same considerations apply to process table entries, and to resource manager connections (scoids).

A certain level of protection is provided by RLIMIT_FREEMEM (see setrlimit()), which prevents applications from allocating memory once the system drops below a certain amount of free memory. Most programs will give up once malloc() starts failing, so this provides a reasonable protection againt inadvertent exhaustion, but this solution isn't complete:

• It doesn't reserve other types of resources, such as process IDs and service connection IDs (scoids).
• It doesn't restrict allocations by proxy, where a process induces a resource manager or the kernel to allocate memory (or other resources) on its behalf. This is difficult to resolve, since the kernel and resource managers can't be limited in the same way as applications, because they're required to ensure functioning of the core system in a resource-exhaustion situation.

The idea is to identify the critical processes that form the core system, and give them the ability to allocate as many resources as they need. All other processes are resource-constrained. For each resource that is prone to exhaustion, a threshold is defined, and a constrained process is refused allocation if the amount of free resources is below the threshold. The following system limits specify the constraints (see sysconf()):

_SC_RCT_MEM

The minimum amount of memory that the system retains as a resource constraint threshold.

_SC_RCT_SCOID

The minimum number of server connection IDs (scoids) that a server retains as a resource constraint threshold.

The proxy resource-manager case is handled by communicating the client's status to the server using a bit in the _msg_info structure. The server can then temporarily constrain itself while handling all or part of a request from a constrained client.

The kernel case is handled by having the kernel consider whether the client is constrained or unconstrained when it's asked to allocate a resource.

The ability PROCMGR_AID_RCONSTRAINT (see procmgr_ability()) is given to the critical processes only, and allows them to operate without being subject to resource constraint thresholds. A ThreadCtl() command, _NTO_TCTL_RCM_GET_AND_SET, allows a thread to constrain itself or free itself from constraint, but if the thread is a member of a process that lacks the PROCMGR_AID_RCONSTRAINT ability, it's effectively constrained regardless.

To handle resource-constraint modes:

• If the server isn't a critical process (as defined above), it can run in constrained mode, and no special code is required.
• If the server is a critical process, but can service requests from the core system without allocating any resources (directly or indirectly), then it can run in constrained mode, and no special code is required.
• If the server is a critical process and may have to allocate resources in order to handle a request from the core system, it must run in unconstrained mode. This means that the server has the responsibility to ensure that it doesn't allow itself to be used by constrained clients to allocate a resource in excess of the currently defined resource-constraint threshold. Unless the server is actually managing the resource itself, compliance generally means adopting the client's constraint mode when handling a request. This can be accomplished in these ways:
  • Automatically, by setting RESMGR_FLAG_RCM in the resmgr_attr_t structure (see the entry for resmgr_attach() in the QNX Neutrino C Library Reference). This is applicable only to servers using the resource-manager framework:

    resmgr_attr.flags |= RESMGR_FLAG_RCM;
    resmgr_attach(dpp, &resmgr_attr, name, _FTYPE_ANY, 0, &connect_funcs,
                  &io_funcs, &io_attr))
        

  • Manually, by checking for _NTO_MI_CONSTRAINED in the flags member of the _msg_info structure, available from a call to MsgReceive() or MsgInfo(). If this bit is set, the message was received from a constrained client. At appropriate moments, a thread that allocates resources on behalf of a constrained client should constrain itself using ThreadCtl():

    int value = 1; // 1 to constrain, 0 to remove constraint
    ThreadCtl(_NTO_TCTL_RCM_GET_AND_SET, &value); /* swaps current state with value */
    
    /* Handle the request... */
    
    ThreadCtl(_NTO_TCTL_RCM_GET_AND_SET, &value); /* restores original state */
        

When a server runs as a constrained process, or when it constrains one of its threads, it may find that its resource allocation requests fail when there are still resources available. The server is expected to handle these failures in the same way it would handle a failure caused by complete exhaustion of resources, generally by returning an error to the client. For the sake of overall system stability, it's important for servers that can continue to process messages to do so, even when allocation failures occur.