Learning
Documentation
Community
Open Exchange
Global Masters
Home / Class Reference / %SYS namespace / %SYSTEM.Cluster
Private  Storage   

%SYSTEM.Cluster


class %SYSTEM.Cluster extends
%SYSTEM.Help

This class provides an API for node-level configuration of sharded clusters.

It can be used via the special $SYSTEM object, for example:

set status = $SYSTEM.Cluster.Initialize()

API Usage

Call Initialize to set up an InterSystems IRIS instance as the first node of a new nonmirrored sharded cluster (that is, a sharded cluster in which all data nodes are nonmirrored).

Call InitializeMirrored to set up an InterSystems IRIS instance as the first node of a new mirrored sharded cluster (that is, a sharded cluster in which all data nodes are mirrored).

Call AttachAsDataNode to add an instance to a cluster as a nonmirrored data node.

Call AttachAsMirroredNode to add an instance to a cluster as a mirrored data node.

Call AttachAsComputeNode to add an instance to a cluster as a compute node.

Call ListNodes to print a list of a cluster's nodes.

Call GetMetadata to retrieve an overview of a cluster's metadata.

Call ClusterNamespace to get the name of the cluster namespace for the current instance.

Terminology

Sharded Cluster
An interconnected set of nodes which provides horizonal scalability of users and data.
Node
An InterSystems IRIS instance that participates in a sharded cluster. Nodes are of two types, "data" and "compute", but all nodes can b used interchangeably as application servers, providing transparent access to all data in the cluster. Data nodes can be mirrored or nonmirrored. Compute nodes are always non-mirrored.
Data Node
A node that stores a partition (shard) of sharded data locally.
Compute Node
A node that does not store sharded data locally, but provides compute access to the sharded data on a corresponding data node, via global mapping.
Cluster URL
The address of a node, specified in the format "IRIS://host:port/cluster-namespace".
Cluster Namespace
A namespace that exists, with the same name, on every node of a sharded cluster, providing transparent access to all sharded and nonsharded data, and all code, on the cluster.
Master Namespace
A namespace that exists only on node 1 of a sharded cluster. Metadata and nonsharded data stored in its globals databases, and code stored in its routines database, are accessible in the cluster namespace on every node of the cluster, via database mapping.
Shard Database
A database on each compute node that stores a partition (shard) of sharded data. A shard database is not the default database of any namespace, but globals stored in it are visible within the cluster namespace on the same data node, or any associated compute node, via global mapping.

Architecture of a Sharded Cluster

Sharded clusters configured using methods of this class have the following architecture:

  • Each node consists of a separate InterSystems IRIS instance on a separate machine, and participates in only one sharded cluster.
  • Every node has a cluster namespace, which all have the same name on every instance of the cluster ("IRISCLUSTER", by default).
  • A cluster has one or more data nodes, each of which has a shard database which stores one partition (shard) of the cluster's sharded data. Adding data nodes increases the degree of partitioning (number of shards) of sharded data.
  • A cluster is either mirrored or nonmirrored. In a mirrored cluster, all data nodes are mirrored. In a nonmirrored cluster, all data nodes are nonmirrored.
  • A cluster has zero or more compute nodes. These do not store any data, but have mapped access to the sharded data of an associated data node. Zero or more compute nodes can be associated with each data node, and are transparently balanced across data nodes as more compute nodes are added. Adding compute nodes increases the computing bandwidth for executing sharded queries.
  • All nonsharded data, and all code, is stored in the globals and routines databases, respectively, of the master namespace on the first node of the cluster, and is transparently accessible in the cluster namespace on every node, via database mapping. This master namespace, which exists only on the first node, is largely transparent to users, as all user data access can (and should) be done in the cluster namespaces.
  • There is one special case where the master namespace should be used directly: Any globals, package, or routine mappings that should be visible in the cluster namespace on every node should be defined by users in the master namespace, from which they are automatically propagated to the cluster namespaces. Mappings defined by users in a cluster namespace are not propagated, and are only visible in the cluster namespace of the node on which they are defined.
  • All sharded data is transparently accessible via SQL in the cluster namespace of every node. The underlying globals of each partition of sharded data are only transparently accessible on the data node where that partition is stored, or on any compute nodes associated with that data node (although they can be accessed from any node via extended global references).
  • Every node plays a dual role, as an application server providing transparent access to all data and code, and as a shard server which locally executes those portions of sharded queries and updates that access sharded data stored on that node (in the case of a data node), or mapped to that node (in the case of a compute node). User connections can be load balanced across all nodes, so adding nodes of either type increases the computing bandwidth for user applications.

Prequisites for using this API

  • Machines must be provisioned and must be mutually accessible via TCP/IP.
  • InterSystems IRIS instances must be installed, with sharding-enabled licenses.
  • InterSystems IRIS instances must have the ECP configuration parameters MaxServers and MaxServerConn set to values at least as great as the anticipated number of nodes in the sharded cluster. Note that if these values are changed after installation, the instances must be restarted for the new values to take effect.

Inventory


Parameters Properties Methods Queries Indices ForeignKeys Triggers
13


Summary


Methods


• classmethod AttachAsComputeNode(pClusterURL As %String, pHostIPAddress As %String = "") as %Status
Attaches the current InterSystems IRIS instance to a specified cluster as a compute node.

This method automatically and transparently performs all steps needed to enable this instance as a compute node of the specified cluster (subject to these prerequisites), including:

  • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections to match the list on the node specified by pClusterURL.
  • Creates the cluster namespace if it doesn't already exist, configuring it to match the settings on the node specified by pClusterURL.
  • Associates this compute node with a data node that previously had the minumum number of associated compute nodes, so as to automatically balance compute nodes across data nodes.
  • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace on node 1, and to the shard database on the the associated data node.
  • Propagates any user-defined mappings from the master namespace on node 1 to the cluster namespace on the current instance.
  • Sets all SQL configuration options to match their settings on the node specified by pClusterURL.

This method returns an error if the current InterSystems IRIS instance is already a node of a sharded cluster.

Parameters:

pClusterURL
Identifies the cluster to which to attach, by specifying the address of any existing node of the cluster, in the format "IRIS://host:port/cluster-namespace". Namespace may optionally be omitted, since there can be at most one cluster namespace in an InterSystems IRIS instance.
pHostIPAddress
IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.

Returns:

Status code reporting success or failure of this API call.

Notes:

  • A cluster namespace, with same name as on all other nodes of this cluster, is created if a namespace of that name doesn't already exist. Note that if it does already exist, its globals database is "orphaned", as the global database of the cluster namespace is mapped to the globals database of the master namespace on node 1, and a compute node does not need to reuse this database as a shard database.
  • If the cluster namespace doesn't exist, it is created with interoperability enabled or not according to whether it is enabled for the cluster namespace on the node specified by pClusterURL.
  • All SQL configuration settings are set to match those on the node specified by pClusterURL.
  • Compute nodes are assigned a numeric id, in sequence starting from 1001, in the order they are attached to the cluster. Nodes are listed by id by ListNodes and GetMetadata.
• classmethod AttachAsDataNode(pClusterURL As %String, pHostIPAddress As %String = "") as %Status
Attaches the current InterSystems IRIS instance to a specified cluster as a nonmirrored data node.

This method automatically and transparently performs all steps needed to enable this instance as a data node of the specified cluster (subject to these prerequisites), including:

  • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections to match the list on the node specified by pClusterURL.
  • Creates the cluster namespace and shard database if they don't already exist, configuring them to match the settings on the node specified by pClusterURL.
  • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace on node 1, and to the shard database on the current instance.
  • Propagates any user-defined mappings from the master namespace on node 1 to the cluster namespace on the current instance.
  • Sets all SQL configuration options to match their settings on the node specified by pClusterURL.

This method returns an error in any of the following cases:

  • The current InterSystems IRIS instance is already a node of a sharded cluster.
  • The current instance is a mirror member.
  • The specified cluster namespace already exists, and its globals database is mirrored.
  • The node specified by pClusterURL is mirrored.

Parameters:

pClusterURL
Identifies the cluster to which to attach, by specifying the address of any existing node of the cluster, in the format "IRIS://host:port/cluster-namespace". Namespace may optionally be omitted, since there can be at most one cluster namespace in an InterSystems IRIS instance.
pHostIPAddress
IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.

Returns:

Status code reporting success or failure of this API call.

Notes:

  • A cluster namespace, with same name as on all other nodes of this cluster, is created if a namespace of that name doesn't already exist. If it does already exist, its globals database becomes the shard database for this node.
  • If the node specified by pClusterURL is a data node, it is used as the template for configuration settings on this node. If the specified node is a compute node, the corresponding data node is used the template.
  • If the cluster namespace doesn't exist, it is created with interoperability enabled or not according to whether it is enabled for the cluster namespace on the template node, and with its globals database settings, including directory path, matching those of the shard database on the template node.
  • All SQL configuration settings are set to match those on template node.
  • Data nodes are assigned a numeric id, in sequence starting from 1, in the order they are attached to the cluster. A data node's id is the same as its shard number. Nodes are listed by id by ListNodes and GetMetadata.
• classmethod AttachAsMirroredNode(pClusterURL As %String, pMemberRole As %String, pHostIPAddress As %String = "", pArbiterIP As %String = "", pArbiterPort As %Integer = 0, pSSLDirectory As %String = "", pMirrorName As %String = "", pMemberName As %String = "") as %Status
Attaches the current InterSystems IRIS instance to a specified cluster as a mirrored data node.

This method automatically and transparently performs all steps needed to enable this instance as a data node of the specified cluster (subject to these prerequisites), including:

  • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections to match the list on the node specified by pClusterURL.
  • If the current instance is not already a member of a mirror, performs mirror configuration according to role specified by pMirrorRole:
    • "primary" Creates a new mirror set, and configures the current instance as its first failover member.
    • "backup" Configures the current instance as the backup failover member of the mirror set whose primary failover member is specified by pClusterURL.
  • Creates the cluster namespace and shard database if they don't already exist, configuring them to match the settings on the node specified by pClusterURL. If pMemberRole is "backup" and pClusterURL specifies node 1 of this cluster, also creates the master namespace and master database if they don't already exist.
  • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace on node 1, and to the shard database on the current instance.
  • Propagates any user-defined mappings from the master namespace on node 1 to the cluster namespace on the current instance.
  • Sets all SQL configuration options to match their settings on the node specified by pClusterURL.

This method returns an error in any of the following cases:

  • The current InterSystems IRIS instance is already a node of a sharded cluster.
  • The specified cluster namespace already exists, and its globals database is not mirrored.
  • pMemberRole is "backup", pClusterURL specifies node 1 of this cluster, and a namespace with the same name as the master namespace already exists, and is not mirrored.
  • The node specified by pClusterURL is not mirrored.
  • pMemberRole is "backup", and the node specified by pClusterURL belongs to a mirror set which already has two failover members.
  • pMemberRole is "primary", but the current instance is the backup failover node of an existing mirror set.
  • pMemberRole is "backup", but the current instance is the primary failover node of an existing mirror set.

Parameters:

pClusterURL
Identifies the cluster to which to attach, by specifying the address of am existing node of the cluster, in the format "IRIS://host:port/cluster-namespace". Namespace may optionally be omitted, since there can be at most one cluster namespace in an InterSystems IRIS instance. If pMemberRole is "primary", pClusterURL may specify any existing node. If pMemberRole is "backup", pClusterURL must specify the node which is the primary failover member of the mirror set for which the current instance will be attached as the backup failover member.
pMemberRole This node's role as a mirror member:
  • "primary" Attach as primary failover member.
  • "backup" Attach as backup failover member of mirror whose primary failover member is specified by pClusterURL.
pHostIPAddress
IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.
pArbiterIP
Arbiter's hostname or IP address. Specify a non-null value to configure this mirror to use an arbiter, overriding the value inherited from the node specified by pClusterURL. Ignored when attaching a backup failover member.
pArbiterPort
Arbiter's port. Specify a non-zero value to configure this mirror to use an arbiter, overriding the value inherited from the node specified by pClusterURL. Ignored when attaching a backup failover member.
pSSLDirectory
SSL/TLS keys directory. Specify a non-null value to configure this mirror to use SSL/TLS, overriding the value inherited from the node specified by pClusterURL. Directory is expected to contain three files named "CAFile.pem", "CertificateFile.pem, and "PrivateKeyFile.pem".
pMirrorName
Name of mirror set to create. Default: IRISMIRRORN, where N is a mirror naming sequence number unique within this cluster (usually the same as the node id).
pMemberName
Name by which this instance is known to the mirror. Default: The mirror name, with the letter 'A' appended for the primary failover member, or 'B' for the backup failover member.

Returns:

Status code reporting success or failure of this API call.

Notes:

• classmethod ClusterNamespace() as %String
Gets the name of the cluster namespace on the current InterSystems IRIS instance.

Returns:

Name of the cluster namespace, or empty string if this instance is not a node in a cluster.
• classmethod GetMetadata(ByRef pMetadata) as %Status
Retrieves an overview of the metadata of the cluster to which the current InterSystems IRIS instance belongs.

The metadata is returned in an array, passed by reference, under the following subscripts:

  • ClusterNamespace - the name of the cluster namespace, visible on all nodes.
  • MasterNamespace - the name of the master namespace on node 1.
  • MasterGlobalsDatabase - the name of the globals database of the master namespace.
  • MasterRoutinesDatabase - the name of the routines database of the master namespace.
  • Node - information specific to individual nodes, stored under two additional subscripts: the node's numeric id (with ":Primary" or ":Backup" appended for mirrored nodes), and one of the following:
  • NodeType - "Data" or "Compute".
  • DataNodeId - for compute nodes only, the id of the corresponding data node.
  • ShardDatabase - for data nodes only, the name of the database containing this node's portion of sharded data.
  • Host - the hostname or IP address of the machine hosting the node's InterSystems IRIS instance.
  • SuperServerPort - the default port (superserver port) of the InterSystems IRIS instance.
  • WebServerPort - the web server port of the InterSystems IRIS instance.
  • IRISVersion - the InterSystems IRIS version number.
  • ShardingVersion - the sharding manager version number.


  • Parameters:
    pMetadata
    Array in which metadata is returned, passed by reference.

    Returns:

    Status code reporting success or failure of this API call.
  • • classmethod Initialize(pClusterNamespace As %String = "IRISCLUSTER", pMasterNamespace As %String = "IRISDM", pAllowedConnections As %String = "", pHostIPAddress As %String = "") as %Status
    Sets up the current InterSystems IRIS instance as the first data node of a new cluster. This method automatically and transparently performs all steps needed to enable this instance as the first node of a cluster (subject to these prerequisites), including:
    • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections, if specified via the pAllowedConnections argument.
    • Creates the cluster and master namespaces and their databases if they don't already exist.
    • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace, and to the shard database.

    This method returns an error in any of the following cases:

    Parameters:

    pClusterNamespace
    Cluster namespace, visible on all nodes of this cluster, providing access to both sharded and nonsharded data. If specified namespace does not exist, it is created. Default name: IRISCLUSTER.
    pMasterNamespace
    Master namespace for this cluster. If specified namespace does not exist, it is created. Default name: IRISDM.
    pAllowedConnections
    List of hosts allowed to participate in this cluster, specified as a semi-colon-separated list of IP addresses or hostnames. Default: No restriction of which hosts can participate.
    pHostIPAddress
    IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.

    Returns:

    Status code reporting success or failure of this API call.

    Notes:

    • classmethod InitializeMirrored(pClusterNamespace As %String = "IRISCLUSTER", pMasterNamespace As %String = "IRISDM", pAllowedConnections As %String = "", pHostIPAddress As %String = "", pArbiterIP As %String = "", pArbiterPort As %Integer = 0, pSSLDirectory As %String = "", pMirrorName As %String = "IRISMIRROR1", pMemberName As %String = "IRISMIRROR1A") as %Status
    Sets up the current InterSystems IRIS instance as the first data node of a new mirrored cluster. This method automatically and transparently performs all steps needed to enable this instance as the first node of a mirrored cluster (subject to these prerequisites), including:
    • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections, if specified via the pAllowedConnections argument.
    • If the current instance is not already the primary failover member of a mirror, creates a new mirror set, and configures the current instance as its first failover member.
    • Creates the cluster and master namespaces and their databases if they don't already exist.
    • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace, and to the shard database.

    This method returns an error in any of the following cases:

    • The current InterSystems IRIS instance is already a node of a sharded cluster.
    • The current instance is already a mirror member, but is not currently the primary failover member.
    • The specified cluster namespace or master namespace already exists, but its globals database is not mirrored.

    Parameters:

    pClusterNamespace
    Cluster namespace, visible on all nodes of this cluster, providing access to both sharded and nonsharded data. If specified namespace does not exist, it is created. Default name: IRISCLUSTER.
    pMasterNamespace
    Master namespace for this cluster. If specified namespace does not exist, it is created. Default name: IRISDM.
    pAllowedConnections
    List of hosts allowed to participate in this cluster, specified as a semi-colon-separated list of IP addresses or hostnames. Default: No restriction of which hosts can participate.
    pHostIPAddress
    IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.
    pArbiterIP
    Arbiter's hostname or IP address. Specify a non-null value to configure this mirror set to use an arbiter.
    pArbiterPort
    Arbiter's port. Specify a non-zero value to configure this mirror to use an arbiter.
    pSSLDirectory
    SSL/TLS keys directory. Specify a non-null value to configure this mirror to use SSL/TLS. Directory is expected to contain three files named "CAFile.pem", "CertificateFile.pem, and "PrivateKeyFile.pem".
    pMirrorName
    Name of mirror set to create. Default: IRISMIRROR1.
    pMemberName
    Name by which this instance is known to the mirror. Default: IRISMIRROR1A.

    Returns:

    Status code reporting success or failure of this API call.

    Notes:

    • classmethod ListNodes(pOutputFile As %String = "") as %Status
    Lists the nodes of the cluster to which the current InterSystems IRIS instance belongs, to the console or to a specified output file.

    The list contains a row for each node, with information in columns under the following headings:

  • NodeId - a numeric id. For data nodes, this is identical to the shard number.
  • NodeType - "Data" or "Compute".
  • DataNodeId - for compute nodes only, the id of the corresponding data node.
  • Host - the hostname or IP address of the machine hosting the node's InterSystems IRIS instance.
  • Port - the default port (superserver port) of the InterSystems IRIS instance.
  • Mirror - the mirror name, if the node is mirrored.
  • Failover - for mirrored nodes only, the type of failover member, either "Primary" or "Backup".
  • VIP - for mirrored nodes only, the VIP, if one is configured.

    Parameters:
    pOutputFile
    The pathname of a file to which output is written. Default: Prints output to the console.

    Returns:

    Status code reporting success or failure of this API call.