Class Reference
IRIS for UNIX 2019.2
InterSystems: The power behind what matters   
Documentation  Search
  [%SYS] >  [%SYSTEM] >  [Cluster]
Private  Storage   

class %SYSTEM.Cluster extends %SYSTEM.Help

This class provides an API for node-level configuration of sharded clusters.

It can be used via the special $SYSTEM object, for example:

set status = $SYSTEM.Cluster.Initialize()

API Usage

Call Initialize to set up an InterSystems IRIS instance as the first node of a new sharded cluster.

Call AttachAsDataNode to add an instance to a cluster as a data node.

Call AttachAsComputeNode to add an instance to a cluster as a compute node.

Call ListNodes to print a list of a cluster's nodes.

Call GetMetaData to retrieve an overview of a cluster's metadata.

Call ClusterNamespace to get the name of the cluster namespace for the current instance.

Terminology

Sharded Cluster
An interconnected set of nodes which provides horizonal scalability of users and data.
Node
An InterSystems IRIS instance that participates in a sharded cluster. Nodes are of two types, "data" and "compute", but all nodes can be used interchangeably as application servers, providing transparent access to all data in the cluster.
Data Node
A node that stores a partition (shard) of sharded data locally.
Compute Node
A node that does not store sharded data locally, but provides compute access to the sharded data on a corresponding data node, via global mapping.
Cluster URL
The address of a node, specified in the format "IRIS://host:port/cluster-namespace".
Cluster Namespace
A namespace that exists, with the same name, on every node of a sharded cluster, providing transparent access to all sharded and nonsharded data, and all code, on the cluster.
Master Namespace
A namespace that exists only on node 1 of a sharded cluster. Metadata and nonsharded data stored in its globals databases, and code stored in its routines database, are accessible in the cluster namespace on every node of the cluster, via database mapping.
Shard Database
A database on each compute node that stores a partition (shard) of sharded data. A shard database is not the default database of any namespace, but globals stored in it are visible within the cluster namespace on the same data node, or any associated compute node, via global mapping.

Architecture of a Sharded Cluster

Sharded clusters configured using methods of this class have the following architecture:

  • Each node consists of a separate InterSystems IRIS instance on a separate machine, and participates in only one sharded cluster.
  • Every node has a cluster namespace, which all have the same name on every instance of the cluster ("IRISCLUSTER", by default).
  • A cluster has one or more data nodes, each of which has a shard database which stores one partition (shard) of the cluster's sharded data. Adding data nodes increases the degree of partitioning (number of shards) of sharded data.
  • A cluster has zero or more compute nodes. These do not store any data, but have mapped access to the sharded data of an associated data node. Zero or more compute nodes can be associated with each data node, and are transparently balanced across data nodes as more compute nodes are added. Adding compute nodes increases the computing bandwidth for executing sharded queries.
  • All nonsharded data, and all code, is stored in the globals and routines databases, respectively, of the master namespace on the first node of the cluster, and is transparently accessible in the cluster namespace on every node, via database mapping. This master namespace, which exists only on the first node, is largely transparent to users, as all user data access can (and should) be done in the cluster namespaces.
  • There is one special case where the master namespace should be used directly: Any globals, package, or routine mappings that should be visible in the cluster namespace on every node should be defined by users in the master namespace, from which they are automatically propagated to the cluster namespaces. Mappings defined by users in a cluster namespace are not propagated, and are only visible in the cluster namespace of the node on which they are defined.
  • All sharded data is transparently accessible via SQL in the cluster namespace of every node. The underlying globals of each partition of sharded data are only transparently accessible on the data node where that partition is stored, or on any compute nodes associated with that data node (although they can be accessed from any node via extended global references).
  • Every node plays a dual role, as an application server providing transparent access to all data and code, and as a shard server which locally executes those portions of sharded queries and updates that access sharded data stored on that node (in the case of a data node), or mapped to that node (in the case of a compute node). User connections can be load balanced across all nodes, so adding nodes of either type increases the computing bandwidth for user applications.

Prequisites for using this API

  • Machines must be provisioned and must be mutually accessible via TCP/IP.
  • InterSystems IRIS instances must be installed, with sharding-enabled licenses.
  • InterSystems IRIS instances must have the ECP configuration parameters MaxServers and MaxServerConn set to values at least as great as the anticipated number of nodes in the sharded cluster. Note that if these values are changed after installation, the instances must be restarted for the new values to take effect.

Inventory

Parameters Properties Methods Queries Indices ForeignKeys Triggers
9


Summary

Methods

• classmethod AttachAsComputeNode(pClusterURL As %String, pHostIPAddress As %String = "") as %Status
Attaches the current InterSystems IRIS instance to a specified cluster as a compute node.

This method automatically and transparently performs all steps needed to enable this instance as a compute node of the specified cluster (subject to these prerequisites), including:

  • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections to match the list on the node specified by pClusterURL.
  • Creates the cluster namespace if it doesn't already exist, configuring it to match the settings on the node specified by pClusterURL.
  • Associates this compute node with a data node that previously had the minumum number of associated compute nodes, so as to automatically balance compute nodes across data nodes.
  • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace on node 1, and to the shard database on the the associated data node.
  • Propagates any user-defined mappings from the master namespace on node 1 to the cluster namespace on the current instance.
  • Sets all SQL configuration options to match their settings on the node specified by pClusterURL.

This method returns an error if the current InterSystems IRIS instance is already a node of a sharded cluster.

Parameters:

pClusterURL
Identities the cluster to which to attach, by specifying the address of any existing node of the cluster, in the format "IRIS://host:port/cluster-namespace". Namespace may optionally be omitted, since there can be at most one cluster namespace in an InterSystems IRIS instance.
pHostIPAddress
IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.

Returns:

Status code reporting success or failure of this API call.

Notes:

  • A cluster namespace, with same name as on all other nodes of this cluster, is created if a namespace of that name doesn't already exist. Note that if it does already exist, its globals database is "orphaned", as the global database of the cluster namespace is mapped to the globals database of the master namespace on node 1, and a compute node does not need to reuse this database as a shard database.
  • If the cluster namespace doesn't exist, it is created with interoperability enabled or not according to whether it is enabled for the cluster namespace on the node specified by pClusterURL.
  • All SQL configuration settings are set to match those on the node specified by pClusterURL.
  • Compute nodes are assigned a numeric id, in sequence starting from 1001, in the order they are attached to the cluster. Nodes are listed by id by ListNodes and GetMetadata.
• classmethod AttachAsDataNode(pClusterURL As %String, pHostIPAddress As %String = "") as %Status
Attaches the current InterSystems IRIS instance to a specified cluster as a data node.

This method automatically and transparently performs all steps needed to enable this instance as a data node of the specified cluster (subject to these prerequisites), including:

  • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections to match the list on the node specified by pClusterURL.
  • Creates the cluster namespace and shard database if they don't already exist, configuring them to match the settings on the node specified by pClusterURL.
  • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace on node 1, and to the shard database on the current instance.
  • Propagates any user-defined mappings from the master namespace on node 1 to the cluster namespace on the current instance.
  • Sets all SQL configuration options to match their settings on the node specified by pClusterURL.

This method returns an error if the current InterSystems IRIS instance is already a node of a sharded cluster.

Parameters:

pClusterURL
Identities the cluster to which to attach, by specifying the address of any existing node of the cluster, in the format "IRIS://host:port/cluster-namespace". Namespace may optionally be omitted, since there can be at most one cluster namespace in an InterSystems IRIS instance.
pHostIPAddress
IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.

Returns:

Status code reporting success or failure of this API call.

Notes:

  • A cluster namespace, with same name as on all other nodes of this cluster, is created if a namespace of that name doesn't already exist. If it does already exist, its globals database becomes the shard database for this node.
  • If the node specified by pClusterURL is a data node, it is used as the template for configuration settings on this node. If the specified node is a compute node, the corresponding data node is used the template.
  • If the cluster namespace doesn't exist, it is created with interoperability enabled or not according to whether it is enabled for the cluster namespace on the template node, and with its globals database settings, including directory path, matching those of the shard database on the template node.
  • All SQL configuration settings are set to match those on template node.
  • Data nodes are assigned a numeric id, in sequence starting from 1, in the order they are attached to the cluster. A data node's id is the same as its shard number. Nodes are listed by id by ListNodes and GetMetadata.
• classmethod ClusterNamespace() as %String
Gets the name of the cluster namespace on the current InterSystems IRIS instance.

Returns:

Name of the cluster namespace, or empty string if this instance is not a node in a cluster.
• classmethod GetMetadata(ByRef pMetadata) as %Status
Retrieves an overview of the metadata of the cluster to which the current InterSystems IRIS instance belongs.

The metadata is returned in an array, passed by reference, under the following subscripts:

  • ClusterNamespace - the name of the cluster namespace, visible on all nodes.
  • MasterNamespace - the name of the master namespace on node 1.
  • MasterGlobalsDatabase - the name of the globals database of the master namespace.
  • MasterRoutinesDatabase - the name of the routines database of the master namespace.
  • Node - information specific to individual nodes, stored under two additional subscripts: the node's numeric id, and one of the following:
  • NodeType - "Data" or "Compute".
  • DataNodeId - for compute nodes only, the id of the corresponding data node.
  • ShardDatabase - for data nodes only, the name of the database containing this node's portion of sharded data.
  • Host - the hostname or IP address of the machine hosting the node's InterSystems IRIS instance.
  • SuperServerPort - the default port (superserver port) of the InterSystems IRIS instance.
  • WebServerPort - the web server port of the InterSystems IRIS instance.
  • IRISVersion - the InterSystems IRIS version number.
  • ShardingVersion - the sharding manager version number.


  • Parameters:
    pMetadata
    Array in which metadata is returned, passed by reference.

    Returns:

    Status code reporting success or failure of this API call.
  • • classmethod Initialize(pClusterNamespace As %String = "IRISCLUSTER", pMasterNamespace As %String = "IRISDM", pAllowedConnections As %String = "", pHostIPAddress As %String = "") as %Status
    Sets up the current InterSystems IRIS instance as the first data node of a new cluster. This method automatically and transparently performs all steps needed to enable this instance as the first node of a cluster (subject to these prerequisites), including:
    • Enables the ECP and sharding services, and sets the sharding service's list of allowed connections, if specified via the pAllowedConnections argument.
    • Creates the cluster and master namespaces and their databases if they don't already exist.
    • Creates all necessary mappings from the cluster namespace to the globals and routines databases of the master namespace, and to the shard database.

    This method returns an error if the current InterSystems IRIS instance is already a node of a sharded cluster.

    Parameters:

    pClusterNamespace
    Cluster namespace, visible on all nodes of this cluster, providing access to both sharded and nonsharded data. If specified namespace does not exist, it is created. Default name: IRISCLUSTER.
    pMasterNamespace
    Master namespace for this cluster. If specified namespace does not exist, it is created. Default name: IRISDM.
    pAllowedConnections
    List of hosts allowed to participate in this cluster, specified as a semi-colon-separated list of IP addresses or hostnames. Default: No restriction of which hosts can participate.
    pHostIPAddress
    IP address other nodes of this cluster should use to connect to this node. Specify a value for for this parameter if the default hostname known to InterSystems IRIS does not resolve to an appropriate address, or no hostname is available. Default: Default hostname will be used for inter-node connections.

    Returns:

    Status code reporting success or failure of this API call.

    Notes:

    • If either the cluster namespace or the master namespace already exists, the existing namespace is used. If one of them exists and the other doesn't, the globals database for the one that doesn't exist is created using all of the same configuration settings as the globals database of the namespace that already exists (other than directory path). If neither namespace exists, databases for both are created using default settings.
    • If neither namespace exists, the namespaces are created without enabling interoperability. If one namespace exists, the other namespace is created with interoperability enabled or not, according to whether it is enabled in the existing namespace.
    • If the cluster namespace exists, its globals database becomes the shard database for node 1. The globals database of the master namespace is used to store all nonsharded data for the cluster, and its routines database is use to store all code for the cluster.
    • classmethod ListNodes(pOutputFile As %String = "") as %Status
    Lists the nodes of the cluster to which the current InterSystems IRIS instance belongs, to the console or to a specified output file.

    The list contains a row for each node, with information in columns under the following headings:

  • NodeId - a numeric id. For data nodes, this is identical to the shard number.
  • NodeType - "Data" or "Compute".
  • DataNodeId - for compute nodes only, the id of the corresponding data node.
  • Host - the hostname or IP address of the machine hosting the node's InterSystems IRIS instance.
  • Port - the default port (superserver port) of the InterSystems IRIS instance.
  • Mirror - the mirror name, if the node is mirrored.
  • Failover - for mirrored nodes only, the type of failover member, either "Primary" or "Backup".
  • VIP - for mirrored nodes only, the VIP, if one is configured.

    Parameters:
    pOutputFile
    The pathname of a file to which output is written. Default: Prints output to the console.

    Returns:

    Status code reporting success or failure of this API call.


  • Copyright (c) 2019 by InterSystems Corporation. Cambridge, Massachusetts, U.S.A. All rights reserved. Confidential property of InterSystems Corporation.