Sending Messages to Kafka from a Production
InterSystems IRIS becomes a Kafka Producer when an interoperability production includes a business operation that uses the Kafka outbound adapter. This adapter allows the business operation to send messages to a Kafka topic. You have two options when using a production as a Kafka Producer: use a built-in business operation that leverages the outbound adapter or build your own business operation that uses the adapter.
Adapter Details
The Kafka outbound adapter is the class EnsLib.Kafka.OutboundAdapterOpens in a new tab. It includes the following settings, all of which appear in the Management Portal as settings for the business operation that uses the adapter:
-
Servers defines a comma-separated list of IP address:port entries that identify servers in the Kafka broker cluster.
-
ClientID defines the Kafka client ID of the Producer.
-
Credentials defines the InterSystems credentials that correspond to the username and password of a Kafka client. For details on creating credentials, see Defining Reusable Items for Use in Settings.
-
SecurityProtocol specifies the security protocol which secures connections to your Kafka broker cluster. Currently, this property supports two values:
-
SASL_PLAINTEXT, which performs SASL authentication of the client over an unencrypted channel.
-
SASL_SSL, which uses the truststore and keystore information you provide to establish an SSL/TLS connection over which SASL authentication takes place.
-
-
SASLMechanism specifies the SASL authentication mechanism used to authenticate the Producer using the credentials specified by Credentials. Currently, only PLAIN is supported.
-
TrustStoreLocation (optional) specifies the file system path to the truststore which contains the certificate authority certificates necessary to validate a certificate from your Kafka broker cluster and establish an SSL/TLS connection.
-
TrustStoreCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to the truststore at the location specified by truststorelocation.
-
KeyStoreLocation (optional) specifies the file system path to the keystore which contains the keys necessary to establish an SSL/TLS connection with your Kafka broker cluster.
-
KeyStoreCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to the keystore at the location specified by keystorelocation.
-
KeyCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to a private key within the keystore at the location specified by keystorelocation.
For general information about building a custom business operation that uses an outbound adapter, see Defining Business Operations.
Send Kafka Messages
The class of the messages sent by the outbound adapter to Kafka is EnsLib.Kafka.MessageOpens in a new tab, which contains the following properties:
-
topic defines the Kafka topic where the Producer is sending messages.
-
value (a %String) defines the content of the Kafka message. If value is set, you should not set binaryValue.
-
binaryValue (a binary stream of arbitrary length) defines the content of the Kafka message when the length of the message exceeds the maximum length of a %String. If binaryValue is set, you should not set value.
-
key defines an optional tag for the Kafka message.
The outbound adapter's SendMessage() method takes a EnsLib.Kafka.MessageOpens in a new tab object as its sole argument, and sends that message to the Kafka server. For example, a custom business operation could call the adapter’s method by including the following:
Do ..Adapter.SendMessage(pRequest)
Built-in Business Operation
Rather than building a custom business operation that uses the outbound adapter, you can simply add EnsLib.Kafka.OperationOpens in a new tab to the production and define the adapter properties using the Management Portal settings. This business operation calls the adapter's SendMessage() method when it receives a request from another business host in the production. This request should contain a Kafka message of type EnsLib.Kafka.MessageOpens in a new tab.
For basic information about adding a business operation to a production, see Adding Business Hosts.
See Also
-
Using the Kafka Messaging API (for use without a production)