Retrieving Kafka Messages from within a Production
InterSystems IRIS becomes a Kafka Consumer when an interoperability production includes a business service that uses the Kafka inbound adapter. This adapter allows the business service to retrieve messages (as Kafka message objects) from a Kafka topic. You have two options when using a production as a Kafka Consumer: use a built-in business service that leverages the inbound adapter or build your own business service that uses the adapter.
Built-in Business Service
Rather than building a custom business service that uses the inbound adapter, you can simply add EnsLib.Kafka.ServiceOpens in a new tab to the production and define the adapter properties using the Management Portal settings. As the business service retrieves messages from Kafka at the interval determined by the Basic Settings > Call Interval setting, these messages are forwarded to another business host in the production using asynchronous requests. The business host where these requests are forwarded is determined by the Basic Settings > Target Config Names setting.
For basic information about adding a business service to a production, see Adding Business Hosts.
Adapter Details
The Kafka inbound adapter is the class EnsLib.Kafka.InboundAdapterOpens in a new tab. It includes the following settings, all of which appear in the Management Portal as settings for the business service that uses the adapter:
-
Topic defines the Kafka topic from which the Consumer is retrieving messages.
-
GroupID defines the ID of the Consumer's consumer group.
-
ReceiveSettings (optional) a JSON string defining settings for message retrieval. The list of available settings is the same as the list of properties made available by the Kafka API KafkaReceiveSettings class, with each property name serving as the key.
-
Servers defines a comma-separated list of IP address:port entries that identify servers in the Kafka broker cluster.
-
Credentials defines the InterSystems credentials that correspond to the username and password of a Kafka client. For details on creating credentials, see Defining Reusable Items for Use in Settings.
-
SecurityProtocol specifies the security protocol which secures connections to your Kafka broker cluster. Currently, this property supports two values:
-
SASL_PLAINTEXT, which performs SASL authentication of the client over an unencrypted channel.
-
SASL_SSL, which uses the truststore and keystore information you provide to establish an SSL/TLS connection over which SASL authentication takes place.
-
-
SASLMechanism specifies the SASL authentication mechanism used to authenticate the Consumer using the credentials specified by Credentials. Currently, only PLAIN is supported.
-
TrustStoreLocation (optional) specifies the file system path to the truststore which contains the certificate authority certificates necessary to validate a certificate from your Kafka broker cluster and establish an SSL/TLS connection.
-
TrustStoreCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to the truststore at the location specified by truststorelocation.
-
KeyStoreLocation (optional) specifies the file system path to the keystore which contains the keys necessary to establish an SSL/TLS connection with your Kafka broker cluster.
-
KeyStoreCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to the keystore at the location specified by keystorelocation.
-
KeyCredentials (optional) defines the InterSystems credentials which can be used to gain password-protected access to a private key within the keystore at the location specified by keystorelocation.
For general information about building a custom business service that uses an inbound adapter, see Defining Business Services.
See Also
-
Using the Kafka Messaging API (for use without a production)