Skip to main content

Inbound Adapter for Cloud Storage

The inbound adapter allows a business service to retrieve data from the cloud so it can be processed within an interoperability production. This adapter takes data from cloud storage and puts it into an InboundInputOpens in a new tab object, which your business service consumes to work with the data.

The classname for the inbound adapter is EnsLib.CloudStorage.InboundAdapterOpens in a new tab.

In addition to the properties used to connect to cloud storage, the inbound adapter includes properties that determine which blobs are retrieved and whether the cloud blob is deleted after it is retrieved by the adapter.

Retrieving Blobs

The inbound adapter includes two properties, Blob Name Prefix and Blob Name Pattern, that determine which blobs are retrieved from cloud storage. Looking at an example is the easiest way to understand how these properties work together. Consider an AWS S3 bucket that contains the following blobs:

foo/bar/baz
foo/bar/bash
foo/bar/bang
foo/boo

AWS uses / in blob names to create virtual hierarchies, similar to how a file system organizes files into directories. Within this scheme, the Blob Name Prefix works like a directory name. For example, foo/ chooses all blobs, while foo/bar/ only selects the first three blobs. This selection happens on the AWS server side.

After the client gets a list of blobs from the server, Blob Name Pattern is used to filter the list further. For example, if BlobNamePrefix="/foo/bar/" and BlobNamePattern="*ba?", the adapter retrieves just the first blob. This filtering happens on the client side. The Blob Name Pattern property supports the wildcards * and ?.

If more than one blob meets the criteria set by Blob Name Prefix and Blob Name Pattern, the adapter forwards each blob to the business service individually in separate InboundInput objects.

Creating a Business Service

As you develop a custom business service to use the inbound adapter, you need to define the onProcessInput method so that it accepts an EnsLib.CloudStorage.InboundInputOpens in a new tab object as its first argument. The inbound adapter places the data from cloud storage into this object type. As your business service works with the InboundInput object, it can access its three properties: Name (the name of cloud storage blob), Meta (metadata associated with the cloud storage blob), and Content (a stream that contains the data from cloud storage).

If you are unfamiliar with developing a custom business service, you can refer to Defining Business Services for details.

Reprocessing a Blob

Normally, if a blob has been processed, it will be ignored by future processing. If you need to force the production to reprocess a blob, you can remove the global node that tells the system the blob has been processed:

^Ens.AppData(..BusinessHost.%ConfigName, BucketName, BlobName)

For example, suppose that the business host MyProduction.MyService processes blobs and suppose that its configuration name is BucketProcessor. Also suppose that this business host has processed a blob named mysample/data1 in a bucket named Bucket1. To force this business host to reprocess this blob, use the ObjectScript shell, go to the namespace where the production is running, and execute the following command:

 kill ^Ens.AppData("BucketProcessor", "Bucket1", "mysample/data1")
FeedbackOpens in a new tab