Cloud Storage APIs

Creating a Client

Before working with a cloud storage provider’s buckets and blobs, your code must create a cloud storage client using the following syntax:

 Set myClient = ##class(%Net.Cloud.Storage.Client).CreateClient(javaServer,
    provider,credentialsFile,region,.tSC,endPoint)

Where:

• javaServer is the name of an InterSystems external server for Java (also known as a Java gateway). To use the default Java external server rather than creating a custom one, simply leave this argument empty.

• provider is an integer that indicates which cloud storage provider is being accessed with the client. For S3 buckets, use 0.

• credentialsFile is a file that contains the credentials used to access the cloud storage provider. The file must be formatted according to the provider’s specifications. If you are accessing an S3 bucket, you can leave this argument empty to use the default credential provider chainOpens in a new tab.

• region is the region containing the buckets you want to work with. For a list of AWS regions, see Amazon Regions, Availability Zones, and Local ZonesOpens in a new tab.

• tSC, which is returned by reference, is the status code returned by the method call.

• endPoint is an optional endpoint for AWS PrivateLinkOpens in a new tab.

Closing a Client

Once you are done working with a provider’s buckets and blobs, be sure to use the Close() method to close the client that you created. For example:

 Do myClient.Close()

Working with Buckets

The cloud storage client includes a set of methods designed to work with a provider’s buckets, which are the storage containers for blobs. The signatures of these methods are:

Method BucketExists(bucketName As %String) As %Boolean
Method GetBucketInfo(bucketName As %String) As BucketInfo
Method ListBuckets() As %ListOfObjects
Method CreateBucket(bucketName As %String)
Method DeleteBucket(bucketName As %String)

For example, to create a cloud storage client in order to retrieve details about a bucket, enter:

 Set bucketName = "s3-bucket"
 Set myClient = ##class(%Net.Cloud.Storage.Client).CreateClient(,0,
                "/home/AWSCredentials", "us-east-1", .tSC)
 Set bucketDetails = myClient.GetBucketInfo(bucketName)
 Do myClient.Close()

Retrieving Blob Information

The cloud storage client uses the following methods to retrieve information about blobs in a particular bucket:

Method BlobExists(bucketName As %String, blobName As %String) As %Boolean
Method GetBlobInfo(bucketName As %String, blobName As %String) As BlobInfo
Method ListBlobs(bucketName As %String) As %ListOfObjects

The client provides separate methods to download the content of a blob.

As an example, if you wanted to retrieve details about a particular blob, like its size, you could enter:

Set bucketName = "s3-bucket"
Set blobName = "s3-object-blob"
Set myClient = ##class(%Net.Cloud.Storage.Client).CreateClient(,0,"/home/AWSCredentials", "us-east-1", .tSC)
Set blobDetails = myClient.GetBlobInfo(bucketName, blobName)
Do myClient.Close()

Uploading Blobs

The cloud storage APIs allow you to upload data and files to cloud storage from InterSystems IRIS®. You can use any of the following methods to upload blobs to a cloud storage provider, depending on the source of the blob’s data:

Method UploadBlobFromString(bucketName As %String, blobName As %String, content As %String)
Method UploadBlobFromFile(bucketName As %String, blobName As %String, filePath As %String)
Method UploadBlobFromStream(bucketName As %String, blobName As %String, stream As %GlobalBinaryStream)

For example, to upload a file to an S3 bucket, you could include:

 Set bucketName = "s3-bucket"
 Set blobName = "s3-object-blob"
 Set myClient = ##class(%Net.Cloud.Storage.Client).CreateClient(,0,
              "/home/AWSCredentials", "us-east-1", .tSC)
 Do myClient.UploadBlobFromFile(bucketName, blobName, "/usr/file.jpg")
 Do myClient.Close()

Downloading Blobs

You can use the cloud storage APIs to download data from a cloud storage provider in order to work with it in InterSystems IRIS. Various methods are available, allowing you to choose the target format of the data:

Method DownloadBlobToString(bucketName As %String, blobName As %String) As %String
Method DownloadBlobToFile(bucketName As %String, blobName As %String, filePath As %String)
Method DownloadBlobToStream(bucketName As %String, blobName As %String) As %GlobalBinaryStream

For example, to download a blob from an S3 bucket and store it in a stream, enter:

 Set bucketName = "s3-bucket"
 Set blobName = "s3-object-blob"
 Set myClient = ##class(%Net.Cloud.Storage.Client).CreateClient(,0,
                "/home/AWSCredentials", "us-east-1", .tSC)
 Set IRISStream = myClient.DownloadBlobToStream(bucketName, blobName)
 Do myClient.Close()

Single Method for Uploading Blobs

The cloud storage APIs allow you to upload data and files to cloud storage without using an interoperability production. These class methods allow you to make a single call which will create a client, upload the blob, then close the client:

• SingleUploadBlobFromFileOpens in a new tab

• SingleUploadBlobFromStreamOpens in a new tab

• SingleUploadBlobFromStringOpens in a new tab

For example, to upload a file to Amazon S3, you could include:

 Set bucketName = "s3-bucket"
 Set blobName = "s3-object-blob"
 Set credentials = "/home/AWSCredentials"
 Set region = "us-east-1"
 Set filePath = "/usr/file.jpg"
 Set status = ##class(%Net.Cloud.Storage.Client).SingleUploadBlobFromFile(, 0, 
               credentials, region, bucketName, blobName, filePath)

Single Method for Downloading Blobs

You can use the cloud storage APIs to download data from a cloud storage provider in order to work with it in InterSystems IRIS. These class methods allow you to make a single call which will create a client, download the blob, then close the client:

• SingleDownloadBlobToFileOpens in a new tab

• SingleDownloadBlobToStreamOpens in a new tab

• SingleDownloadBlobToStringOpens in a new tab

 Set bucketName = "s3-bucket"
 Set blobName = "s3-object-blob"
 Set credentials = "/home/AWSCredentials"
 Set region = "us-east-1"
 Set status = ##class(%Net.Cloud.Storage.Client).SingleDownloadBlobToStream(, 0, 
              credentials, region, bucketName, blobName)

Deleting Blobs

Like the other cloud storage APIs, the method to delete a blob from cloud storage is straightforward. All it requires is the name of the blob you want to delete, including the bucket where it is stored.

Method DeleteBlob(bucketName As %String, blobName As %String)

For example, to delete a blob from an S3 bucket, enter:

 Set bucketName = "s3-bucket"
 Set blobName = "s3-object-blob"
 Set myClient = ##class(%Net.Cloud.Storage.Client).CreateClient(,0,
                "/home/AWSCredentials", "us-east-1", .tSC)
 Do myClient.DeleteBlob(bucketName, blobName)
 Do myClient.Close()

Working with Presigned Blobs

%Net.Cloud.Storage.ClientOpens in a new tab includes methods for generating and working with presigned blobs. These methods are as follows:

GeneratePresignedURL()

Method GeneratePresignedUrl(bucketName As %String, 
                            blobName As %String, 
                            duration As %BigInt, 
                            method As %String, 
                            metadata As %String = "") As %String

Generates a presigned URL. The arguments are as follows:

• bucketName specifies the bucket.

• blobName specifies the blob.

• duration specifies the duration of the URL in milliseconds.

• method specifies the URL method to use. The options depend on the cloud storage provider:

  • Amazon S3 supports DELETE, GET, HEAD, PATCH, POST, and PUT.

  • Azure supports a subset of  "acdrw"(ADD,CREATE,DELETE,READ,WRITE).

  • GCS supports DELETE, GET, HEAD, OPTIONS, POST, and PUT.

• metadata specifies any user metadata as a JSON string. Note the following points:

  • For Amazon S3, user metadata must be specified when generating the URL and cannot be later modified. The property names have the prefix x-amx-meta-

  • Azure ignores user metadata when generating a presigned URL.

  • For GCS, user metadata must be specified when generating the URL and cannot be later modified. The property names have the prefix x-goog-meta-. In this case, the generated signature contains user metadata information.

GetBlobMetadata()

Method GetBlobMetadata(bucketName As %String, 
                       blobName As %String) As %String

Returns user metadata (as a JSON string) for the given blob. The arguments are as follows:

• bucketName specifies the bucket.

• blobName specifies the blob.

Note that in this metadata, property names have no prefixes.

GetBlobMetadataWithPresignedUrl()

ClassMethod GetBlobMetadataWithPresignedUrl(presignedUrl As %String, 
                                            Output metadata As %String,
                                            javaServer As %RawString = "") As %Status

Retrieves metadata (as a JSON string) from the blob, accessible via a presigned URL. The arguments are as follows:

• presignedUrl is the presigned URL.

• metadata, which is returned as output, is the metadata as a JSON string.

• javaServer is the name of the Java External Language Server to use.

This method returns a status value, which you should check before proceeding.

UploadBlobWithPresignedUrl()

ClassMethod UploadBlobWithPresignedUrl(presignedUrl As %String, 
                                       input As %Stream.Object, 
                                       metadataJson As %String = ""
                                       javaServer As %RawString = "") As %Status

Uploads data to the given blob, accessible via a presigned URL. The arguments are as follows:

• presignedUrl is the presigned URL.

• input is an instance of %Stream.ObjectOpens in a new tab, containing the data to be uploaded.

• metadataJson is the optional metadata as a JSON string. Note the following points:

  • For Amazon S3, user metadata should not be provided here; it can be provided only when generating the URL.

  • For Azure, the property names have the prefix x-ms-meta-. Also, the property x-ms-blob-type with value BlockBlob must be included.

  • For GCS, user metadata should exactly match what was provided when generating the presigned URL.

• javaServer is the name of the Java External Language Server to use.

This method returns a status value, which you should check before proceeding.

DownloadBlobWithPresignedUrl()

ClassMethod DownloadBlobWithPresignedUrl(presignedUrl As %String, 
                                         start As %BigInt = "", 
                                         end As %BigInt = "", 
                                         Output outStream As %Stream.GlobalBinary,
															 javaServer As %RawString = "") As %Status

Downloads data from the given blob, accessible via a presigned URL. The arguments are as follows:

• presignedUrl is the presigned URL.

• start is the first position within the blob to be downloaded. The first position in the blob is 0. If this argument is a negative value, it is treated as 0. If this argument is an empty string, the entire blob is downloaded.

• end is the last position within the blob to be downloaded. If this argument is a negative value, the last position downloaded is the last position in the blob. If this argument is an empty string, the entire blob is downloaded.

• output, which is returned as output, is an instance of %Stream.ObjectOpens in a new tab containing the downloaded data.

• javaServer is the name of the Java External Language Server to use.

This method returns a status value, which you should check before proceeding.