New in InterSystems IRIS for Health 2024.1
This page describes the new and enhanced features in the 2024.1 release of InterSystems IRIS® for Health™, which is an extended maintenance (EM) release. Some of these features were also available in the continuous delivery (CD) releases since 2023.1, the previous EM release.
For a more exhaustive list of the changes included in this release, refer to the Upgrade ChecklistOpens in a new tab.
Release Information for 2024.1
The current release is 2024.1. The posting for 2024.1 is build 2024.1.0.263.0.
Healthcare Interoperability
SMART Scopes
InterSystems FHIR Server now supports SMART on FHIR OAuth2 access scopes using “CRUDS” (create/read/update/delete/search) style permissions. This enhancement aligns the application with the latest standards in healthcare interoperability and security, ensuring seamless integration with electronic health record systems and better data protection.
To learn more, refer to the HL7 documentation on scopes for using SMART to delegate access: https://hl7.org/fhir/smart-app-launch/scopes-and-launch-context.html#scopes-for-requesting-fhir-resourcesOpens in a new tab
For backward compatibility, InterSystems continues to support the 1.0 FHIR scopes.
JsonAdvSQL
InterSystems is excited to announce the release of JsonAdvSQL, a new FHIR search strategy designed to enhance performance and compliance. This release addresses critical limitations in the previous JSON strategy and introduces significant improvements.
JsonAdvSQL represents a significant step forward in FHIR search capabilities, offering faster, more compliant, and scalable solutions. This upgrade is poised to greatly enhance user experience and operational efficiency in managing healthcare data.
Some of the features of the enhanced conformance JsonAdvSQL provides are as follows:
-
Multi-resource compartment searches with wildcards and _type
-
Full support for _include / _revinclude and wildcards and :iterate
-
Canonical searches now include version support
-
New modifiers/prefixes and improved handling of implicit ranges
-
Enhanced capability in text, identifier, and URI searches
-
Better handling of Date/Quantity/Number search parameters, including new prefixes like ‘sa’, ‘eb’, and ‘ap’
-
Improved expression of SearchInclude/SearchRevInclude wildcards and system vs. type level operations in CapabilityStatement
New FHIR Object Model
InterSystems IRIS for Health 2024.1 introduces a new FHIR Object Model to enhance the developer experience when working with FHIR data in ObjectScript. Key features of the new FHIR Object Model include:
-
Static Class Representation: A package of %RegisteredObject classes that closely parallel the structure and naming conventions of FHIR resources and data elements. This provides better tooling support like IntelliSense, auto-completion, and documentation pop-ups when working with FHIR data structures.
-
Convenience Methods: The model classes include convenient methods to allocate nested structures and collections on demand, following the naming conventions of the FHIR specification. This allows developers to synthesize FHIR payloads more easily, driven by IDE suggestions.
-
Abstract Set API Support: The FHIR model classes support a subset of the Abstract Set API, providing a uniform coding pattern across dynamic (%DynamicAbstractObject) and static (%RegisteredObject) data models when working with FHIR data.
-
Data Conversion: The classes include methods to convert between the static class model and the dynamic %DynamicAbstractObject representation, facilitating interoperability between the two models.
-
Query Engine: The Abstract Set API integration allows developers to query FHIR data structures using the JSON Path Language (JPL), without the need for indexing or persisting the data.
The new FHIR Object Model aims to reduce the cognitive burden of working with the extensive FHIR data structures, providing better tooling support, consistent coding patterns, and efficient querying capabilities, ultimately enhancing the developer experience when building FHIR-based applications in InterSystems IRIS for Health. It is fully compatible with VS Code.
Interoperability Usage Metrics
InterSystems introduces Interoperability Usage Metrics in this release. With the substantial volume of data flowing through the interoperability engine, gaining insights into how the platform is utilized has become imperative for efficient operation and resource planning. Interoperability now provides a centralized monitor for counting the number of used interfaces for a given deployment. Users can access a granular breakdown of interface usage by date and namespace, this enables a more detailed understanding of how the system is utilized over specific periods and within specific contexts. The Interoperability Usage Metrics are conveniently accessible via API, facilitating the creation of comprehensive reports. API access empowers users to integrate usage data seamlessly into their reporting systems.
HL7 Production Validator
It can be expensive to bring a new interface live and hard to trust machine-generated parts of your logic (for example, logic produced by the HL7 Productivity Toolkit), especially when you lack a good testing facility. Interface verification is one of the major blockers for adopting new releases due to the cost and effort it requires. To ease the burden of upgrading systems, there is a need to have verification and trusted transformation logic to ensure correct system behavior.
To solve this common issue, InterSystems introduces the HL7 Production Validator to the 2024.1 release. The HL7 Production Validator is a terminal based utility that extracts HL7 Service headers, messages, and Operation messages to a temporary database. This production data is copied to your upgraded instance (the “target system”) and reconstituted for comparison against the original instance (the “source system”). Differences are evaluated based on recorded changes to the messages; these differences are represented to the user such that they can easily identify if the behavior is the same and identify where adjustments may be needed.
The HL7 Production Validator is particularly beneficial to customers who are migrating from HealthShare® Health Connect to InterSystems IRIS for Health, upgrading their current InterSystems IRIS for Health instance, or upgrading to an InterSystems IRIS for Health instance from Caché.
Enhancing Developer Experience
Using vectors in ObjectScript
As part of the new Vector Search capability (see related note in the Enhancing Analytics and AI section), InterSystems is making the $vector ObjectScript language feature available to all developers. InterSystems introduced $vector as an internal-only feature in 2022.2 to address the need for a new datatype and corresponding set of operators to work with large arrays of elements of a single datatype, and its highly efficient encoding format and use of SIMD functions for bulk operations contributed to the order-of-magnitude performance improvements brought by Columnar Storage. The Vector Search capability introduced with this release offers another use case for working with this kind of datatype, notably to capture embeddings and to perform fast bulk operations on the entire vector at once. Given the feature addresses two distinct use cases in modern computing challenges well, InterSystems has decided to make the feature available to all ObjectScript developers.
You can find full documentation of the new syntax and the current set of operators in the ObjectScript reference.
At first sight, you may think $vector has a lot of overlap with $list and wonder which one to use. They are, however, distinct features that serve different use cases. In general, if you are dealing with a single datatype and always access and operate on data elements in bulk, you may fit the $vector use case.
Support for JSON_TABLE
InterSystems IRIS for Health 2024.1 introduces support for the JSON_TABLE function according to the SQL standard. This function, intended for use in the FROM clause of a query, maps JSON values to columns in a relational table, enabling you to query JSON data in the context of SQL and providing very powerful capabilities in combination with lateral joins.
SQL Development
The SQL standard includes a WITH clause that helps in breaking complex nested queries down into small blocks, similar to how you’d use SQL views, but restricted to a single statement. In 2024.1, you can now use such Common Table Expressions (CTEs) in SELECT statements to add more structure and transparency to your queries. Many tools that generate SQL statement can also use CTEs.
As with each release, InterSystems IRIS SQL 2024.1 includes a number of enhancements that improve query performance:
-
When executing queries, any intermediate results that can be shared between different worker threads are now cached. This saves significantly on I/O when running complex queries on large datasets.
-
Space efficiency for tables that use columnar storage where all table columns meet the requirements for pure columnar storage has been optimized.
-
The Adaptive Parallel Execution mechanism introduced in prior releases now applies to a broader set of queries, including those that employ complex conditions, and virtually eliminates the cases in which traditional parallel subqueries would be used in the background.
-
The set of cases where filter and other query predicates for statements involving foreign tables can be pushed down to a remote database has been expanded. This makes foreign tables more performant by limiting the amount of data that gets passed back to your instance of InterSystems IRIS for Health.
Flexible Embedded Python Runtime
Flexible Python Runtime allows an administrator to choose the Python runtime that Embedded Python uses. This is perfect for upgrading to a specific version of Python or using a distribution like Anaconda. The Python runtime library can be configured through the Configuration Parameter File. This feature is available only for Linux-based builds of InterSystems IRIS for Health in 2024.1
Embedded Python for AIX
Embedded Python is now available on AIX systems.
Python BPL Editor
The Business Process Language editor now gives developers the option to write their business processes using Python. While most BPL scripts require no coding, there are a few blocks that enable the developer to implement their own logic in ObjectScript, JavaScript, or (new in the 2024.1 release) Python.
WSGI Web Apps
InterSystems IRIS for Health 2024.1 includes, on an experimental basis, the ability to create, secure, and host Web Applications that conform to the Python WSGI standard. With this enabled, you can run WSGI-compliant frameworks, such as flask or Django, inside your instance and take advantage of all the power of Embedded Python for moden Python web development. Even better, authentication is handled by the Web Gateway just like with all web applications. This feature is experimental in the 2024.1 release.
InterSystems welcomes feedback on how well this feature meets your needs.
Enhancing Analytics and AI
Vector Search
Vector Search is a new experimental feature in InterSystems IRIS for Health 2024.1 that provides the core functionality for semantic and vector-powered search of structured data, the cornerstone of generative AI applications, such as those implementing the Retrieval Augmented Generation (RAG) pattern. Vector search in 2024.1 comprises a new VECTOR SQL datatype and a set of related functions, including TO_VECTOR (which converts strings to vectors) and both VECTOR_DOT_PRODUCT and VECTOR_COSINE (which compute similarity between vectors).
In a typical application, specialized statistical language models called “embedding models” (smaller versions of LLMs like ChatGPT) convert sequences of text (or images, if the embedding model has been trained to process images) into embedding vectors, dense numeric arrays that specify a point in high-dimensional “latent embedding” space. Vectors are then stored in a SQL table column and the similarity functions are used to compare them to an input or query vector, the result of applying the same embedding model to the query string. The results of that search are the most similar vectors, and this can be a higher quality search capability than traditional “keyword” search because the embedding models have been trained to capture the “semantics” of the text. Future releases of InterSystems IRIS will increase the speed of search and introduce SQL syntax that abstracts the process of converting stored text as well as queries into vectors thereby providing a convenient way to implement powerful “semantic search” over data in InterSystems IRIS without having to deal with the underlying vectors directly.
Business Intelligence Improvements
InterSystems continues to improve the performance of InterSystems IRIS BI. In this release, the Analyzer user interface’s filter list management has been enhanced. In addition, this release includes:
-
Handling of deeply nested AND/OR conditions in the Advanced filters in Analyzer. This change introduces a means for computing structures consisting of deeply nested CROSSJOIN and %OR functions in MDX slicer clauses.
-
Enhanced APIs to allow accessing pivot table and dashboard metadata to allow for other visualization user interfaces to access that stored data.
Enhancing Speed, Scale and Security
Multi-Volume Databases
As customer databases grow, so do database files. To avoid those files becoming unmanageably large, or hit hard filesystem limits, InterSystems IRIS for Health now supports splitting your database across multiple physical “volumes” transparently. This new capability is easily configured: for any database, you can now configure a threshold size. When your initial .DAT file is about to hit this size and new global data needs to be written, InterSystems IRIS for Health will transparently create a new “database volume” file and start writing new data to that file. When that volume hits the threshold, another file is created, and so on.
There is no impact on applications and code accessing the data, as they continue to see the full database's content, no matter how many volumes it might be spread across. Furthermore, you can configure which directory the next volume should be created in, and when needed, you can rearrange database volumes across directories as a maintenance operation. Combined with planned work to increase the overall maximum database size, this will ensure your data remains easy to manage, well into the petabyte range.
Fast Online Backup
InterSystems for a long time has offered two main backup options: External Backup and Online Backup. With External Backup, users can freeze the write daemon for a brief period of time such that external solutions can create a consistent snapshot of all database, journal and other files. Those external solutions may track what changed versus the previous snapshot to produce an incremental snapshot that is much smaller than a full copy. In Online Backup, InterSystems IRIS for Health tracks which blocks changed, and can produce incremental backups while the system remains fully online (with only a very brief pause in writing).
This release introduces a new mechanism for incremental and full backups to the Online Backup feature, writing separate backup files per database using parallel processes. For large systems with many databases, this provides very significant improvements in the overall time it takes to perform a backup. The new Backup.Online class is the entry point for the new capability.
This change is the first part of a broader major project improving the performance and interface of the Online Backup feature, and covers the parallelism described above. As we intend to make further changes to the API and output format, the new API is currently labeled as experimental.
InterSystems welcomes feedback from customers through the Early Adopter ProgramOpens in a new tab and look forward to incorporating such feedback in an upcoming release.
During this phase, the pre-existing API in Backup.General and other utilities continue to invoke Online Backup’s pre-existing implementation and can continue to be used without any changes.
Platform Updates
Minimum Supported CPU Models
InterSystems IRIS for Health 2024.1 now has set a minimum CPU instruction set policy for Intel and AMD (amd64/x86_64) processors. InterSystems now requires all CPUs to have the AVX and BMI instructions, which are generally available on the following CPU architectures:
-
For Intel processors: Haswell and up
-
For AMD processors: Steamroller and up
InterSystems is taking advantage of newer instructions to improve product performance. InterSystems IRIS for Health 2024.1 takes advantage of the AVX instruction to reliably speed up vector operations.
Operating System Updates
InterSystems IRIS for Health 2024.1 will support Ubuntu 24.04 shortly after the OS is generally available.
MacOS Sonoma is added as a supported version of MacOS.