classmethod %BuildCube(pCubeName As %String, pAsync As %Boolean = 1, pVerbose As %Boolean = 1,
pIndexOnly As %Boolean = 1, pMaxFacts As %Boolean = 0, pTracking As %Boolean = 0,
ByRef pBuildStatistics As %String = "", pFactList As %String) as %Status
Where:
-
pCubeName is the logical name of the cube as given in its XData block; this is not case-sensitive.
-
pAsync controls whether the system performs the build in multiple background processes. If this argument is true, the system uses multiple processes and does not return until they are all complete. If this argument is false, the system uses a single process and does not return until it is complete.
Note:
If you have specified the cube option Initial build order, the system ignores the value of pAsync and uses a single process to build the cube. These options are described in Specifying Cube Options.
WARNING:
Custom code for build processes must not invoke HALT. Terminating a DeepSee agent may cause a cascade of lower priority agent terminations and result in the build hanging due to a lack of available agents for new tasks.
-
pVerbose controls whether the method writes status information. If this argument is 1, the system writes status updates to the command line. (This argument does not affect whether the method writes build errors or other logging information.)
-
pIndexOnly controls whether the method only updates the indices. If this argument is 1, the system only updates the indices of the fact table.
-
pMaxFacts specifies the maximum number of rows in the fact table. This determines the number of rows of the base table that the system uses when building the cube.
If pMaxFacts is 0, the default, all rows of the base table are processed.
-
pTracking is for internal use.
-
pBuildStatistics returns an array of information about the cube build. This array has the following nodes:
Node |
Value |
pBuildStatistics(“elapsedTime”) |
Elapsed build time, in seconds. |
pBuildStatistics(“errors”) |
Number of build errors. |
pBuildStatistics(“factCount”) |
Number of facts that were built and indexed. |
pBuildStatistics(“missingReferences”) |
Number of missing references. |
pBuildStatistics(“expressionTime”) |
Time spent processing sourceExpressions to build the cube elements. |
pBuildStatistics(“iKnowTime”) |
Time spent building NLP indices. |
-
pFactList is a list of specific Property names in the cube's fact class. If pFactList is supplied, the build will only update the columns listed in that fact list. This list can have either comma-delimited or $LB format. The specific facts being updated will be individually marked as unavailable for queries and queries referencing dimensions based on those facts will throw an error on prepare.
This method returns a status. If errors occur during the cube build, the status code indicates the number of build errors.
For example:
set status = ##class(%DeepSee.Utils).%BuildCube("patients")
This method writes output that indicates the following information:
-
Number of processors used.
-
Total elapsed time taken by the build process
-
Total amount of time spent evaluating source expressions, summed across all processors.
For example:
Building cube [patients]
Existing cube deleted.
Fact table built: 1,000 fact(s) (2 core(s) used)
Fact indices built: 1,000 fact(s) (2 core(s) used)
Complete
Elapsed time: 1.791514s
Source expression time: 0.798949s
If Source expression time seems too high, you should re-examine your source expressions to be sure that they are as efficient as possible; in particular, if the expressions use SQL queries, double-check that you have the appropriate indices on the tables that the queries use.
Cube Build Status
If there is a build in progress, you can monitor its progress using the %BuildStatus()Opens in a new tab method. In the Terminal, call:
DO ##class(%DeepSee.Utils).%BuildStatus("cubeName")
%BuildStatus() reports on the progress of a build regardless of whether you initiated the build programmatically or using the Architect. If there is no build in progress, %BuildStatus() displays the timestamp of the most recent build: There is no build in progress. Last build was finished on 06/23/2020 11:31:07.
The build dialog in the Architect also reports on the progress of builds which are initiated programmatically.
Minimizing Cube Size During Development
While you are developing a cube, you typically recompile and rebuild it frequently. If you are using a large data set, you might want to limit the number of facts in the fact table, in order to force the cube to be rebuilt more quickly. To do this, do one of the following:
Note that all these options are ignored during a selective build.
Using Parallel Processing During a Cube Build
If all the following items are true, the system uses multiple cores to perform the build:
When you build a cube asynchronously, the system sets up %SYSTEM.WorkMgrOpens in a new tab agents to do the work, if it is possible to use parallel processing.
Note:
These agents are also used to execute queries.
On rare occasions, you might need to reset these agents. To do so, use the %Reset()Opens in a new tab method of %DeepSee.UtilsOpens in a new tab. This method also clears any pending tasks and clears the result cache for the current namespace, which would have an immediate impact on any users. This method is intended for use only during development.
Build Errors
When you build a cube, pay attention to any error messages and to the number of facts that it builds and indexes. This section discusses the following topics:
For more information on troubleshooting options, see the InterSystems Developer CommunityOpens in a new tab.
Seeing Build Errors
When you build a cube in the Architect or in the Terminal, the system indicates if there are any build errors but does not show all of them. To see all the recorded build errors, do either of the following:
-
Look for the log file install-dir/mgr/DeepSeeUpdate_cube_name_NAMESPACE.log, where cube_name is the name of the cube, and NAMESPACE is the namespace in which this cube is defined.
The time stamp in this file uses $NOW to write the local date and time, ignoring daylight saving time.
-
Use the %PrintBuildErrors()Opens in a new tab method of %DeepSee.UtilsOpens in a new tab, as follows:
do ##class(%DeepSee.Utils).%PrintBuildErrors(cubename)
Where cubename is the logical name of the cube, in quotes.
This method displays information about all build errors. For example (with added line breaks):
SAMPLES>do ##class(%DeepSee.Utils).%PrintBuildErrors("holefoods")
1 Source ID: 100000
Time: 05/09/2019 14:12:52
ERROR #5002: ObjectScript error: <DIVIDE>%UpdateFacts+106^HoleFoods.Cube.Fact.1
2 Source ID: 200000
Time: 05/09/2019 14:12:41
ERROR #5002: ObjectScript error: <DIVIDE>%UpdateFacts+106^HoleFoods.Cube.Fact.1
3 Source ID: 300000
Time: 05/09/2019 14:13:13
ERROR #5002: ObjectScript error: <DIVIDE>%UpdateFacts+106^HoleFoods.Cube.Fact.1
...
10 build error(s) for 'holefoods'
Important:
In some cases, the system might not generate an error, so it is important to also check the fact count as discussed in the next section.
Checking the Fact Count
When you build a cube, the system reports the number of facts that it builds and indexes.
Each fact is a record in the fact table. The fact table should have the same as the number of records in the base table, except in the following cases:
Also, when the system builds the indices, the index count should equal the number of records in the fact table. For example, the Architect should show the same number for Building facts and for Building indices. If there is a discrepancy between these numbers, check the log files.
Possible Causes of Build Errors
If you see build errors or unexplained discrepancies in the fact count, do the following:
-
Examine any levels that use range expressions, and verify that these levels do not drop records. See Validating Your Levels.
An error of this kind affects the index count but not the fact count.
-
Try disabling selected dimensions or measures. Then recompile and rebuild to isolate the dimension or measure that is causing the problem.
<STORE> Errors
In some cases, the build log might include errors like the following:
ERROR #5002: ObjectScript error: <STORE>%ConstructIndices+44^Cube.cube_name.Fact.1
This error can occur when a level has a very large number of members. By default, when the system builds the indices, it uses local memory to store the indices in chunks and then write these to disk. If a level has a very large number of members, it is possible to run out of local memory, which causes the <STORE> errors.
To avoid such errors, try either of the following:
-
Build the cube with a single process. To do so, use %BuildCube() in the Terminal, and use 0 for its second argument.
-
In the <cube> element, specify bitmapChunkInMemory="false" (this is the default). When this cube is built using background processes, the system will use process-private globals instead of local variables (and will not be limited by local memory).
Missing Reference Errors
If your cubes have relationships to other cubes, the build log might include errors like the following:
ERROR #5001: Missing relationship reference in RelatedCubes/Patients: source ID 1 missing reference to RxHomeCity 4
This error can mean that you have built the cubes in the wrong order. See Building Cubes That Have Relationships in Advanced Modeling for InterSystems Business Intelligence. Note that if you use the Cube Manager, the Cube Manager determines an appropriate build order.
The missing relationship reference error can also occur when new source data becomes available during the cube build process — that is, after only some of the cubes have been built. For example, consider the sample cubes RelatedCubes/Cities and RelatedCubes/Patients (which are available in the SAMPLES namespace). Suppose that you build the cube RelatedCubes/Cities, and after that, the source table for RelatedCubes/Patients receives a record that uses a new city. When you build the cube RelatedCubes/Patients, there will be a missing relationship reference error.
If you are certain that you have built the cubes in the correct order, see the next section for information on recovering from the errors.
Recovering from Build Errors
The system provides a way to rebuild only the records that previously generated build errors, rather than rebuilding the entire cube. To do this:
-
Correct the issues that cause these errors.
-
Use the %FixBuildErrors()Opens in a new tab method of %DeepSee.UtilsOpens in a new tab, as follows:
set sc=##class(%DeepSee.Utils).%FixBuildErrors(cubename)
Where cubename is the logical name of the cube, in quotes. This method accepts a second argument, which specifies whether to display progress messages; for this argument, the default is true.
For example:
Fact '100' corrected
Fact '500' corrected
Fact '700' corrected
3 fact(s) corrected for 'patients'
0 error(s) remaining for 'patients'
Or rebuild the entire cube.
Business Intelligence Task Log
The system creates an additional log file (apart from the previously described build logs). After it builds the cube or tries to build the cube, the system also writes the DeepSeeTasks_NAMESPACE.log file to the directory install-dir/mgr. You can use the %SetLoggingOptions method of the %DeepSee.WorkMgrOpens in a new tab class to turn on logging for background agents that the system used during the build process. To do so, make a call like the following:
do ##class(%DeepSee.WorkMgr).%SetLoggingOptions(,,1)
To see this file from the Management Portal, select Analytics > Admin > Logs.
Tip:
This file also contains information about runtime errors of various kinds such as listing errors and KPI errors.
The time stamps in this files use the local date and time (taking daylight saving time into account).