Custom Lister

Lister name

In order to be able to work with the lister, the lister needs to specify the format in which the external id for each source is presented. The external id for a lister consists of the lister name and the full reference. The full reference consists of the groupName and the localRef. An external id is shown in the following example:

:MYLISTER:groupname:localref

In this example, MYLISTER is the lister name alias. If you don't provide an alias, the full classname of the lister class is used. To determine the alias for your lister, use the GetAlias()Opens in a new tab method.

A lister name alias must be unique within the current namespace. If you specify a lister name alias that already exists, NLP generates a $$$IKListerAliasInUse error.

SplitFullRef() and BuildFullRef()

You must specify a SplitFullRef() instance method for your custom lister. This method is used to extract the groupName and the localRef from the fullRef string. Its results are supplied to the SplitExtIdOpens in a new tab class method. Assume your lister has an external id format like this: :MYLISTER:groupname:localref.

In this simple example, fullRef consists of the string groupname:localref, so the groupName is $PIECE(fullRef,":",1) and localRef is $PIECE(fullRef,":",2). Note that this is a very simple example; it does not work if the groupName or localRef parts contain ":" characters.

You must specify a BuildFullRef() instance method for your custom lister. This method is used to combine the groupName and the localRef to form the fullRef string. Its results are supplied to the BuildExtIdOpens in a new tab class method.

Default Processor

A Processor is a class that takes as input the list populated by the Lister, reads the corresponding source text, and directs the source data to the NLP engine for indexing. It can, optionally, pass this source data through a Converter.

NLP Processors are subclasses of the %iKnow.Source.ProcessorOpens in a new tab class. Each processor subclass is designed to read sources of a specific type, such as the %iKnow.Source.File.ProcessorOpens in a new tab which reads files from a directory.

Every Lister has a default processor that is capable of processing the sources from that lister. By default, it uses a class called Processor in the same package as the Lister. If there is no processor corresponding to the specified lister, or if you wish to use the generic %iKnow.Source.Temp.ProcessorOpens in a new tab, you should override the DefaultProcessor() method and specify the desired default processor.

Expand List

The ExpandList()Opens in a new tab method is responsible for listing all sources that need to be indexed. This method should be overridden by user-defined subclasses that implement how to scan through the particular type of source location or structures for your custom Lister. The parameters for this method are the same as those used when invoking the corresponding AddListToBatch() method. The parameters may differ, depending on the Lister that you implement. Make sure that the Lister-specific ExpandList() parameters are documented, so that a user knows which parameters to supply to the lister.AddListToBatch() method or the loader.ProcessList() method.

The ExpandList() parameters are as follows (in order):

• Path: the location where the sources are located, specified as a string.

• Extensions: one or more file extension suffixes that identify which sources are to be listed. Specified as a %List of strings.

• Recursive: a boolean value that specifies whether to search subdirectories of the path for sources.

• Filter: a string specifying a filter used to limit which sources are to be listed.

For further details, refer to Lister Parameters in the chapter “Loading Text Data into NLP”.