Pears Index Routines

Contents

Introduction
[index_definition] for Index Routine
[index_definition] for Query Normalizer
Index Routine Class Diagram
Notes for the Index Routine Reference
Index Routines for Phrase Indexes
Index Routines for Keyword Indexes
Other Index Routines

Introduction

As described in the Pears System Overview, a Pears index routine is a Java class that can play two roles – index routine and term normalizer.

When creating a Pears database, an index routine creates index terms from specified field(s) in a database record.

When searching a Pears database, an index routine may also serve as a query (term) normalizer in WebZ or Database Builder's Record Builder application. A query normalizer allows you to manipulate patrons' search terms to better match specific database indexes.

This document is a comprehensive reference to the index routines shipped with SiteSearch 4.2.0. It is designed for users who wish to use these index routines when creating Pears databases or when creating WebZ or Record Builder database configuration files for Pears databases. Should you need to extend one of these index routines, see the Javadoc for more technical information.

[index_definition] for Index Routine

In a database description configuration file, an [index_definition] section defines how to create an index for a Pears database. A database description configuration file usually contains many [index_definition] sections, each with a unique name. An index definition section always includes the following variables:

index = index_id
routine = Pears_index_routine
tagpath* = BER_tagpath OR none
...

Next, there are variables that define parameters for the Pears_index_routine specified in the routine variable.

[index_definition] for Query Normalizer

To use an index routine as a query normalizer in WebZ or Record Builder, specify the index routine in the filter variable in its [index_definition] section in its database configuration file, like this:

filter = ORG.oclc.pears.IndexRoutines.Words

Follow the filter variable with variables that contain parameters applicable to the index routine, as needed. These are usually the same parameters you specified when creating the index. See examples in Pears-Newton Index Routine Comparison, including a few exceptions to the above guideline.

Return to Contents

Index Routine Class Diagram

The following diagram shows the relationships among the index routine classes shipped with SiteSearch 4.2.0, which all belong to the ORG.oclc.pears.IndexRoutines class package. Click a class name to jump to a description of the class and its required and optional parameters. Note that:

• Phrase and all classes that extend Phrase implement ORG.oclc.TermNormalizer. This allows these index routines to serve as query normalizers in the WebZ environment.
• Phrase and Words (which extends Phrase) serve as the basis for most Pears index routines.

Return to Contents

Notes for the Index Routine Reference

This section provides links to the index routines shipped with SiteSearch 4.2.0, organized by function. Click a link for information about an index routine. This includes the index routine's name, the class it extends, a description of the index routine, its parameters, and any parameters it inherits. For some index routines, there may also be links to examples or other notes.

Optional and Required Parameters

All parameters are optional and apply to all databases unless otherwise indicated. When a parameter applies to a specific type of database, this information appears under its name.

Repeatable Parameters

When a parameter can appear more than once in an index definition, the word "Repeatable" appears under the parameter's name. If a parameter is repeatable, use a separate line for each instance of the parameter. Use an asterisk (*) at the end of the parameter's name (such as indexUpTo*) or number each instance sequentially (such as indexUpTo1, indexUpTo2, indexUpTo3, and so on).

Unicode Characters in Index Definitions

To include the Unicode representation of a character in a Pears or WebZ index_definition, use \unnnn, where nnnn is the character's four-digit hex Unicode equivalent. For example, use \u0020 to represent a space.

Examples

The Examples section, if present, provides links to index definitions in the Pears-Newton Index Routine Comparison. Although that document pertains specifically to using Pears index routines to replicate commonly-used Newton index routines, its examples also illustrate use of index routines more generally.

Index Routines for Phrase Indexes

Index Routines for Keyword Indexes

Some of these index routines extend Phrase directly, rather than extending Words, but we classify them as keyword routines because they create keyword indexes.

Other Index Routines

• StopwordEnforcer

Return to Contents

Index Routines for Phrase Indexes

Phrase
Extends		ORG.oclc.pears.util.IndexRoutine
Description		The Phrase class is an index routine that normalizes input fields and builds lists of terms for phrase indexes. It takes fields from BER records that have been converted to appropriate UTF8-encoded (Unicode) data and converts them to index terms. It is the base index routine that most other index routines extend.
Parameters		Parameter Description bounds Specifies a pair of characters that surround data within a field. These characters surround data that should be included OR should be excluded from an index term. Examples:    bounds = "" bounds = () bounds = ""()[]{} Note:    The Phrase routine checks for this parameter, but does not use it. It is available for routines that extend Phrase, such as PhraseMinusBoundPhrases, PhraseWithinBoundPhrases, and WordsMinusBoundPhrases. collapse Indicates that Phrase should remove all of the characters in the collapse list from the field when it creates an index term. Example:    collapse = ?!,"':;&_-< >[]   debugPhrase Turns on debugging for all processing performed by the Phrase class. Possible values are: TRUE    Turns on debugging. FALSE   Performs no debugging. (default) extraIndex* Repeatable Indicates that another index should receive any terms extracted for this index by specifying the unique ID for another index defined for the database (as defined in the index parameter of its [index_definition] section. Example:    extraIndex* = 20 extraTrimChars Adds characters to the default list of trimChars for the current index only. This may be simpler than modifying the value of trimChars if you only want to add one or two more trim characters. Example:    extraTrimChars = @^  indexAfter* Repeatable Contains a character or string that indicates the start of the phrase to index. Use a separate line for each character or string. Examples:    indexAfter* = ( indexAfter* = \u0020 Note:    This is less efficient than using a bounds parameter. indexUpTo* Repeatable Contains a character or string that indicates the end of the phrase to index. Use a separate line for each character or string. Examples:    indexUpTo* = ) indexUpTo* = \u0020 Note:    This is less efficient than using a bounds parameter. indicator1 Optional for: MARC Indicates that indicator1 for this field must match the specified indicator(s) for the field to be indexed. Example:    indicator1 = 1   indicator2 Optional for: MARC Indicates that indicator2 for this field must match the specified indicator(s) for the field to be indexed. Example:    indicator2 = 2  indicators Optional for: MARC Requires that both indicator1 and indicator2 for this field must match the specified indicators for the field to be indexed. Example:    indicators = 12  joinFieldsWith Indicates the character or string to insert between subfields when creating a single index term from one or more subfields, as specified with the subfield* parameter. Example:    joinFieldsWith = \u0020\u0020 Note:    Do not separate characters with spaces unless you want to use a space as part of the string. maxLength Shortens the index term to the specified number of characters. The practical limit for an index term is approximately 2000 characters. Example:    maxLength = 80 nonFilingIndicator1 nonFilingIndicator2 Optional for: MARC Indicates that the value of the field's first indicator (for nonFilingIndicator1) or second indicator (for nonFilingIndicator2) determines the number of characters to remove from the beginning of the field. Possible values are: TRUE    Use the value of nonFilingIndicator1 or nonFilingIndicator2 to determine the number of characters to remove from the beginning of the field. FALSE   Do not use the value of nonFilingIndicator1 or nonFilingIndicator2 to determine the number of characters to remove from the beginning of the field. (default) Example:    nonFilingIndicator1 = true notIndicators Optional for: MARC Specifies that the two indicators for this field must not be equal to (or must be different from) the specified indicator(s) for the field to be indexed. Example:    notIndicators = 04 replace Indicates that you want to replace characters when indexing a term. Refers to another section in the index definition with a list of characters and the replacement value for each character in the indexed term. Example:    replace = replaceSect replaceSect Required: When using replace Section that specifies a list of characters and the replacement values to use when indexing these characters. Use with replace. Example:    [replaceChars] \u03B1 = alpha \u03B2 = beta \u03B3 = gamma startOffset Indicates the number of characters to ignore at the beginning of the field when creating an index term. Note:    Phrase counts off the startOffset for a field before removing any trim characters (see trimChars and extraTrimChars) from the start or end of a field or removing any collapse characters from the field. Example:    startOffset = 10 stripHTML Indicates whether to remove any HTML codes from the term to be indexed, where an HTML code is a word or phrase within angle brackets (<>). Possible values are: TRUE    Remove HTML codes from the term. FALSE   Do not remove HTML codes from the term. (default) subfield* Repeatable Indicates a subfield to include in an index term. Use to selectively include subfields from a field when you don't want to use all subfields in a field. Use in conjunction with joinFieldsWith. Note:    If you use this parameter, remember to specify a tagpath parameter one level higher than the subfields. For example, to index subfields 1 and 2 from the MARC 245 field: tagpath* = 245 subfield* = 1 subfield* = 2 trimChars Indicates the characters to remove from the beginning or end of the field when creating an index term. Type the list of characters without spaces between them. If you embed a space between two other characters, a space becomes one of the trim characters. Example:    trimChars = _ '&.,:* (default) Notes:    To add characters only, use extraTrimChars instead. Phrase (or other index routines that extend Phrase) adds the characters in extraTrimChars to the default list of trim characters. To modify the default list of trim characters in any other way (such as removing trim characters or both adding and removing trim characters), specify a value for trimChars that includes all trim characters for the index.
Examples		Click the links below to see how to use Phrase to emulate these Newton index routines:
		combad() ddc() greekphrase() phrase2() phrbhyp()	repnum() sgmlphrase() ugsbjcl() uptoparen()

Return to Index Routine List

Return to Contents

Index Routines for Keyword Indexes

Other Index Routines

See Also

Pears System Overview
Pears-Newton Index Routine Comparison
Pears Database Description Configuration File
Creating a New Pears Database
Converting a Newton Database to a Pears Database