An Explanation of the Database Build Process

Main -> Documentation -> Database Builder – Newton -> An Explanation of the Database Build Process

An Explanation of the Database Build Process

The database build process is the procedure used to transform your source data into a searchable Open SiteSearch Database Builder database. During this process, your source data is converted to ASN.1/BER records, the physical database files (HEDR, HDIR, POST, PDIR, and INDX) are initialized according to your database description (.dsc) file, and the physical database files are built.

Note:

This document will give you a conceptual model for what is going on "behind the scenes" in the Database Builder software as you build your database. The tasks you will need to perform to actually build the database are described in step 6 of creating a new SiteSearch database.

SSDOT and the Database Builder Utility Programs

The SiteSearch Database Operations Tool (SSDOT), a component of the Database Builder software, is a program used to manage the database utility programs that perform the build process. SSDOT has a menu-driven interface that automates the process described below.

Description of the Process

Several steps are involved in the database build process, beginning with the conversion of your source data and ending with the completion of the five physical database files. After converting the data in the initial step, SSDOT is used to manage and execute the necessary database building utility programs for the remaining tasks. The steps described below provide an overview of the system tasks involved in the build process and the results of each step.

1. Convert source data to ASN.1/BER format.

Data must be in ASN.1/BER format for you to use the Database Builder software to build a searchable database. The Database Builder software contains two conversion utilities, marcconv and sgmlconv, that can be used to convert MARC- and SGML-formatted data into ASN.1/BER format. For source data that is not in ASN.1/BER, MARC, or SGML formats, customized conversion programs must be developed.

2. Initialize database files.

SSDOT executes the initdb utility program which uses the .dsc file to create the physical database files (HEDR, HDIR, POST, PDIR, and INDX). A copy of the .dsc file is stored in region 0 of the HEDR file.

3. Store ASN.1/BER records and extract index terms.

SSDOT executes the pippin utility program which stores your ASN.1/BER data in the HEDR file, adds pointer information about the physical record location in the HEDR file to the HDIR file, and extracts index terms for storage in a temporary New Index and Postings (NIP) file.

4. Sort extracted index terms.

SSDOT executes the sortnip utility program which sorts the NIP file alphabetically, and, according to any special indexing instructions, consolidates the NIP file, and stores the data in a temporary Sorted NIP (SNIP) file.

5. Update INDX, POST, and PDIR files.

SSDOT executes the rome utility program which adds the indexed terms, their associated information, and pointers to the relevant records to the INDX, PDIR, and POST files to complete the entire database build process.

[Main][Documentation][Technical Reference][Glossary][Search]