Reindex Data Files

Toolbar and menu

Field Description

File

Select All Selects all files for re-indexing.

Reindex

Begins the re-indexing process once you have selected at least one file.

Once completed, the system displays the number of files you selected to reindex (as well as the number of files successfully re-indexed) and prompts you to allow operators to log back in to SYSPRO.

You are given the option of printing a report (log) of the re-indexed information. This includes:

the size of each file re-indexed
the size of all re-indexed files
the file size percentage for those files that are greater than or equal to 90% of the maximum allowable file size (printed in the Error message column).

Option	Description
Company id	Indicates the id of the company for which you want to reorganize files. This is only requested if you selected a file that requires the company id.
Language code	Indicate the language code to use for the files selected. This is only requested if one or more of the files you selected has the language code specified against it.

Check Files

Performs a general 'health check' on C-ISAM files. It can be performed at any time, but does not replace the requirement for re-indexing.

The system opens each selected file, verifies that both the data and index portions exist and that the index portion contains the correct key and record length for the SYSPRO data dictionary.

Export

Saves your current selections to a file that can be used at a later stage to perform the re-indexing process using the Import function. The filename defaults to xIMPNDX.EDI (where x indicates the company id).

This option enables you to configure the re-indexing process to be run in background mode (see Tips and tricks).

Import

Performs the re-indexing process based on file selections that were saved to an export file.

All the file selections originally made and saved to the export file are re-indexed, as well as any additional files you may have selected.

Options

Preferences Select this to indicate preferences that must be used for the current run of the program. You can also save these preferences against the operator code (see Preferences).

Rebuild

Reindex R/W User File Re-indexes a file that you created (e.g. a file you created using the Report Writer module).

Reindex Lot Archive File

Re-indexes specific archived lot files if you used the Purge and Archive program to archive lot information for completed lots.

If you don't specify the full path name, the system assumes the file is located in the \Base folder of your SYSPRO installation.

Rebuild

This enables you to selectively rebuild files that do not have an alternate key on which they can be re-indexed.

When they are updated during the normal course of processing, it is possible for one file to be updated and not the other. This causes them to be out of sync. The files need to be rebuilt to correct this.

Option Description

WIP Master/Sub Job File

This restores the integrity of the WIP Master/Sub Job index file by rebuilding the master/sub job links. The function reads the Work in Progress master file and identifies sub jobs for which a master job exists. These sub jobs are written to the master/sub jobs file.

You typically use this function if you used the Master Sub-jobs program to create links between a master job and sub jobs within a bill of jobs or you created links between a master and sub jobs when you created a job using the Job Entry program.

GRN Master File

This recreates a GRN Master file from the records in the GRN Details file and the Inventory Inspection file (optionally updating the GRN control record).


	If you have enabled the Check for duplicate GRNs option (Set Key Information) then you need to select this function to create the GRN Master file which enables the system to prevent the creation of duplicate GRN numbers.

At the Update control record with next GRN number field you must indicate the action you want to take if a discrepancy is found between the last GRN number used to create the GRN Master file and the next GRN number on the GRN control record. Options include:

Automatically - updates the GRN control record.
Warn prior to - displays a warning message so that you can optionally choose whether to update the GRN control record.
Do not - does not update the GRN control record.

These options are not available if manual GRN numbering is enabled (Set Key Information).

A/R Master/Sub Account

This restores the integrity of the Accounts Receivable Master/Sub Account index file by rebuilding the Master/Sub Account cross-references.

You would only use this function if you used the Master Sub-accounts program to create master and sub-account relationships between customers.

Lot Traceability Cross Ref Recreates the Lot Traceability Cross Reference file.

Inventory Serial Cross Ref

Recreates the Inventory Serial Cross Reference file.

This can be used to recreate the INVSTR file when a 'Record not found' error (ERROR 23) occurs on the file when running the Stock Take Selection program.

Preferences

Field

Description

Options

Prevent logins during reindex

Prevents non-system administrator operators from logging into SYSPRO once the reindex process has begun.

After all files have been re-indexed, you can choose whether to allow operators to load SYSPRO again.

This option has no effect on operators already logged into the company.

Use temporary directory

Copies the temporary file created by the re-indexing program to the operator's temporary folder (instead of the standard working folder).

This option is particularly useful when large files are being re-indexed and the working folder cannot easily be used.

Include Data Dictionary files

Includes the SYSPRO Data Dictionary files in the re-indexing process.


	These files do not typically require re-indexing. Including them will increase the time taken to complete the re-indexing process.

Display options

Operator intervention required

Enable this if you do not want to run the re-indexing process in background mode.


	If you disable this option, then the Notify option against each preference field is disabled.

Display running count

Indicates the number of records that must be processed before displaying a record count on the screen.

This option is only enabled if you enabled the Operator intervention required option.

If you select this option then the Cancel function is enabled during the reindex process, enabling you to cancel the reindex process.

Records processed per tickover

Select the number of records (i.e. 1, 10, 100, 1000) that must be processed before the program displays a count of records processed by the program.

Duplicate records

A duplicate record occurs if a data file contains more than one record with the same record key information. This can only occur if the indexed portion of the index file is corrupt.

Option	Description
Notify	Notifies you whenever the program encounters a duplicate record. If you then choose to ignore the record, the re-indexing process resumes. If you choose not to ignore the record then the re-indexing process is cancelled and no further files are re-indexed.
Skip file	Bypasses duplicate records and retain the original files.
Repair file	Ignores duplicate records and proceeds with the creation of new index files.
Abort	Cancels the re-indexing process whenever the program encounters a duplicate record.

Invalid end-of-file

This error usually occurs if a program does not terminate correctly (e.g. power failure). It normally indicates that the last record in the data file is shorter than the correct length of the record for that file.

Option	Description
Notify	Notifies you whenever the program encounters an invalid end-of-file marker. If you then choose to ignore the record, the re-indexing process resumes. If you choose not to ignore the record then the re-indexing process is cancelled and no further files are re-indexed.
Skip file	Bypasses invalid end-of-file marker errors and retains the original files.
Repair file	Ignores invalid end-of-file marker errors and proceeds with the creation of new index files.
Abort	Cancels the re-indexing process whenever the program encounters an invalid end-of-file marker error.

Zero sized file warning

This indicates that a file has no active data records (i.e. a file which contains only deleted records).

Option

Description

Notify

Notifies you whenever the program encounters a file with no active records.

If you choose to null the file, then the file is recreated as a zero size data file and the re-indexing process resumes.

If you choose not to null the file, then the original file is left intact and the re-indexing process resumes.

However, if the program encounters a file that does not exist, then the re-indexing procedure aborts and does not resume automatically.

Skip file

Bypasses files with no active records and retains the original file.

Ignore warning (clear file)

Ignores the warning and recreate files with no active records as zero-sized data files.

Significant file reduction warning

This indicates that the size of a file has dramatically reduced in size (i.e. 90% reduction). This could indicate a problem, which may result in the loss of data, or could merely reflect that records have been deleted.

Option	Description
Notify	Notifies you whenever the program encounters a file that is significantly reduced in size.
Skip file	Bypasses files that are significantly reduced in size and retains the original file.
Ignore warning	Ignores the warning and recreates files with the new file size.

Notes and warnings

Restrictions and limits

A warning is displayed if a file is found to be within 10% of the maximum allowable file size, providing the re-indexing is not running in background mode. The number of bytes in a file is calculated as: (record length + 1) x record count. If this value is greater than or equal to 90% of the maximum allowable file size, the warning message is displayed.

Currently, the maximum file size is 4GB per file on Windows and Unix based platforms.
You cannot reindex the same file (or group of files) within one module from more than one screen.

Process aborted or cancelled

The re-indexing process creates a new index file (in your SYSPRO working folder) from the existing live data file. Once this has occurred, the new index file is physically copied to the existing file, and the new index file is then deleted. It is possible that the copy process itself may fail (if your disk is full and there is no space in which to copy the new file). In this event the indexing process will abort and no further re-indexing will take place.
If a data file to be re-indexed does not exist, it is ignored by the re-indexing process. However, if the data file cannot be opened for some reason, then a message is displayed; the re-indexing process is cancelled; and no further files are re-indexed.

Translation considerations

The list of files displayed in the index selection window is generated from the IMPFIL.IMP file which is located in your program folder. If you want to modify this file (for translation purposes) then you must copy the file to your user-defined screen set folder, and modify it there. The re-indexing utility will first scan this folder before attempting to load the file from your program folder.

C-ISAM considerations

One of the issues with a C-ISAM based system is that it is inherently less robust than a SQL Server system. A number of data corruptions can occur. However it is virtually impossible that a regular SYSPRO program can corrupt a file (i.e. SYSPRO cannot write a program to corrupt a file). Therefore, data corruption occurs due to an outside influence, such as a computer crash, a virus, a hard-disk problem, etc.

A re-index does not 'fix' bad data (i.e. it does not scan each character of data to see if it is appropriate). It reads sequentially through the data portion of the indexed file (.DAT) and for each record that ends in an appropriate delimiter/terminator writes it to a new file. If the re-index process detects a logically deleted record (or any other delimiter) it discards the record (i.e. it is not written to the new indexed file). Upon completion the new indexed files (.DAT and .IDX) are copied across over the top of the previously existing files.

An important note is that if there is a data corruption in a data file (.DAT) then that can cause all subsequent records to be treated as 'bad'. The reason is as follows:

Each ISAM file has a record length of X bytes (e.g. 1000). This means that there is 1000 bytes of data followed by a line feed character indicating a valid record. If the row is deleted it is marked with a NULL character instead of the line feed character. This is the deleted record indicator.
The re-index process reads the data (.DAT) portion completely ignoring the index (.IDX) portion of the file. It does this by reading X+1 bytes (e.g. 1001) at a time. It then checks the last character (the delimiter) and if a line feed is found the record (1000 bytes) is written to a new indexed file (.DAT and .IDX) thus creating a new file that just contains the valid records from the old file. Note that if the corruption (e.g. a virus or disk corruption) causes a record in the middle of the file to be 1 byte short then all subsequent reads of 1001 bytes will be out by 1 and the line feed delimited is not going to be found.

Hints and tips

Re-indexing in background mode

This enables you to use your workstation for other tasks during the re-indexing process.

Disable the Operator intervention required preference.
Create an export file of your required selections.

Assign parameters to a copy of the SYSPRO icon so that selected data files are re-indexed automatically when SYSPRO is loaded from the icon.

Parameter Description

/prog=impndx Loads the Reindex Files program.

/link-xfilename.edi Specified after the /prog=impndx parameter, this uses the file selections created using the Export Selections function, where xfilename.edi is the name of the file exported.

/comp= Indicates the company that must be used.

/oper= Indicates the operator code that must be used.

/h

Loads the Reindex Files program, hiding all other SYSPRO screens. When the re-indexing is complete the program closes and exits.

If you have not enabled SYSPRO's Print Preview mode, then the program prints a log of the files selected for re-indexing, together with the status of the files, before exiting.

Otherwise, the program displays a list of files that were successfully/unsuccessfully re-indexed. You can optionally select to print a log of re-indexing information.

Any errors encountered are indicated in the Error Message column of the listview pane.

Automate the Re-indexing of C-ISAM Data Files

This procedure details how you can automate the re-indexing of you C-ISAM data files

Load the Reindex Files program.
Select/highlight the files you want to re-index in the File Selection listview.
Select the Export option from the File menu.
Enter a meaningful name for the export file. This file is saved in the ..\Base folder.
Create a SYSPRO shortcut.

For example:

/OPER=Shane /COMP=0 /PROG=IMPNDX /LINK=DAILY.EDI.

Target type: Application

Target location: BASE

Target: e /COMP=0 /PROG=IMPNDX / LINK=DAILY.EDI

Start in: C:\SYSPRO7Client\BASE
Test that the shortcut works as expected and ad this to Windows Scheduler.