SearchTitlesBrowse

Table of Contents NAME mg_get - output source texts for processing SYNOPSIS mg_get ...
Table of Contents

NAME

mg_get - output source texts for processing

SYNOPSIS

mg_get collection-name [-init|-i|-text|-t|-cleanup|-c]

DESCRIPTION

This program is the default one used by mgbuild to generate the source text for the MG system. Any program may be used to generate the source text for mgbuild as long as it confirms to the interface specified here.

OPTIONS

The collection-name must appear before any other option. Only the first option has any significance. If no option is specified -text is assumed.

-init and -i
The program is called once with this flag at the start of building a collection.

-text and -t
The program is called with this flag multiple times during the building of a collection. The program outputs (on stdout) the text of the collection. "Documents" within the collection are separated with by ctrl-B's (ASCII code 2). "Paragraphs" within the collection are separated with ctrl-C's (ASCII code 3). A collection need not contain paragraphs unless a level 3 inverted file is being generated (see mgbuild and mg_passes).

-cleanup and -c
The program is called once with this flag at the completion of building a collection.

ENVIRONMENT

 MGDATA
If this environment variable exists then its value is used a the default directory where the mg collection files are. If this variable does not exist then the directory "." is used by default. The command line option -d directory overrides the directory in MGDATA.

MG_GETRC
This environment variable specifies where the file containing the users mg source configurations is. If not set, mg_get uses a default of ~/.mg_getrc.

This file contains TAB delimited lines of the form CollectionName CollectionType files or directory

CollectionName is the name of the collection supplied to mg_get.

CollectionType specifies how mg_get should process the named files and directories and are descibed below.

files or directory is either a list of files separted by blanks or a single directory. Some of the CollectionTypes deal with files and others with just a single directory. Any files used ending with .gz or .Z are decompressed with gzip before processing. References to `~' expands to the users HOME directory.

CollectionType is one of

PARA For text based documents. The list of files specified are treated as a series of paragraphs separated by blank lines. Each paragraph becomes a seperate document on the indexed collection.

MAIL for mail files. The list of files specified are treated as UNIX mail files separated by lines starting with `From'. Each mail message becomes a seperate document on the indexed collection. As a extra feature any embedded tarmail encoded contents (enclosed by a `xbtoa Begin' and `xbtoa End' pair are removed.

DIR (and DIR2) For a single directory of files. Each file in the directory (and in any of it's subdirectories) are treated as a single document. With DIR the pathname of the file is prefixed to the contents of each file as an extra line while this is not done for DIR2 collections.

BIB for biblography files. The list of files specified are treated as a series of biblography files (eg BIBTEX or TROFF) separated by lines starting with `@'. Each reference becomes a seperate document on the indexed collection.

TXTIMG
for integrated collections of text and images. The single directory should contain files that have the same prefix if they are related. For example, monaLisa.pgm might be a gray-level image, and monaLisa.txt would be a textual file describing the image. The suffixes recognised are: .txt for ascii text .ptm for scanned text stored as a bilevel image .pbm for a black and white image (typically a line drawing) .pgm for a gray-scale image Inaddition,ifnocorrespondingascii a .pbm or .pgm file, then one is created with suffix .tmp.txt, and it stores the name of the image file (in principle it could store the OCR of a .txt.pbm file). At present the .tmp.txt files are deleted by the `-cleanup' option.

EXAMPLE

 An example ~/.mg_getrc file might look like: alice PARA /users/alice13a.txt.Z mail95 MAIL ~/MAIL/1995/*
doc DIR ~/documents
bibs BIB ~/etc/REFS/*.bib ~/etc/REFS/CC/*.bib
letters DIR ~/LETTERS
davinci TXTIMG ~/images/davinci

SEE ALSO

 mg_compression_dict(1), mg_invf_dict(1), mg_invf_dump(1), mg_invf_rebuild(1), mg_make_fast_dict(1), mg_passes(1),
mg_perf_hash_build(1),
mg_text_estimate(1), mg_weights_build(1), mgbilevel(1), mgbuild(1), mgdictlist(1), mgfelics(1), mgquery(1), mgstat(1), mgtic(1), mgticbuild(1), mgticdump(1), mgticprune(1), mgticstat(1)

Table of Contents