5. Guidelines

The following guidelines are meant to summarize the key points of this study. They may serve as an index to the fuller argument and examples presented in the report itself, as each guideline contains, marked off by square brackets, a cross-reference to the relevant chapter and paragraph(s) of the text itself. In general, the guidelines follow the order of the previous four chapters and focus on the actions or directions appropriate for the archivist and records manager dealing with records containing personal information.

Such records in this study are limited to those in paper (or textual) format only [see 1.15] which are organized or retrieved by such personal identifiers as name or social insurance number [2.15]. Subject and policy files for the purposes of appraisal in this study are not to be considered as personal information records. Although there may be occasional exceptions, the personal information found in such records is incidental to their main purpose and they must be appraised as part of the larger file registry systems and series to which they belong, using somewhat different or modified appraisal criteria. Similarly, case files arranged, labelled, and retrieved by names of groups, associations, and companies, rather than of individuals, should also be appraised separately.

Archivists whose first language is not English may wish to refer to the special section [2.3 - 2.10] where the archival terms used in the study are defined.

The principal guidelines from this study are given below:

1. Records containing personal information have value to archives because they

-- protect the rights of citizens [1.2],

-- encourage historical and other research to uncover the collective memory of society [1.3],

-- aid in the development of public policy [1.4], and

-- support genealogical interests [1.5].

2. There are several major problems in appraising records containing personal information:

-- their enormous bulk [1.7],

-- their fragmented and uneven quality [1.8],

-- increasing concerns about violating personal privacy by permitting the collection of such data in the first place or its later diffusion [1.9], and

-- political pressure placed on archives to accept many personal information records which have limited or no archival value by other standards [1.10].

3. Archivists in North America especially have not developed sound appraisal theory and their resulting acquisitions of records have often been piecemeal, uncoordinated, and of poor quality [1.12]. This glaring theoretical omission, while serious for all types of records, is particularly troubling for those containing personal information because of the special problems they present (see Guideline 2 above).

4. Personal information is defined as any information about an identifiable individual recorded in any form [2.11]. Many examples of such information are given in this section of the study, as well as many of the types of records in which personal information is found and many of the contexts or government functions in which it is created [2.11 - 2.13].

5. A physical typology of the six major categories of records or aggregations of records where identifiable personal information can be found is presented in some detail. These are

-- forms,

-- lists and registers,

-- letterbooks and volumes,

-- particular instance case files,

-- continuing events case files, and

-- indexes [2.16: a. to f.].

In appraising personal information records, the physical linkages between records [2.17], other related media and artifacts [2.18], and the arrangement of the records [2.19] must be considered.

6. Personal information records also contain important characteristics based on their context, and these contextual factors include:

-- scope and coverage,

-- circumstances surrounding creation, and

-- location in the administrative hierarchy [2.20].

7. There are certain categories of personal information which are essential and must be kept by all archivists around the world. These are

-- records proving civil status,

-- land registration records,

-- certain court and legal records, and

-- the national census of the population [2.22 - 2.23].

8. The appraisal of personnel records of government employees is, for convenience, handled separately, although most such records are continuing events case files. While the electronic version of these records is most important, some of the paper case files should be retained. Selection criteria are proposed, including using levels in the hierarchy and special employee categories to segregate important files, as well as other factors to identify their possible informational value. It is suggested that personnel records of the national police and armed forces of a country be treated in an analogous way [2.24 - 2.28].

9. The "politics of appraisal" are discussed, whereby archives are sometimes forced by political or public pressure to keep certain records which they would not retain otherwise. The growing interest in genealogy is a major concern here. While genealogical information is best collected through the essential records category (see Guideline 7) or other sources, and will appear incidentally in many series and samples of archival records acquired for other reasons, the vast majority of case files from most operational programmes, should not be retained by a national archives purely for genealogical value. If that is their only value, then they should almost always be destroyed [2.29].

10. It is important to distinguish (as Chapters 3 and 4 of this study do) between personal information records -- primarily series of case files and individual case files -- which have permanent value, on the one hand, for what they reveal collectively about the phenomenon they describe or for the evidence of government operations they contain, and, on the other hand, those which have informational value because of what they reveal about specific persons, places, or things [2.30]

11. Chapter 3 addresses the first and most difficult of these concerns: how to identify the collective and evidential value in case file series. It advances a theoretical model of "macro-appraisal" which focuses first on the dynamic social forces brought to bear in the creation of personal information records. It is necessary to understand the nature of the citizen-state interaction, and the many variables present therein, before the archivist can pinpoint the best series to appraise. In the first instance, then, the appraisal of case files containing personal information must move from the physical to the conceptual, from the actual records to the functions and structures in society responsible for the creation of those records [3.1 -3.5].

12. This conceptual approach to appraisal requires a research agenda for archivists in their daily activities. Appraisal is a work of careful analysis and of archival, diplomatic, and historical scholarship, not a mere procedure or process. Applying guidelines or checklists in the appraisal process, as well as developing broader acquisition strategies, only works if such application is based on a rich understanding by the archivist of

-- the history of the records creator,

-- its official functions and legal mandates,

-- its internal organizational structure,

-- its decision-making processes,

-- its records-creating procedures, and

-- the changes in all these over time [3.6].

13. Such a conceptual approach to appraisal should be coupled with an active archivally driven emphasis to records management and records disposition activity. In an archivally directed appraisal environment, archivists focus actively on the 5 per cent (or less) of records in a government's holdings which have permanent value rather than passively on approving for destruction the 95 per cent without continuing value [3.7 and 4.8].

14. Archivists must incorporate institutional acquisition policies and mandates into their appraisal framework, as these will also define (and usually narrow) the field from which records can be acquired [3.8].

15. The present and anticipated use of records, and thus by implication consultations with users, should not play a significant role in the appraisal process. It is better for archivists to speculate less on possible uses for records tomorrow and to concentrate more on developing criteria to ensure that the records being acquired reflect the values, patterns, and functions of the society contemporary to the records creators [3.10, 3.6].

16. The departments and agencies of government will over time reflect the "image" of society -- that is, they will reflect the hopes, aspirations, activities, and frustrations articulated by its citizens and this will be most evident where the citizen-state interaction is most intense. The model presented in this chapter attempts to explain the mechanisms or loci in society where the citizen interacts with the state to produce the clearest insights (and thus the best documentary evidence) into societal dynamics and issues. Appraisal consists then of ensuring that the quality of the "image" reflected in records selected for archival retention is high [3.10 - 3.16].

17. The model contains three factors which define the citizen-state interaction:

-- the programme [3.18 - 3.22],

-- the agency [3.23 - 3.26], and

-- the citizen [3.27 - 3.28].

From this interaction are created the records containing personal information which the archivist must appraise. Taking these three factors in turn and analyzing the key variables of each, these sections of the chapter explore the "macro-appraisal" approach suitable for selecting such records. This explanation of the interaction of the programme, agency, and citizen focuses on the mechanics or location or nature of the formation of the image of society which the archivist must consider. This interaction varies widely from one set of citizen-state interactions to another, and thus from one series to another of the resultant personal information records. Depending on the type of variation involved, the records have greater or less permanent value. In short, the general rule for appraisal in the model is that the narrower and less flexible the range of interaction between the citizen on the one hand and the administrator (or agency) and programme on the other, the less significance the personal information case files will have in honing the societal image [3.29 - 3.31].

18. Archivists should appraise series of records containing personal information comprehensively, as part of a larger information universe. Not to do so is to start at the bottom of the records pyramid with the most voluminous and repetitive records having the least value, rather than at the top. There should be three broad phases in appraisal of case files:

-- Start with the policy files, followed by subject files, electronic records, central government records, procedure and forms manuals, and published and near-published information.

-- After determining which policy and subject records and which electronic data bases will be preserved by the archives, as well as the availability of other relevant information, only then will the archivist be able to assess both the sharpness of the image in the citizen-state interaction and the value of the connected case records containing personal information for demonstrating collectively or holistically that image.

-- Finally, if the case file series do not have value in sharpening the collective societal image, they should still be appraised, as a last step, for the informational value they may have about specific individuals, events, and places.

In this comprehensive approach, therefore, actually looking at the records containing personal information is, ironically, the last rather than the first step in appraising such records [4.1 - 4.3].

19. There is a close link between archival appraisal and a sound records management programme. While there may be differences in priorities which can work against this comprehensive approach to records disposition described in Guideline 18, a commitment to strategic planning of the scheduling process can minimalize these problems [1.13, 4.4, 4.7].

20. Archivists should adopt the "cluster concept" when they appraise records. If there are several interrelated series of personal information records, these should be appraised together so that overlapping information may be more readily identified and thus a better appraisal made [4.5].

21. Records centre storage can be used to gain distance and perspective in making appraisal decisions, especially for those series of records involving public issues or government functions which are controversial, hotly debated in public forums, and emotion-laden for many citizens (including archivists) at the time of their occurrence. Records managers and archivists must guard against excessive use of this strategy in order to avoid filling centres with useless records and incurring ever-mounting storage costs.

22. The main working rule for archivists in appraising all records is to destroy them. The focus for the archivist should not be on explaining what is being destroyed, but rather on justifying what is being kept. [4.8, also 3.7].

23. Case file series and other personal information records should usually be kept in electronic rather than paper format, where both exist. The advantages of the machine-readable version of the record are numerous, in addition to obvious savings in space and storage costs:

-- manipulability of the information,

-- ease of anonymization permitting public access in light of tougher privacy laws,

-- linkage to other data to create "new" information bases, and

-- potential for aggregation and statistical analysis [4.11, see also 2.24].

24. Records containing personal information should not normally be kept only to document the historical significance of a programme or an agency per se (as opposed to the concept of sharpening the societal image or preserving information on specific people, places, and things). The only exception is that a small example of case files may sometimes be kept to demonstrate the forms used where the programme was of particular importance and these cannot be obtained elsewhere, which might ironically be especially true for failed programmes [3.20]. Keeping large examples or more formal samples only to show the processes of the agency or the nature of its daily operations is rarely justifiable [4.12, also 3.18, 3.26, 3.27].

25. Archivists must ensure that they do not give undue weight to various types of records. They cannot appraise a large series of case files by "spot-checking" or by accepting the word of the agency's officials that various records are duplicated in other series and/or in other levels of the administrative hierarchy. A sampling methodology for purposes of appraisal (rather than acquisition) is suggested as a solution [4.15].

26. Once a series has been isolated as valuable for its collective and evidential value to the societal image, there are specific appraisal criteria used in order to assess the series as a whole [4.16 - 4.18]:

-- completeness,

-- authenticity,

-- uniqueness,

-- relationship to other records,

-- inclusive dates and time-span,

-- extent,

-- usability, and

-- a flexibility factor as opposed to rigidity

27. For records valuable only for their informational value residing in special cases within a series, specific appraisal criteria are also proposed. These include cases which

-- established a precedent and therefore resulted in a major policy or procedural change;

-- were involved in extensive litigation;

-- received wide-spread attention from the news media;

-- were widely recognized for their uniqueness by established authorities outside the agency; or

-- were reviewed at length in the agency's annual report to its legislative sponsor.

In addition to various other qualitative factors suggested to determine the few important files from the routine many, the use of the "fat file" method is also discussed and recommended [4.19 - 4.20].

28. Various practical and preservation issues affect archival appraisal:

-- storage and other costs,

-- statutory prohibitions,

-- media conversion,

-- the timing of records transfers,

-- archival networks, and

-- access and privacy restrictions [4.21 - 4.27].

The impact of these factors on appraisal is explained in some detail in these sections.

29. If records are still actively used in an agency or likely to be subject to many freedom of information or privacy requests, it is desirable to extend their formal retention period and thus delay their transfer to the archives in order to avoid excessive reference workloads. Conversely, the records are physically threatened with either outright destruction or rearrangement that would obliterate their original order, then the retention periods should be shortened and the records safeguarded in the archives as soon as possible [4.25].

30. After the appraisal research and analysis are completed, the archivist must choose from among five appraisal options:

-- keep all the records,

-- keep only key documents from files,

-- sample the files,

-- take an example of the records, or

-- destroy all the records [4.28].

31. For the convenience of users of this report, the four principal methods of sampling are summarized, with examples and the advantages and disadvantages of each method noted:

-- random statistical or probability sampling,

-- systematic or physically based sampling,

-- exemplary or qualitatively based sampling, and

-- exceptional sampling [4.30 - 4.36].

32. Most appraisal decisions for series of records containing personal information will be complicated and stratified depending on the agency, programme, their citizens, and the nature of the records. A concrete example is given based on Canadian immigration case files which might be helpful for readers [4.37].

33. In conclusion, when appraising records containing personal information as defined in this study, the archivist must consider four factors in this order:

-- researching and applying the "macro-appraisal" model of the societal "image";

-- utilizing the comprehensive approach to records assessment and scheduling;

-- applying the general working rules and specific appraisal criteria; and

-- tempering the decision, if at this point it is still positive, with whatever relevant practical, preservation, and political considerations may exist.

In most cases, the latter will lead to some form of sampling as the best option.