Digitising records – making an informed choice

Earlier this year, Google vice-president Vint Cerf warned that our digital records may not stand the test of time. In this article, Kelvin Smith, former records management consultant at The National Archives, looks at the feasibility and desirability of digitising records.

[ARTICLE FIRST PUBLISHED 2015]

In the UK the term ‘public records’ has a special legal meaning – perhaps we should call them Public Records (capital P and capital R). These are records created or held in central government departments. Public Records, using this definition, do not, therefore, include newspapers, birth, marriage and death certificates or records created by local authorities. Perhaps we should have another definition of public records (small P and small R) – records that are not private but are created or held by a wide variety of public bodies.

About five years ago, the amount of Public Records that were transferred to the National Archives was approximately one mile of shelving per year.

About five years ago, the amount of Public Records that were transferred to the National Archives was approximately one mile of shelving per year. These were mainly records that were 25-30 years old and represented about 2% of the original bulk (about 98% is destroyed in accordance with a long-established appraisal procedure). This means that in the 1980s central government departments were creating records that occupied roughly fifty miles of shelving.

If we extrapolate this conclusion and apply it to public records we might come up with a figure approaching 200 miles of shelving.

Let us move this forward 25 years to the beginning of the 21st century. Despite the increasing use of computers and other devices, the amount of paper created increased. It is only within the last few years that organisations have come to rely solely on the electronic record, although many still print out electronic documents, sometimes as a back-up.

The retention of many records is linked to legal obligations. For example, under a statute of limitations key financial records are generally kept for 7 years and many health and safety records are kept for 40 years or more. The vast majority of records, however, can be destroyed after relatively short periods (for example, superseded policies, minutes of minor meetings, delivery notes and similar ephemera).

The nature of the historical record has changed.

We undoubtedly live in an information society and people are seeking more information than ever before – mainly because it can now be made available easily. The nature of the historical record has changed with this technological revolution. Since it can be made more easily available and there is more of it, information has a longer-lasting currency.

For example, social researchers want to keep information that will help them to detect trends in health, education and housing. Economists see never-ending ways of analysing financial and fiscal data as a means of predicting future trends. Online retailers retain information about individuals so that they can develop their future strategies.

For people and organisations like these, historical records maintain their relevance and importance. In the paper world this would have been impossible; in the digital world it is almost obligatory.

Nowadays, many records are ‘born digital’ in the sense that the original record is electronic and only takes a paper form if it is printed out.

When considering the feasibility of digitising paper records, there are several things to consider. Digitising records involves:

• preparing the documents for digitisation – assessing their condition (are they too fragile?) and removing staples, paperclips and tags,
• training – operators should be trained in the handling of records and in the scanning of different types of records (they are not all standard A4 pages),
• keeping documents in order,
• selecting an appropriate scanner,
• applying quality control – different resolutions are required for different types of record (typed documents, handwritten documents, photographs),
• creating meaningful metadata – documentation explaining the context and content of the record.

Of course, there is a cost element to achieving these standards. A cost benefit analysis should be carried out when considering any digitisation project, comparing the annual storage costs with annual costs of digitisation. For example, will it be worth digitising documents that will be destroyed after relatively short periods of time?

Digital Preservation

In weighing up the feasibility of digitisation, a key consideration is the strong likelihood that we will have to migrate or convert digital records as technology develops. If we digitise our current records and we need to retain them, whether for legal reasons or for business purposes, we cannot be sure that they will remain accessible in a few years time.

If we have kept digitised records on compact disks or flash drives, will computers of the future have the capacity to read them? Remember 5¼ inch and 3½ inch floppy disks? None of our modern computers can accommodate them. However, it is not the hardware but the software that is primarily of concern.

PDF (Portable Document Format), TIFF (Tagged Image File Format) and JPEG (Joint Photographic Experts Group) are formats that have been relatively stable for the past few years. The software on modern computers can read them. How long is this likely to last? If there are changes in technology and different formats become the norm, we would have to convert all our digitised documents to the new format.

This will have a significant cost and that cost has to be borne in mind when digitisation projects are contemplated. It is clearly important that we make informed choices about what to digitise.

Meet The Team Contact Us Our Legal Services