Project Abacá: What is sensitivity and how do people make judgements about it?

We all feel we have an intuition about when a something is sensitive. Whether it is the hushed conversation with the neighbour over the fence regarding what Mrs Smith from number 45 was doing wearing “that dress” in the street last night or the piece of paper with someone’s bank details that might have slipped from a pocket in the street. Our social instincts tell us what we should do. These instincts, when applied to government records, are encoded as exemptions in the Freedom of Information Act. The act requires that the civil servants and archivists charged with managing public records withhold sensitive records from public view until the sensitivity concerned has decayed with the passage of time.

Of course the sensitivities that government deals with concern many more matters than the personal data examples that we are familiar with from normal life. These sensitivities include:

  • matters of defence and national security
    (e.g. Nuclear missile deployment)
  • Commercial Confidentiality
    (e.g. Notes of contract negotiations)
  • Damage to International Relations
    (e.g. Insulting remarks about a leader of another country (or even a former leader))
  • Personal Privacy and Health & Safety
    (e.g. Religious beliefs or names of informants)

Because the exemptions interfere with the principle of open government that underpins FOIA, many have a “public interest balance test”. In essence, despite the presence of a sensitivity, does the public interest of release outweigh the negative consequences of release? Hence, decisions about what is and is not sensitive and whether it should be exempt are rarely simple.

As a part of the case study funded by the ITaaU, which we discussed in our previous article, Project Abacá has been looking at the research literature that reports studies of decision making; in particular decision making about documents. One thing is clear, this is a complex area of study where the results of experiments in one specific domain are hard to generalise to another. However, in our examination of the literature we have seen a common theme relating to narrative and the order in which documents are presented (e.g. studies of “threshold priming” in the Information Retrieval literature and work reported in wider studies). We have further work to do, but we have some tentative evidence from our initial discussions with government sensitivity reviewers that backs up the view of narrative order being significant. Our longer term aim is to use our interviews with reviewers and our literature study to develop some hypotheses as to the mental processes that sensitivity reviewers use when reviewing documents. We aim to design experiments using future enhancements to our proof-of-concept tool, together with measurements of the use of tool, to test these hypotheses. We will use these results to inspire further rounds of development of tools to assist the reviewer.

We have already produced a limited test of some ideas about assisting reviewers through the proof-of-concept tool we have built to prioritise documents for review based on an estimated likelihood of sensitivity. The estimated likelihood of sensitivity was produced from experiments on text classification using the Terrier Information Retrieval system to examine the statistical occurrence of sensitivity in a collection of documents compared to certain properties or features extracted from the documents. We published the initial results of our experiments at the recent European Conference on Information Retrieval in Amsterdam (ECIR2014). We aim to test our proof-of-concept tools and different ways of presenting the results of the classification through our engagement with real sensitivity reviewers in government departments as a part of the ITaaU-funded case study.

One thing is certain, sensitivity in government records is a tricky concept to nail down and define, but our step by step research is beginning to define the possible shape of tools to help reviewers.

Tim Gollins

Honorary Research Fellow

University Of Glasgow – School of Computing Science