Understanding the logic of forensic identification decisions (without numbers)

Alex Biedermann / Joëlle Vuille *

Over the past few years, a movement has been initiated among forensic science researchers and practitioners, in Switzerland and across Western criminal justice systems more generally, to shift away from categorical assertions of common source (e.g., «this crime scene mark and this control print come from the same person») towards new reporting formats that consider expert conclusions as decisions (e.g., an «identification» or «exclusion» is a decision made by the forensic examiner). As this movement gains momentum, there remains disagreement on how exactly the notion of decision ought to be understood. The call for improvement of the understanding of the logical tenets of forensic identification decisions faces the obstacle that many forensic practitioners shy away from formal and quantitative approaches (e.g., decision theory). The purpose of this contribution is to show that the logical essentials of forensic identification decisions can be captured and conveyed without going into the details of the mathematics of decision theory. We will then present and defend the view that forensic identification requires assessments and value judgments that go beyond the forensic practitioner's area of competence and that this fact requires a reassessment of the distribution of responsibilities between experts and other participants in the legal process.

Citation: Alex Biedermann/Joëlle Vuille, Understanding the logic of forensic identification decisions (without numbers), in: sui-generis 2018, S. 397

URL: sui-generis.ch/83

DOI: https://doi.org/10.21257/sg.83

* Alex Biedermann (alex.biedermann[at]unil.ch), PhD, Associate professor, University of Lausanne, School of Criminal Justice. Joëlle Vuille (joelle.vuille[at]unil.ch), PhD, Senior researcher, University of Lausanne, School of Criminal Justice. This research was supported by the Swiss National Science Foundation through Grant No. BSSGI0_155809. The authors would like to thank Simon Cole, Richard Friedman, Edward Imwinkelried and David Moran for their helpful comments on an earlier draft. Alex Biedermann gratefully acknowledges the support of the University of Michigan Law School (Michigan Grotius Research Fellowship). This paper was presented at the 18th Annual Meeting of the ENFSI Fingerprint Working Group, September 4th–7th 2018, University of Lausanne.


I. Introduction

After comparing a fingermark found on a crime scene with reference prints taken from a suspect, many forensic examiners, in Switzerland, Europe and beyond, will offer conclusions such as «Source identification (i.e., came from the same source)» or «Source exclusion (i.e., came from different sources)». Such conclusions are common in many forensic disciplines, including comparative handwriting examination, tool and shoe mark analysis, etc. But they are most typically encountered - and expected by recipients of expert information - in the area of fingerprint comparisons.

A reported conclusion of the kind «the crime scene trace and the reference mark came from the same source» (i.e., source identification) may sound self-explanatory. The problem is that such a statement is indefensible. The mere assertion by a forensic practitioner that a mark and a reference print come from the same source does not imply or prove that the source of the mark has been established. The reasons for this are threefold: empirical, argumentative and regulatory. On an empirical account, forensic practitioners usually do not conduct an exhaustive comparison against all potential sources of a recovered fingermark. A fingerprint examiner does not compare a mark found at a crime scene with the reference prints from all conceivable donors of the mark, to find all of them - except one - not to correspond.[1] It does not mean either that the examiner has excluded all potential donors from a certain suspect population. Most often, it is not even clear what the population of potential sources is (size, composition, etc.).[2] A further empirical consideration is that, both in real cases and in tests under controlled conditions, examiners have been found to provide erroneous conclusions. Thus, even if an exhaustive investigation of all potential sources is feasible (closed-set situation), assertions of common source are still not warranted, because of the potential of error.

On the argumentative side, an examiner's assertion that a mark and a reference print «came from the same source» is merely an item of information that is to be distinguished from the proposition (or, hypothesis) that the mark and the reference print come from the same source. Even if it is highly probable that an examiner will report that a mark and a print «came from the same source» when that proposition is actually true, this does not imply that the proposition of common source is necessarily true. Claiming the contrary would amount to committing the so-called inversion fallacy, by which a deductive statement is wrongly transposed into an inductive statement.[3]

On a regulatory account, the profession itself cautions against the use of identification conclusions.[4] Recently, for example, directions were issued according to which examiners shall «not assert that two friction ridge impressions originated from the same source to the exclusion of all other sources», «not assert that latent print examination is infallible or has a zero error rate» and «not use assertions of reasonable certainty».[5] This regulatory movement started in the United States approximately a decade ago following the publication of a report by the National Research Council.[6] Debates around those questions have now also spread to Europe and, to a limited extent, Switzerland. In Switzerland, there are currently no written guidelines or standards mandating fingerprint examiners to adopt a given format when reporting on their conclusions.[7]

From the above considerations, it follows that the conclusion «came from the same source» (i.e., identification) does not imply or establish that the suspect is indeed the source of the crime scene trace. This is a problem as it makes conclusions difficult to understand for the recipients of such expert testimony. In an effort to remedy those shortcomings, forensic practitioners have initiated a shift away from categorical assertions and towards new reporting formats centring on the notion of «decision». According to this new perspective, a «[source identification] conclusion is an examiner's decision that the observed friction ridge skin features are in sufficient correspondence».[8] This suggests that source identification is achieved whenever the examiner decides so - leaving it unclear when exactly (i.e., under which conditions) such a decision is warranted, and what it really means in the context of the case at hand.

In a descriptive sense, the choice of the term decision for labelling expert conclusions seems sensible since, indeed, examiners[9] need to decide, at some point, what conclusion to render in any given case. One could argue that this terminology has the advantage of making it clear that a judgment on the part of the examiner is required, and that a «match» is not the reflection of some incontrovertible ground truth that the examiner has merely «discovered». In that perspective, calling an identification a «decision» is an improvement over earlier formulations in that it makes it explicit that a (fallible) human evaluation is the basis for the expert conclusion provided to the fact-finder.

But the notion of decision can also be approached in more formal terms, in particular through decision theory. In its normative perspective, decision theory defines optimal decisions to be made by an agent with rational aspirations, on the basis of that agent's beliefs about uncertain states of nature, and the agent's assessment of the relative desirability[10] of the various consequences[11] of each possible choice.[12] According to Cole, however, forensic expert working groups do not appear to approach decisions in the rigorous sense of decision theory.[13] Informal conversations with practitioners suggest that the quantitative nature of probabilities and utilities, and their coherent treatment according to decision theory, are seen as hurdles that cannot be overcome in this context. One of the aims of this contribution is to show that the quantitative and computational aspects of classic decision theory are not necessarily required for the logical thinking about decision problems, and hence should not deter practitioners from considering formal approaches to guide their evaluative reporting.

Besides, as it may be tempting to conclude that decision-theoretic principles are practically irrelevant and can be dealt with dismissively, we will explain why this attitude is short-sighted. Specifically, we will present and defend the view that all forensic conclusions, most notably identification decisions, have underlying logical tenets that can be explained in basic decision-theoretic terms, without quantitative assessments, and independently of whether or not reporting examiners actively endorse and rely on decision-theoretic principles when they evaluate the evidence in the case at hand. Our paper is structured as follows. Section II will clarify the fundamental terms that all forensic conclusions comprise. We will use standard decision-theoretic terms, but unlike previous publications in this context, we will avoid formulae and decision matrices, favouring verbal accounts of qualitative considerations and graphical means. Based on these elements, Section III will critically review and discuss current reporting conventions, showing that the legitimacy of the stance taken by a majority of practitioners on the topic of identification conclusions must be seriously questioned due to the intricate issues and stakes involved in forensic identification practice.

II. Clarifying the logic of identification decisions without numbers

1. Decision trees: the anatomy of forensic identification decisions

The logic of forensic identification decisions can be represented in graphical terms, such as decision trees. Consider Figure 1, which shows a decision tree for a simplified forensic identification problem. It is simplified because only two decisions are considered: identifying (d1) or not identifying the person of interest (d2) as the source of the crime scene mark, the latter decision being a generic placeholder for all conclusions other than «identification» (for example, «inconclusive»).[14]

Decision trees have two types of nodes. The first are decision nodes, taking the form of squared boxes. Their emanating branches represent the different possible decisions in the case at hand. These decisions are mutually exclusive and exhaustive: only one of them can be taken, and one of them must be taken. Circles are another type of nodes - so-called chance nodes - whose emanating branches represent states of nature of the present, past or future. These states of nature are mutually exclusive and exhaustive.

When constructing decision trees, one usually starts on the left, by defining the relevant decision point. In our example, the possible decisions are represented by the branches «identifying the person of interest» (d1), or «not identifying the person of interest» (d2) as the source of the crime scene mark. Moving on each of these two branches to the right leads to chance nodes, representing the possible states of nature under which the decisions are being made. Indeed, when deciding how to conclude, it is not known which state of nature actually holds: i.e., it is not known whether the person of interest or an unknown person is the source of the crime scene mark. At the far right-hand side are decision outcomes, defined by combinations of decisions and particular states of nature: e.g., identifying the person of interest d1 when in fact the person of interest is the source of the crime scene mark represents a correct identification, whereas a false identification occurs when the expert identifies the person of interest as the source of the crime scene mark when in fact an unknown person is the source of the mark. Similarly, a false non-identification and correct non-identification occur from deciding d2 when in reality the person of interest and an unknown person, respectively, is the source of the mark.


Figure 1: Decision tree for a simplified forensic identification problem. The squared node represents a decision point, with two branches representing the two feasible decisions «identify», d1, and «do not identify» (the person of interest), d2. The circled nodes represent states of nature (in terms of the branches emanating to the right), i.e. the person of interest and an unknown person is the source of the trace, respectively.

2. Thinking about possible decision outcomes

Figure 1 provides a descriptive summary of the basic elements of a generic forensic identification problem. But there is more to decision problems than decisions and states of nature. Indeed, when making a decision, it is natural to pay attention to the relative desirability of various decision consequences. Clearly, not all decision consequences are equally desirable: accurate conclusions, i.e., correct identification and non-identification, are preferred to wrong determinations, i.e., false identification and false non-identification (i.e., a missed identification). For the purpose of the current discussion, it is sufficient to express the relative desirability of decision outcomes qualitatively: we use a checkmark in a white box for accurate conclusions, and a cross for erroneous conclusions. The overall worst conclusion, a false identification, is shown with a cross in a box with a black background. Note that this is a particular assumption that seems suitable for identification decisions at advanced stages in the criminal process (e.g., at trial). Assessments may be different during the investigative phase where a false non-identification may deprive authorities from a relevant investigative lead. A false identification during the investigation may be considered less adverse than a false non-identification because it is expected other information accumulated in the case will help uncover the false association.[15] Figure 2 provides a summary of these considerations.


Figure 2: Decision tree previously defined in Figure 1, extended here - on the far right-hand side - with an indication of the relative desirability of the various possible decision outcomes. Checkmarks represent desirable outcomes, and crosses indicate less desirable outcomes. The overall worst consequence, a false identification, is highlighted with a cross in a box with a black background.

3. Thinking about how to decide

Forensic scientists commonly assert that when rendering identification conclusions, they adequately take into account the fact that conclusions may be erroneous and, hence, represent undesirable outcomes. But how exactly can this be done? To examine this question, consider again Figure 2 and imagine a scientist who pursues a prudent approach. In particular, assume that the scientist takes a pessimistic perspective, in the sense that, whatever decision is made (i.e., «identifying», or «not identifying»), the focus is on undesirable outcomes: a false identification or a missed identification. Thus, if the scientist's concerns are adverse decision outcomes, one possible strategy - known as the minimax strategy - would be to always decidenot to identify the person of interest as the source of the mark (d2). This attitude would be meaningful in the sense that, even if the expert does not identify the person of interest as the source of the mark when this person is in fact the source of the mark (missed identification), this consequence is less adverse than committing a wrong identification (the adverse consequence associated with d1).[16] Indeed, the latter is the overall worst consequence (see Sect. 2.2). In Figure 2, the branch corresponding to decision d2 («do not identify»), the minimax decision, is highlighted in bold, and the branch representing decision d1 («identify») is double crossed.

However, always avoiding the overall worst consequence comes at the price of never identifying anyone as the source of a crime scene trace. Clearly, this would severely limit the interest that members of the judiciary may have in the services of scientists who adopt this approach. Conversely, the above discussion also makes it clear that any person who chooses to make identification conclusions, hence all scientists who adhere to the traditional identification practice, cannot do so without accepting the possibility of identifying as the source of the trace a person who is in fact not the source. This gives rise to the following two questions: First, if identification decisions are necessary to the well-functioning of the legal process, and thus must be made, what bases should be required before they can be made? Second, should such decisions be made by forensic scientists, or by the decision-maker? We will address these two questions in turn in the next paragraph.


Figure 3: Decision tree previously defined in Figure 1, highlighting two essential decision ingredients (for the situation in which decision d1, «identify», is made): consideration of the relative adversity of decision consequences, and the probability of the relevant states of nature (i.e., whether the person of interest or an unknown person is the source of the crime scene mark). The dotted lines of the branches indicate that there is uncertainty about which state of nature holds, and hence which decision consequence is obtained.

4. Assessing the suitability of identification decisions

As we have seen, decision-makers who identify the person of interest as the source of a crime scene mark must take into consideration the possibility of making a false identification. The natural follow-up question to this is: How small should the probability of an erroneous determination be in order for the identification decision d1 to be acceptable? And, related to this question, how probable must a correct identification be at a minimum before an identification conclusion is acceptable? It is important to emphasise at this point that the probabilities for the various decision consequences - and hence the answers to these questions - correspond to the decision-maker's probabilities for the two possible states of nature that condition the consequences.[17] Figure 3 summarises the two main decision ingredients discussed so far: the relative adversity of decision consequences and the probability (as assessed by the decision-maker) of each consequence.

This captures many experts' natural way of thinking about forensic identification problems: indeed, it is often argued that, the more an expert regards a false identification as an undesirable consequence, the higher her requirements will be for the probability that the trace comes from the person of interest before she is willing to identify.[18] Stated otherwise, in order to feel 'comfortable' with making the identification decision, decision-makers seek to ensure, as best as possible, that the person of interest is indeed the source of the crime scene mark, rather than an unknown person.

Although straightforward at first sight, decision theory supports this argument in two distinctive and important ways. First, decision theory specifies how assessments of the desirability of decision consequences are to be coherently combined with probabilities for unknown states of nature. Specifically, decision theory says that the losses (or, utilities) assessed for all possible consequences of a decision are to be weighted by the respective probabilities with which the various consequences are thought to occur, leading to the notion of expected loss (or, utility). This result, expected loss (or, utility), provides a basis for comparing the relative merits of rival decisions. This, in turn, may help the decision-maker choose between various decision options.[19] Second, such computations also allow one to sort out exact decision points: for example, given an assessment of the relative desirability of the decision consequences, decision theory defines the limiting probabilities for the states of nature such that one decision becomes preferable to a given rival decision.[20] This also allows one to give statements of the following kind: «When one's probability for the person of interest being the source of the crime scene mark is x times greater than the probability that an unknown person is the source, the decision to identify is warranted only if a false identification of the person of interest is considered less than x times worse than a missed identification».[21]

Again, the quantitative investigation of these results is beyond the scope of this contribution; of main interest here are the more general, qualitative orders of magnitude for the assessment of the key factors, as well as the insight that, in order to be coherent, these assessments ought to still comply with the underlying logical constraints. An example for this is the precept mentioned above that one's probability for the person of interest being the source of the crime scene mark ought to be higher the more adverse a false identification is considered to be compared to a missed identification. Also of interest is the question of what implications for forensic reporting practice these qualitative understandings have. We will address these aspects in Section III.

III. Discussion

1. Decision competence and decision prerogatives

As argued throughout Section II, forensic identification involves intricate issues that raise the question of whether it is appropriate for forensic examiners to render identification decisions. Specifically, the question is whether forensic examiners should take a stance with respect to (i) the probabilities as to whether the person of interest or an unknown person is the source of the crime scene mark, and (ii) the relative desirability of the various decision consequences.

Let us consider aspect (i) first: What does the forensic scientist really know when it comes to judging whether the person of interest is the source of the crime scene mark? The answer is «most often, not much», essentially because the forensic scientist does not have access to the entirety of the case brought against the defendant. Besides, there is a misconception about the scientist's role - aspect (ii). A blunt way to state it is as follows: Why would scientists think they have any competence in assessing how adverse it would be for this defendant to be wrongly associated with the crime scene mark, i.e. suffering the consequences of an erroneous determination made by the examiner? Conversely, why would scientists think they have the competence to assess how adverse it would be for society in this particular case to miss a correct identification? More generally, the question we face is: Do we want examiners to make assessments as to the relative adversity of their erroneous determinations, in particular false identifications, decisions from which they do not suffer themselves the most if they make a mistake?[22] Clearly, these are fundamental questions that cannot be answered from the scientists' perspective alone. These questions touch upon the broader topics of decision competence and decision prerogatives, tightly connected to the rights and duties of the parties at trial, and the broader values at stake in the criminal justice process.

It is true that none of the essential decision ingredients exposed in Section II.3 are original. Indeed, it is a well-known precept that one ought to be concerned about the relative adversity of decision consequences and the probability of their occurrence. And yet, although the fundamental decision ingredients are well known, examiners do not cope with them explicitly. It is a well-established principle that experts should not explicitly opine as to questions of law. There is no reason why they should be allowed to do so implicitly either. The fact-finder and the parties do not receive answers from experts to questions such as how adverse a false identification should be judged, and what the threshold probability necessary to identify should be in any individual case. No scientist can answer these questions, yet scientists identify every day (i.e., render identification decisions).

It can be useful to look at the intricacy of identification decisions from the viewpoint of more general, overarching legal concepts, such as deference.[23] This concept provides a good descriptive account of widespread current identification practice. Indeed, whenever recipients of expert information invite scientists to directly answer questions such as «is the person of interest the source of the crime scene mark?», they defer conclusions of common source to scientists.

Pairing the deferential account of expert witness testimony in the context of forensic identification with the insight provided by the decisional account outlined throughout Section II further clarifies what exactly happens in current reporting practice: not only is the identification decision deferred to the scientist, but so are crucial decision ingredients (i.e., value judgments and probabilities). Scientists may be aware of this decisional burden and at least tacitly assess the inevitable decision ingredients. But since this does not currently take place transparently, and the court is not told about the assumptions being made by scientists, they will impose their assessments onto the court. Worse than this lack of transparency are those situations in which the scientist is not aware of what it really means to identify, and the court is left uninformed about the logical underpinnings of identification decisions. In the latter case, the process will operate without safeguards against flawed modes of reasoning, as for both the expert and the recipients of expert information it remains unclear what exactly is logically and conceptually at stake with identification conclusions.

What emerges from this cross-perspective between the deferential account of forensic evidence and forensic decisionalism, is not a solution, but an original way to look at the problem of identification. Apart from providing arguments against deference, the decisional account can help fact-finders and parties better understand that forensic identification requires assessments that intimately relate to the personal situation of the defendant and, thus, that it may be in the interest of the fact-finders and parties to take a close look at how those assessments are actually made. The decisional account thus helps to empower the fact-finders and the parties, by encouraging them to adopt a more active role in the legal process when experts are called to testify. Defence lawyers, especially, may wish to keep control over the assessment of factors relevant to their clients' situation.

Our analysis thus far does not answer the question of who should conclude or decide on the identification (or otherwise) of the person of interest, but it shows that identification goes beyond purely scientific considerations. These wider considerations depend on the internal structure and policy choices of the respective legal order.[24]

2. Confronting different reporting formats

The attentive reader will have noticed that, throughout the analyses and discussion presented so far in this paper, the focus was on questions regarding facts:

Who is the source of the crime scene mark?

Let us call this the traditional identification paradigm. It differs from questions of the following kind, called here the evaluative account of forensic reporting[25]:

What is the (probative) value of the similarities and differences observed between the crime scene mark and the reference material taken from the person of interest?

and

For which (if any) of the two competing propositions do the observations provide support? - Do the observations support the proposition that the person of interest is the source of the crime scene mark, or the proposition that an unknown person is the source of the mark?

Let us illustrate the above two perspectives in further detail. Consider a comparison between a fingermark found on a crime scene and a reference fingerprint taken from a suspect. After analyzing the mark and comparing it to the print, the analyst will give consideration to the entirety of the observations, i.e. both similarities and discrepancies. Analysts who adhere to the evaluative account will focus their attention on the assignment of a probability for observing the similarities and discrepancies under the assumption that the person of interest is the source of the crime scene mark. Similarly, they will also assess the probability of making the observations given the assumption that someone else than the person of interest is the source of the crime scene mark. Such evaluations are based on the expert's knowledge, ideally informed by structured data published in the scientific literature. Personal experience in the form of comparisons performed on specimens known to come from the same and different sources, respectively, may also be invoked provided that the relevant work experience has been documented and can be disclosed for review and inspection.[26] The result of these considerations is a ratio of probabilities, commonly known as a likelihood ratio. In essence, the likelihood ratio multiplies by V (where V is the value of the likelihood ratio) the prior odds in favor of the proposition that the person of interest is the source of the crime mark, rather than an unknown person. These are the odds held by the decision-maker on the basis of the elements in the casebefore the scientific evidence is heard. It is clear at this inferential stage that the discussion focuses only on how the scientific findings inform the decision-maker's probability for the proposition that the person of interest, rather than an unknown person, is the source of the crime scene mark. As seen on the far right-hand side in Figure 3, this represents only one of the essential decision ingredients. Other elements are needed in order to decide whether - or declare that - the person of interest or an unknown person is the source of the crime scene mark.

As we have seen, the decision-maker will also need to take into account how adverse it is to wrongly associate the person of interest with the crime scene mark, compared to wrongly excluding the person of interest as the source of the crime scene mark. Contrary to the evaluation aspects of the task, for which the scientist can indeed be of help to the decision-maker, these latter elements of decision are of a very different nature, and relate to the values attached to the consequences of the decision made in that particular criminal case. The main insight here is that while inference and decision are not the same, they are connected: inference is a necessary preliminary for decision. In the context of a criminal case, this means that the scientist can - at best - help with inference, but cannot be in charge of the complete inference process,[27] and even less so of the subsequent decision stage.[28]

Let us consider now the case of an expert who works in the identification paradigm. Such an expert will also assess the probative weight of the observed similarities and differences between the crime scene mark and the reference print provided by the suspect, though this expert will use a somewhat different terminology. For example, if the expert uses the U.S. Department of Justice approved uniform language, her conclusions will be a direct statement of the probability that the person of interest, rather than an unknown person, is the source of the crime scene mark. Specifically, the expert will assert that «[a] source identification is a statement of an examiner's belief (...) that the probability that the two impressions were made by different sources is so small that it is negligible».[29] The examiner thus considers it as certain that the person of interest is the source of the crime scene mark, and has identified the person as the source of the mark.[30] We see two main problems with such an assertion.

First, it makes no sense to call a probability «negligible» in and of itself, without acknowledging clearly what is at stake. Consider, for example, the probability of eating a food to which you are allergic when you order a dish in a restaurant. When ordering, you ask the waiter whether the dish contains the ingredient to which you are allergic. The waiter replies that there could be trace amounts of the food in the dish, but that the probability is negligible. But isn't this something you should assess, considering the consequences of eating the food in your particular case? If your allergy tends to give you tickles in the back of the throat (irritating but not serious), what you consider to be a negligible probability of encountering the target ingredient in your plate will be very different from a situation in which eating the food in question could cause you an anaphylactic shock (and potentially kill you). At an extreme, imagine a case in which false identifications are considered unacceptable decision consequences on principle - and hence are to be avoided at all costs. In such a case, no positive probability for an unknown person being the source of the crime stain is «negligible» (i.e., acceptable). Hence, the use of the adjective «negligible» shows that experts indeed, at least tacitly, make judgments as to the values at play with the consequences of their decisions. This, however, poses a problem in terms of transparency, because experts will make value judgments for which they have not been mandated, and for which they have no competence whatsoever.

The second problem is that the reporting format «source identification» is very challenging to comprehend because the exact contribution of the scientific findings (observed similarities and differences) to the process of inference remains opaque. The two stages, inference and decision, are combined in a way that makes it impossible for the decision-maker to sort out what derives from science and can legitimately be expected from the scientist, and what embodies broader considerations for which the scientist holds no competence to give an opinion.

As is apparent, experts operating in the identification paradigm appropriate assessments that, following our discussion in Section III.1, fall outside of their field of expertise. Notably, experts do not have exhaustive information regarding the case. Thus, they are logically and practically unable to combine their findings with the other elements in the case; hence, there is no basis for them to determine whether the suspect is indeed the source of the crime scene mark. Besides, it is not their role to engage in decisional considerations that are of a legal nature (e.g., contemplating the question of how much doubt is allowed regarding the proposition of common source), or constitute policy matters (e.g., the question of how adverse a false identification is). In sum, an expert who identifies the source of a crime scene mark appropriates rights and competences that belong to the fact-finder. As noted by Stoney, doing so renders forensic science practice unscientific.[31]

IV. Conclusions

In this contribution, we have made an attempt to capture, descriptively and analytically, forensic identification in its most fundamental and, hopefully, least controversial terms. This has led us, in essence, to see that any person in charge of forensic identification faces the following situation:

The decision to render an identification conclusion inevitably entails one of several possible consequences; i.e., there is uncertainty about decision consequences. For example, deciding to identify is accompanied with the possibility of wrongly associating the person of interest with the crime mark. The probability of this consequence depends on how sure the decision-maker is, at the time of making the identification decision, that the person of interest is indeed the source of the crime scene mark.

One may say that this is not an original insight, but merely common-sense understanding. While we agree with this observation, we have used this intuitively tractable account only as a starting point to argue that formal analytical approaches to decision analysis, in particular decision theory, are accessible concepts for all discussants involved in forensic identification, and that full mathematical developments are not needed in order to convey general decisional aspects of forensic identification problems.

There are several key-points to be retained from our analysis. On the one hand, all main elements of decision theory can be found in forensic identification problems, in particular: decisions (e.g., the decision to render a particular conclusion, such as «source identification»), states of nature[32] (i.e., whether the person of interest or an unknown person is the source of the crime scene mark), decision consequences (e.g., a correct identification, a false identification, etc.) and expressions of (un-)desirability of decision consequences. But decision theory provides much more: it also provides assistance in the coherent combination of the various decision ingredients, and - most prominently - in how to compare rival decisions. As we have seen, this comparison involves the consideration of the various possible consequences of a decision, their relative (un-)desirability, and the probability with which those consequences may be incurred.

Taken together, these insights uncover what is fundamentally at stake with traditional identification decisions, but has hitherto been left unaddressed in broader circles of the field. Stoney concisely noted:

«For over 100 years the courts and the public have expected, and fingerprint examiners have provided, expert testimony that fuses these three elements: offering testimony not as evidence, but as proof, assuming priors and including decision making preferences. This created an overwhelming and unrealistic burden, asking fingerprint examiners, in the name of science, for something that science cannot provide. As a necessary consequence, fingerprint examiners became unscientific.»[33]

Stoney thus acknowledges that identification amounts to a decision that involves value judgments, but advises that this goes beyond the scientist's area of competence - anticipating thus the conclusions of our discussion (Section III).

It is useful to clarify what exactly the formal decisional perspective to forensic identification can do for us, and what it cannot do. A convenient way to clarify some of the main considerations is a hypothetical conversation in question and answer format, such as follows:

Q: «In a nutshell, what exactly does decision theory say?»

A: «In essence, decision theory focuses on the problem of how to sensibly determine the course of action to be taken when the person in charge of the decision may ‹gain› or ‹lose› through decision consequences because the hypotheses about the present, past or future state of the world are unknown to us to varying degrees.»

Q: «Can you be more precise, based on what elements do or should we decide?»

A: «A short answer to this is: when using decision theory to analyse your decision problem, you consider (1) what you think is probably the case (i.e., how convinced you are that the person of interest is the source of the crime scene mark), and (2) what you (don't) want in terms of values (i.e., how you value the various possible decision consequences and compare them against each other).»

Q: «Okay, but I already keep these things in mind, is there anything more that decision theory can do for me?»

A: «Here is what decision theory provides as added values: First, decision theory tells you that you do indeed focus on the pertinent aspects. Second, decision theory provides you with instructions on how to handle, in a logically sound way, both probability and preferences for decision consequences.«

Q: «Okay, but what exactly are those instructions and how do I apply them?»

A: «There is a mathematical dimension to decision theory that defines precisely how probabilities and value judgments ought to be combined, and rival decisions be compared.[34] On a more general (i.e., non-numerical) account, and applied to the context of forensic identification, decision-theoretic advice can also be given in verbal terms, such as ‹the more is at stake, the more you shall be sure before you decide›.»

Q: «This still does not tell me how exactly I should assess the ‹stakes›, how sure I should be and, finally, who ought to decide.»

A: «You are exactly right: decision theory makes no prescription whatsoever about how and by whom it is to be applied. It is a general theory. It provides decision-makers, whoever they may be, with a framework to help them better analyse and compare the advantages and possible drawbacks of each decision.»

Q: «So, we cannot use decision theory directly in forensic identification, and thus it is of limited practical relevance.»

A: «There is no suggestion of reducing forensic identification to a purely decision-theoretic discourse, or even delegating forensic identification to an abstract theory. Instead, decision theory supports decision-makers in thinking in clear and structured ways about their decision problem in the individual case and before they decide. Beyond the application in individual cases, decision theory helps us clarify the fundamental issues at play with forensic identification, allowing all participants in the legal process to gain a better awareness of the key factors and assessments. This is a contribution towards more transparency.»

The latter paragraph brings the pending key challenges for forensic practitioners and recipients of expert information to the point. Forensic identifications are decisions that, in the end, still need to be made by persons. Formal theories cannot relieve us from this burden. It is up to the forensic and legal communities to use insight from formal analyses wisely, especially for allocating decisional duties and defining respective areas of competence of both forensic experts and legal practitioners in a better-informed and more transparent way. In this sense, we are of the opinion that forensic examiners should limit themselves to opining on the probative value of the evidence, and refrain from opining on the probability of the hypotheses at play in a given criminal case, as such opinions always entail assumptions that lay in the competence of the fact-finder.



[1] In cases where database searches were conducted, it is sometimes thought that an exhaustive comparison was done. This, however, is not the case because databases are limited in size and do not necessarily represent the relevant population (i.e., potential suspects for the case at hand).

[2] The only exception to this are so-called «closed-set situations» in which there is a well-defined pool of potential sources that can be investigated exhaustively.

[3] A general example for the inversion fallacy is holding for equal two statements such as «I have a dog, hence I have a furry animal» and «I have a furry animal, hence I have a dog»; or, more technically, conflating the probability of finding the evidence if one assumes the truth of the hypothesis, and the probability of the hypothesis being true if one observes the evidence; See also Christophe Champod/Franco Taroni, Probabilités au procès pénal - risques et solutions, Revue Pénale Suisse 112(2) 1994, 194-219.

[4] Although not competent to impose rules on Swiss practitioners, U.S. forensic regulatory entities tend to influence practices in Europe as well. The guidelines and regulations adopted by such entities are also a reflection of practices in Europe and Switzerland as European and Swiss experts participate in the drafting of such documents.

[5] U.S. Department of Justice, Approved Uniform Language for Testimony and Reports for the Forensic Latent Print Discipline, 24 September 2018. Despite these cautionary notes, the U.S. Department of Justice recommends that scientists use expressions of common source, which has recently attracted criticism; see notably Simon A. Cole, A discouraging omen: a critical evaluation of the approved uniform language for testimony and reports for the forensic latent print discipline, Georgia State University Law Review 34/2018 p. 1103 ss.

[6] National Research Council, Strengthening forensic science in the United States: a path forward, Washington D.C. 2009.

[7] Without addressing the question of reporting formats, Swiss fingerprint examiners have been mandated to undergo specialized training and continuing education since the so-called «Déclaration de Soleure» adopted in 2007 by the heads of the forensic science units of the cantonal police forces. This «declaration» mandates every operational fingerprint examiner to be certified at one of three levels (depending on the difficulty of the fingerprint comparisons carried out by that person) and to renew their certification at regular intervals. While abolishing the so-called 12-point standard, this «declaration» does not prescribe particular reporting formats for expressing the conclusions of fingerprint comparisons. Currently, a majority of Swiss fingerprint examiners report their conclusions in terms of «identification», «exclusion» or «inconclusive».

[8] U.S. Department of Justice (fn. 5).

[9] In later parts of this paper, we will discuss the question of whether forensic examiners are the best-placed people in the legal process to render identification conclusions.

[10] Note that one may also assess how undesirable (i.e., adverse) decision consequences are. Throughout this text, we consider both desirability and undesirability, though we may only use one term at a time. While technically the two terms are captured with two different but related concepts, i.e. utilities and losses, respectively, the general level of discussion pursued here does not require going into these details. Readers may associate these considerations with Blackstone's famous 10:1 ratio regarding the undesirability of falsely convicting one innocent person compared against the «loss» incurred by freeing 10 culpable individuals. See William Blackstone, Commentaries on the Laws of England, 1769, Vol. 4. Reprinted by: University of Chicago Press, 1979. Note, however, that according to Kaye this ratio seems to refer to an error rate (across multiple cases) rather than an assessment of relative losses for a given case at hand. See David H. Kaye, Clarifying the burden of persuasion: what Bayesian decision rules do and do not do, The International Journal of Evidence & Proof 3/1999, p. 1 ss.

[11] Formally, a decision consequence is defined as the combination of a decision and a state of nature. For example, reporting an identification conclusion (decision) when the person of interest truly is the source of the crime scene mark (state of nature) will result in a correct identification (decision consequence).

[12] For example, when deciding whether to leave their home with an umbrella in the morning, a rational agent will consider both the probabilities of it raining that day, as well as the merit of, for example, having an umbrella if it rains, the respective inconvenience of not taking the umbrella and being rained on, or taking the umbrella and having to carry it around all day although it does not rain. See also, for instance, Franco Taroni/Silvia Bozza/Colin Aitken, Decision Analysis in Forensic Science, Journal of Forensic Sciences 50/2005, p. 894 ss.

[13] Simon A. Cole, Individualization is dead, long live individualization! Reforms of reporting practices for forensic fingerprint analysis in the United States, Law, Probability and Risk 13/2014, p. 117 ss.

[14] It is possible to develop decision trees for more than two decisions, but they may become more «bushy» (see, e.g., Franco Taroni/Alex Bieder­mann/Silvia Bozza/Paolo Garbolino/Colin Aitken, Bayesian networks for Probabilistic Inference and Decision Analysis in Forensic Science, 2nd ed. Chichester 2014). For the purpose of clarity, only two decisions are considered here. For general introductions to decision trees see, for example, Howard Raiffa, Decision Analysis, Introductory Lectures on Choices under Uncertainty, Reading, Massachusetts 1968; Dennis V. Lindley, Making Decisions, 2nd ed., London 1985.

[15] See also Alex Biedermann/Silvia Bozza/Franco Taroni/Paolo Garbolino, A formal approach to qualifying and quantifying the «goodness» of forensic identification decisions, Law, Probability and Risk (in press).

[16] Hence the word «minimax» for the decision rule that selects the option that minimises the maximum loss.

[17] An analogy to this goes as follows: the probability of ending up wet (i.e., adverse consequence) when not taking an umbrella when leaving your home in the morning (i.e., your decision of not taking an umbrella) depends on the probability of rain while you are out during the day (i.e., the probability of the state of nature «rain during the day»). So, your probability of incurring the consequence of interest, getting wet, corresponds to your probability of rain.

[18] Alex Biedermann/Silvia Bozza/Franco Taroni, Normative decision analysis in forensic science, Artificial Intelligence and Law (in press).

[19] A well-known decision criterion in this context is the criterion instructing decision-makers to select the decision that minimises (maximises) expected loss (utility) (e.g., Lindley, fn. 14).

[20] Note that such decision points answer the questions asked at the beginning of this section.

[21] For different numerical examples in the context of forensic identification see, for example, Alex Biedermann/Silvia Bozza/Franco Taroni, Analysing and exemplifying forensic conclusion criteria in terms of Bayesian decision theory, Science & Justice 58/2018, p. 159 ss. ; Alex Biedermann/ Silvia Bozza/Franco Taroni, The decisionalization of individualization, Forensic Science Inter­national 266/2016, p. 29 ss; Alex Biedermann/ Silvia Bozza/Franco Taroni, Decision theoretic properties of forensic identification: underlying logic and argumentative implications, Forensic Science International 177/2008, p.120 ss. For accounts in legal contexts see, for example, Kaye, footnote 10, and Richard D. Friedman, The Elements of Evidence, 4th ed., St. Paul 2017.

[22] We acknowledge at this point that our discussion is entirely focused on the person of interest. The relative adversity of erroneous determinations may also be looked at from the viewpoint of examiners, their organisational units etc. (e.g., impact on reputation, financial loss due to compensation allocated to wrongly identified person).

[23] Ronald J. Allen/Joe S. Miller, The common law theory of experts: deference or education? Northwestern University Law Review 87/1993, p. 1131 ss.

[24] Alex Biedermann/Kyriakos N. Kotsoglou, Decisional dimensions in expert witness testimony - A structural analysis, (in press): David R. Mandel et al. (Eds.)., Frontiers in Psychology; Judgment and Decision Making Under Uncertainty: Descriptive, Normative, and Prescriptive Perspectives.

[26] See ENFSI (fn. 25).

[27] Richard D. Friedman, Controlling the jury-teaching function, Seton Hall Law Review 48/2018, p. 815 ss.

[28] Specifically, in a likelihood ratio approach, the scientist would convey that it is impossible to tell - from a scientific point of view - what the probability is that the defendant is the source of the crime scene mark. For example, in case of a likelihood ratio greater than one, the scientist may give a statement along the following lines: «I cannot tell you what the probability is that the defendant left the fingermark because I do not have a suitable prior probability for that proposition (i.e., before considering the results of my analysis). This prior probability depends on the strength of the other evidence for and against the defendant, which is an assessment that lies beyond my area of competence. All I can tell you is that, whatever your probability is that the defendant rather than an unknown person is the source of the crime mark, based on my analysis, this probability is higher than it was before I did my analysis.»

[29] U.S. Department of Justice (fn. 5).

[30] William C. Thompson/Joëlle Vuille/Franco Taroni/Alex Biedermann, After uniqueness: the evolution of forensic science opinions, Judicature 102/2018, p. 18 ss.

[31] David A. Stoney, Discussion on the paper by Neumann, Evett and Skerrett, Journal of the Royal Statistical Society: Series A (Statistics in Society) 175/2012, p. 399 s. Some writers have gone so far as to recommend that «(...) experts should abandon the identification/individualization conclusion altogether» Christophe Champod/Chris Lennard/Pierre Margot/Milutin Stoilovic, Fingerprints and other Ridge Skin Impressions, 2nd ed., Boca Raton 2016 (at 96).

[32] Associated to states of nature are probabilities, expressing one's uncertainty regarding which state of nature actually holds.

[33] Stoney (fn. 31), p. 400.

[34] As noted earlier in Sect. II.4, the mathematics of decision analysis are beyond the scope of this paper.