What Neuroscience Can and Cannot Answer

Reverse-Inference Errors

A common error I encounter in the presentation of neuroevidence is the reverse-inference error. Generally, this is an error of inference that arises because not all logical inferences are symmetrical. For example, people who go to funerals wear black, but it would be an error of logic to assume that all people who wear black go to funerals. The reverse-inference error is especially prevalent in the interpretation of brain activity in functional neuroimaging studies.

Take for example, a neuroscience expert's claim, relying on quantitative electroencephalogram (qEEG) data, that an individual's amygdala is abnormal and overactive. In addition, based on overactivity and the amygdala's known role as the brain's fear center, the defendant likely had overwhelming levels of fear at the time of an alleged offense, thus arguing for diminished culpability.

Before addressing the reverse-inference error here, it is worth quickly mentioning other problems with this reasoning. qEEG signals have not yet been adequately characterized in the general population, and definitions are needed to distinguish what is a normal or abnormal signal in the first place. Further, even if abnormality could be established, the field currently lacks (with rare exceptions⁹) adequate studies that correlate qEEG signals with legally relevant functional impairments. Without these, qEEG remains unable to distinguish abnormal signals that are simply statistical (e.g., rare but asymptomatic variants) from abnormal signals that imply impairment. Because of these known limitations, the American Academy of Neurology and the American Clinical Neurophysiology Society have adopted a position that recommends against the use of qEEG in civil and criminal judicial proceedings,¹⁰ although it should be noted that there are proponents of qEEG that dissent from this position.¹¹

In addition, there is the problem of time: because people do not walk around wearing scanners, neuroimaging evidence presents information regarding brain structure or function after the fact. Because the brain is such a dynamic organ, one cannot reliably reconstruct from a neuroscan the brain's function at the time of the index event. There is also the question of ecological validity: is measuring the brain activity of an individual who is instructed to do nothing for two minutes in a laboratory setting relevant to brain activity during the alleged offense?

However, the most pernicious error here, one that is not easy to spot, is the claim that because the amygdala is the fear center, activity there indicates that the defendant was experiencing high levels of fear. It is certainly true that many studies have identified the amygdalae (there are two of them, one on each side of the brain) as critical processing centers for the experience of fear. Thus, it would be correct to say that activity in the amygdala may indicate the individual was experiencing fear. However, because the amygdala is active in many other circumstances, it is a reverse-inference error to conclude that amygdala activity necessarily indicates a fearful state.

Initial work focused on amygdala activity triggered by threatening and fear-inducing stimuli¹² because these kinds of stimuli were widely available and evoked robust findings, thus earning the amygdala the reputation as the fear center of the brain. However, later research found that the amygdala is activated in other situations as well, when viewing pictures of donuts,¹³ for example, but only when the subject was hungry, and photographs of seminude women and interesting and novel objects,¹⁴ such as a chrome rhinoceros. Over time, the unifying theory that has emerged is that the amygdala is a salience detector, activating to alert the person to a large variety of stimuli (see Figure 1 in Ref. 15) determined to be important to his needs.¹⁶

Beyond the amygdala, functional imaging studies have demonstrated that generally, brain areas are activated across a very large set of conditions.¹⁷ Phrenology, a pseudoscience invented and developed by its founder Joseph Gall in the 18th century, is rightly ridiculed today because of its simplistic one-to-one model that mapped mental functions (“secretiveness,” “mirthfulness”) to single points on the brain. It is generally accepted now that brain functions are indeed localized (functional specialization¹⁸), but only to a certain extent. The consensus view of modern neuroscience is that the brain accomplishes its tasks by dynamically recruiting networks of interconnected brain modules that combine to process and compute the required solution, a model called distributed processing.¹⁹ This model is analogous to the design of computer circuit boards, which contain interconnected specialized chips that combine dynamically in different configurations, depending on the task at hand.

The reverse-inference error in this case involves qEEG, but because the problem arises from the basic design of the brain (brain areas do multiple things), it applies equally to all other modalities that purport to measure brain activity, such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET). Functional MRI and PET do not measure brain activity directly, but rather signals that derive from neurovascular correlates of brain activity.

The bottom line: forensic psychiatrists must be very wary of assertions in which the presence or absence of activation of a given brain area (e.g., amygdala and frontal lobes) is interpreted to mean that the person experienced a specific mental state. Because all known brain areas are involved in multiple processes, knowledge of activity of a single area cannot by itself establish what that brain area was doing at the time. Because the amygdala activates to threatening images, sexual images, donuts, and chrome rhinoceri, knowledge of amygdala activity alone does not necessarily mean the person was experiencing fear. Not everyone who wears black has been to a funeral.

The Group-to-Individual Inference Problem

The other broad class of error that I frequently encounter involves faulty claims that ascribe functional impairments to localized brain defects in an individual. For example, a structural MRI reveals a brain defect in the frontal lobe, which is then used to justify the assertion that because of the defect, the person has impaired impulse control or impaired rationality. At first glance, this assertion seems reasonable. After all, it is generally accepted, based on a vast amount of clinical evidence and basic research, that the frontal lobes play an important role in cognitive control and decision-making,²⁰ and that individuals with defects in frontal lobe areas such as orbitofrontal cortex, the area of frontal cortex adjacent to the orbits, exhibit impaired impulse control and impaired decision-making, among other findings.²¹

However, let us consider a famous example from the neurolaw literature: the case of Herbert Weinstein.²² This case is considered a landmark criminal proceeding in neurolaw, as it is the first known attempt in New York to use neuroimaging to argue for insanity.²³ Mr. Weinstein, an advertising executive in his mid-60s with no prior psychiatric or criminal history at the time of the incident, was accused of, and later confessed to, killing his wife by throwing her out the window of their 12th-story apartment after a heated argument.²⁴ A structural MRI was obtained after the act, which revealed a large, left-side arachnoid cyst. Subsequent PET scans established glucose hypometabolism in the area of the cyst, as well as surrounding areas.²⁵

Mr. Weinstein's lawyers signaled their intent to use the neuroimages at trial to establish that he was insane. The essential neuro claim made by his team was that Mr. Weinstein's arachnoid cyst impaired his rationality. A Frye²⁶-type prehearing was held in which the judge ruled the scans admissible. However, Mr. Weinstein agreed to a plea deal of manslaughter, and the matter never went to trial. His lawyer suggested that “the prosecutor would never have agreed to a plea if the judge had excluded the PET evidence” (Ref. 27, p 26N).

I encourage readers to view Mr. Weinstein's brain scans, which are widely available on the web and in several journal articles.²⁷ The cyst is impressive, and based on what we know about the function of the frontal lobes, its placement certainly raises the possibility that it impaired his impulse control and rationality. By themselves, the scans cannot answer whether he was impaired, or if impaired, whether the cyst was the cause.

The problem is biovariability, which limits our ability to predict impairments in individuals despite knowledge of averaged group effects of brain defects. This is a well-known problem in the neurolaw literature: the group-to-individual (G2i) inference problem.²⁸ Studies that identify associations of brain defects with impairments typically do so by comparing a group of subjects with a localized defect to a group of subjects without the defect (“healthy controls”). For a hypothetical example, a group of 10 patients with strokes in the orbitofrontal cortex (OFC) is compared with 10 healthy subjects on a test of impulse control and are found to differ on this measure. Inevitably, however, the curves overlap; some stroke patients will have better impulse control than some healthy controls, and some healthy subjects will have worse impulse control than some stroke patients. The problem of overlapping curves is the reason so few neuroimaging-based tests are used in psychiatric diagnosis. Most such tests lack sufficient sensitivity and specificity to be reliable enough for inclusion in diagnostic criteria.

How is it possible that a person can have a brain defect yet not have symptoms? There are several known sources of biovariability that make individual predictions of brain impairment devilishly tricky. Impulse control, like any other complex behavior, depends on the function of many brain areas, some of which can compensate for the other if damaged (the concept of neural redundancy²⁹). Genetic differences between individuals can result in widely divergent recruitment of brain areas for similar tasks. For example, many lefthanders invoke different brain areas compared with righthanders in language processing.³⁰ In addition, for many functions, we have more brain than we need, and thus a certain amount of neural loss can be tolerated before impairments are noticeable. This is the concept of cognitive reserve,³¹ which explains why the symptoms of Alzheimer's dementia, for example, are often not apparent until decades after brain damage is thought to begin. It is also worth keeping in mind that neuroplasticity can compensate up to a certain point for brain loss, especially if the loss is slow, as in aging³² or a slow-growing tumor.³³

Studies of arachnoid cysts in medical populations indicate that arachnoid cysts in adults are a frequent finding, and although some are associated with functional impairment, in fact most cases are asymptomatic,³⁴ obviously limiting the predictions one can make about the functional impact of such cysts in individual cases. Based on its location and size, it is plausible that Mr. Weinstein's cyst contributed to behavioral impairments and thus potentially is relevant to finders of fact, but because of biovariability, the neuroimages alone cannot establish whether he was impaired, nor can it establish, if impaired, to what extent the brain defect was a contributing cause. Furthermore, neuroscience currently lacks the evidence base to predict, based on neuroimaging, how likely cysts like Mr. Weinstein's cause impairment.

These limitations are consequences of the group-to-individual inference problem in neuroscience. Beyond arachnoid cysts, the inability to make individual predictions is a general problem for any claim that a localized brain defect is responsible for a functional impairment in an individual or that an impairment is caused by a particular brain defect. For this reason, the first neurolaw arguments that have gained traction in the U.S. Supreme Court are group-based arguments, for which we can make more confident inferences: Roper v. Simmons,³⁵ which prohibited the death penalty for juveniles as a class; Graham v. Florida,³⁶ which prohibited life without parole for juveniles in nonhomicide offenses; and Miller v. Alabama,³⁷ which prohibited mandatory life without parole sentencing for juveniles.

How can neuroscience as a field move beyond describing groups to making accurate individual predictions? Recent studies that have examined the causes of lack of replicability³⁸ in published research have made clear that neuroscience researchers should sharpen their game. Neuroscience as a field is hindered by underpowered study designs that involve sample sizes that are too small. Not only do researchers fail to detect real effects, but of more concern, they may also falsely determine null effects to be real. In a recent meta-review, Szucs and Ioannidis³⁹ estimated that more than 50 percent of published research findings in psychology and cognitive neuroscience studies are likely to be false. This is a fundamental problem in the field and will only improve with better study designs that include larger sample sizes.⁴⁰

Neuroscience must also embark on large normative studies to understand the prevalence rates of brain defects and functional impairments in the general population. As discussed, small studies in individual laboratories can be useful for demonstrating proof of principle (brain defects in area X appear to cause impairment Y), but such studies cannot assess the strength of the causal relationship (akin to the genetic concept of penetrance). To answer the question of how likely is brain defect X to cause impairment Y, we must have a sense of how many people with the brain defect have impairment and how many do not (if many people have the brain defect but not the impairment, the causal relationship is weak). To answer the inverse question of whether impairment Y is likely to be caused by brain defect X, we must know how many people with impairment have the brain defect, and how many do not (if many people have the impairment but not the brain defect, then another cause is the more likely explanation).

For the testing specialist, the challenge is to ascertain the predictive value of a given brain defect on a proposed functional impairment. Sensitivity and specificity can be estimated with small studies, but ascertaining predictive values requires knowledge of prevalence rates of the defect and impairment in the relevant population.⁴¹ For the nonspecialist, the basic concept to grasp is that without large surveys of brain structure and function in the general population, we cannot know how many people are walking around with brain imaging anomalies but are functioning normally, because such individuals rarely come to the attention of research studies.

Findings of brain defects in individuals may raise valid and plausible claims of impairment. However, because many brain defects do not result in impairment, neuroimaging alone cannot establish, except in rare cases,⁴² whether an individual is impaired, or, if impaired, whether the brain defect is the cause. Neuroscience currently lacks large normative studies that are needed to quantify whether it is likely that a defect in an individual will cause functional impairment.

A Hypothesis Generator

Although neuroscience's proper role in the courts is limited by the problems mentioned above, I also believe that neuroscience evidence can be very useful. As others have opined,⁴³ it may be helpful as one component of an analysis that integrates psychological and behavioral perspectives. As I have already stated, problems arise when neuroevidence is incorrectly viewed as a confirmatory test, when in fact, it is best suited for use as a hypothesis generator.

Neuroevidence may effectively generate hypotheses, but generally cannot answer them. Perhaps this is inevitable, considering the vast complexity of our brains in comparison to the miniscule amount that we know. I have found that although neuroevidence is rarely dispositive on its own, it can be very useful to direct and support other kinds of evidence, such as neuropsychological testing and old-school behavioral analysis. These three types of evidence work well together because they can compensate for each other's relative weaknesses, while combining their strengths.

Integrating Neuroimaging, Psychology, and Behaviors

Neuropsychological testing, unlike neuroimaging for the purposes of cognitive assessment, is generally extensively validated and normed. Modern neuropsychological tests are well characterized in terms of specificity, sensitivity, and predictive values. However, it is a dry kind of evidence, abstract and statistical, limiting its persuasive impact. Relevance can be a concern as well, as it is often unclear how exactly certain neuropsychological test concepts, such as executive functioning, line up with legally relevant mental states and capacities.

Behavioral evidence is the gold standard for determining functional impairment. We are well-suited to analyze behaviors, having evolved both neural hardware (expanded areas of the brain that support theory of mind)⁴⁴ and software (folk psychology)⁴⁵ to ascribe intentions to the behaviors of others as a matter of survival.⁴⁶ However, the same areas of brain that allow mentalization also enable deception⁴⁷ because we can best deceive when we know how other minds work; behaviors can be faked, so malingering is a perennial concern.

Neuroevidence such as brain scans have several strengths. Unlike behaviors, certain kinds of neuroimaging, such as structural MRIs, are not possible to fake, aside from deceptions like switching the films, and can thus allay malingering concerns. It is worth mentioning, however, that effective countermeasures for functional neuroimaging-based tests such as EEG⁴⁸- and fMRI⁴⁹-based lie detection are known to exist. Unlike neuropsychological testing, neuroimages are intuitive and concrete (everyone understands that a “hole in your head” may cause thinking or behavior problems) and naturally command attention because of their novelty, beauty, and associations with scientific authority. However, as discussed above, neuroscience-based claims are limited by problems of reverse inference and group-to-individual inference and thus can rarely go beyond establishing that an impairment is plausible.

The presence of brain defects can certainly raise plausible questions of mental impairment, but can only rarely answer them. For confirmation, we must look to other kinds of evidence. For example, I have found that neuroimaging findings can be useful in directing relevant follow-up neuropsychological testing and bringing attention to important behavioral details that might otherwise have been missed. When the findings of biology, psychology, and behavioral analysis converge, the argument becomes very convincing.

Consider a clinical example: a patient walks into your office complaining of back pain and asks for opiates. She provides you with an extensive history of complaints and descriptions of functional limitations. As clinicians, we all know that pain is a complex phenomenon and that frequently an organic cause is not found. But how much more comfortable would you be in prescribing opiates if her case were accompanied by an MRI showing disk degeneration? Although disk degeneration by itself is only poorly predictive of back pain,⁵⁰ I think most would agree that the combination of the radiographic finding with the history makes the case much stronger.

On the other hand, what should we do if the neuroevidence conflicts with behavioral evidence? This appears to have been the case in People v. Weinstein. Careful review of Mr. Weinstein's thoughts and behaviors before and during the homicide by the prosecution's expert did not seem to support the presence of rational or volitional impairment suggested by his frontal lobe cyst. According to that expert, Mr. Weinstein attempted to hide and destroy evidence after the homicide and attempted to stage the crime scene to make his wife's death appear to be a suicide. To find behavioral evidence that could corroborate or disconfirm the presence of cognitive impairment, the expert examined “personal writings, journals, datebooks, calendars, checkbook records, and financial records … for a three year period surrounding the time of the offense” and concluded “this analysis showed no evidence of impairment or change in his management of his everyday affairs” (Ref. 51, pp 191–192).

When behavioral evidence conflicts with neuroimaging findings, in general the high percentage move will be to side with the behavioral, because neuroscience is so poor at predicting individual outcomes of brain defects. In other words, at this point, in most cases careful behavioral analysis continues to be more reliable than neuroimaging in ascertaining the relevant mental states, capacities, and behaviors that form the actual basis of legal criteria. Of course, analysis of thoughts and behaviors is the cornerstone of good forensic psychiatric work, and for this reason we do not have to fear that neuroscience is going to put us out of a job anytime soon.⁵²

Future Developments in Neuroscience

I have spent much of this editorial sketching out neuroscience's evidentiary limitations, but the envelope is pushed with each advance. Neuroscience continues to experience stunning progress in several important areas. In the basic sciences, optogenetics,⁵³ a technology invented by psychiatrist Karl Deisseroth in 2005,⁵⁴ continues to reap rich rewards. This technology, which allows researchers to precisely target individual brain circuits in a living brain and turn them on and off with light, has vastly accelerated our functional understanding of neural circuitry. Another technique invented in his laboratory, CLARITY,⁵⁵ renders the brain transparent and, coupled with fluorescent molecular dyes, has allowed us to see for the first time intact brain circuits that traverse the whole brain. At the other extreme of the scale, advances in computing power are enabling researchers to create automated three-dimensional reconstructions of electron microscope slices of brain, albeit, in small volumes thus far, at molecular scale resolution.⁵⁶

Regarding more clinically relevant imaging, the magnetic strength, and therefore resolution, of MRI machines continues to advance. Most modern scanners have three Tesla (T) magnets that can resolve brain tissue down to 1 mm (a 1-mm³ block of brain contains approximately 20,000 neurons),⁵⁷ but the most powerful MRI machine under construction will surpass them all at 11.75 T, which is expected to be able to resolve brain tissue down to 0.1 mm.⁵⁸ Furthermore, magnetic particle imaging (MPI) promises to increase significantly the resolution of functional MRI by injecting magnetic nanoparticles that act as contrast agents. Researchers believe that with MPI, resolutions can be boosted to the theoretical equivalent of a 30 T MRI scanner.⁵⁹

Beautiful, high-resolution images are impressive, but for legal applications, what neuroscience needs is more data, particularly in the form of large, normative survey studies, as mentioned earlier. The first of these large collaborative efforts is finally starting: the Adolescent Brain Cognitive Development (ABCD) study.⁶⁰ This ground-breaking work will collect brain scans and a rich set of neuropsychological and behavioral data on a cohort of approximately 10,000 children aged 9–10 from the general population, and track their scans and development over time. The resulting gold mine of brain–behavior correlative data will allow neuroscience experts to make far more accurate individual inferences. We will also finally get a good sense of the range of what brains in the general population look like and how they change over time.

Questions Neuroscience Will Never Answer

In closing, I emphasize that although neuroscience can inform, it will never be able to answer ultimate legal questions of culpability and desert. Such determinations are essentially moral judgments that require understanding behaviors and mental states against the backdrop of cultural norms. The human element is embedded in the law with words like appreciation, sufficiency, and reasonableness, all of which require human interpretation. Although science may prove to be helpful in ascertaining behaviors and mental states, it will always be blind to the cultural and moral context needed to judge their appropriateness in a given situation. In other words, although we may be guided by science in making moral decisions, ultimately they remain ours to make. Despite the effort it takes and the fraught nature of decision-making in which freedom, life, and treasure hang in the balance, that is the way it should be. What makes us best suited for judging other people is that we are people.