Factors Associated With Agreement Between Experts in Evidence About Psychiatric Injury

Article Figures & Data

Table 1

Kappa Statistics for Common Psychiatric Diagnoses After Motor Vehicle Accidents

Specific Psychiatric Diagnosis	Diagnosis in Any of 148 Report Pairs n (%^*)	κ for Opposing Adversarial Role n = 97 Pairs	κ for Same Adversarial Role n = 51 pairs^†
Traumatic brain injury	36 (12)	.31	.61
Post-traumatic stress disorder	103 (35)	.27	.26
Depressive disorders^‡	88 (30)	−.12	.29
Other anxiety disorders	61 (21)	−.10	.38
Pain-related disorder	24 (8)	.26	.43
Adjustment disorder	25 (8)	.18	.19
Pathological grief	11 (4)	.18	.00
Any mental disorder (includes others, n = 5)	250 (84)	.09	.74
Average κ for all disorders	−	.14	.31

* Total exceeds 100% due to multiple diagnoses.
† Six of 51 report pairs lacked statistical independence. An analysis that excluded all the report pairs from claims with three reports from the same side found a minimally changed agreement for traumatic brain injury (κ = .62), post traumatic stress disorder (κ = .31), depression (κ = .34), other anxiety disorders (κ = .34), and any mental disorder (κ = .68).
‡ Major depression, dysthymia and other depressive diagnoses.

Table 2

Variables in the Principal Diagnosis Groups

Independent Variables	No Agreement n = 30	Agree About Mental Disorder n = 65	Agree on Principal Diagnosis n = 53		p
Report writer factors
Same expert status, n (%)	16 (53)	39 (69)	30 (57)	χ²=.40	.82
Same adversarial side, n (%)	3 (10)	24 (37)	24 (45)	χ²=10.87	.004
Same profession, n (%)	21 (70)	37 (57)	32 (60)	χ²=1.48	.48
Plaintiff factors
Male, n (%)	18 (60)	38 (58)	27 (51)	χ²=1.93	.75
Age, mean years (SD)	38.9 (7.3)	34.3 (7.0)	32.6 (8.1)	F=1.72	.18
IS score, median (IQR)	12 (16)	12 (21)	12 (21)	KW=.70	.70
Fatal accident, n (%)	0 (0)	16 (25)	14 (26)	χ²=9.62	.008

Table 3

GEE Examining Factors Associated With Agreement Between Experts

Variable	B	SE	95% Wald CI	Wald χ²	df	p
Agreement about the presence of mental disorder
Intercept	−4.807	1.262	−7.281 to −2.332	14.494	1	.000
Same adversarial side	2.157	.862	.468 to 3.846	6.268	1	.012
Fatal accident^*	1.835	.926	0.20 to 3.650	3.927	1	.048

Agreement about the principal psychiatric diagnosis
Intercept	−.253	.4167	−1.070 to .564	.368	1	.544
Same adversarial side	.989	.447	.112 to 1.865	4.882	1	.027
Fatal accident	−.168	.424	−1.00 to .663	.158	1	.691

* A conservative estimate of fatal accidents was calculated by the inclusion of a false positive in the group with no agreement about mental disorder, to prevent having a zero in the denominator.