Researchers, journalists, and activists alike commonly set out to explore racial/ethnic disparities in the justice system and, more specifically, policing outcomes. Nowhere is this more common in policing than trying to examine “racial profiling” or “DWB” in vehicle/traffic stops. Since the early 2000s, following media attention and federal intervention into the New Jersey State Police, a number of states created data collection systems to track the race/ethnicity of all stopped motorists. The early adopters were states like Missouri and Illinois; however, there are (as of 2021) 23 states plus Washington, DC that require the collection of standardized measures when an individual on the road is stopped by police. In most cases, those department-level measures of vehicle stops across racial/ethnic groups are publicly available, and they provide opportunities for anyone to take a look.
A common practice is to then compare the race/ethnicity of stopped motorists to an external benchmark – or an estimate of the racial/ethnic distribution of drivers who could be expected to be pulled over if officers were engaged in an unbiased or neutral manner. Comparisons show whether drivers of a certain race/ethnicity are stopped equally or if they are over- or under-represented relative to the benchmark used. By far, the most frequently used benchmarks (especially early on in research) were population-based measures from the US Census Bureau. Many researchers, government reports, journalists, activists, and police organizations themselves still rely on these population-based benchmarks to this day when testing for racial/ethnic disparities.
Yet, these “first generation” benchmarks are incredibly flawed. Using the racial/ethnic composition of those aged 16+ makes a ton of assumptions, namely that everyone of age has a car and drives. There is no imaginable way to possibly account for driving behavior, such as speeding and other traffic/safety violations, across racial/ethnic groups. “Second and third generation” benchmarks – like footage from red light cameras, at-fault versus not-at-fault crashes, systematic social observation, and “veil of darkness” tests – are arguably better. Still, these measures are often difficult and time consuming to come by. In fact, I once worked with a police department that decided it would take too much time and effort to gather and synthesize a few years’ worth of their OWN crash data in order to calculate this type of benchmark! For these reasons, population-based benchmarks are used a lot due to convenience and data availability. Admittedly, I’ve used census-level benchmarks of the racial/ethnic composition of those aged 16+ in municipalities for analyses when no alternative benchmarks exist.
Related to this first-generation benchmarking problem, states who provide publicly-available stop data aggregate counts of drivers’ race/ethnicity without any other information to clarify where those individuals live. There is no specificity as to whether a stopped motorist is a resident of the municipality where he/she is pulled over OR if they are visiting/passing through. This means that all analyses that rely on population-based benchmarks are potentially biasing the presence or absence of racial/ethnic disparities in vehicle stops by lumping residents and non-residents/visitors together.
I have a research note currently under review that explores this very issue. In 2018, Missouri added a driver residency question that began distinguishing between traffic stops of residents of the jurisdiction where the stop occurred separately from the total number of stops (i.e., stops of residents and stops of non-residents). This new feature allows for the calculation of the number and percentage of stops each department makes of those who live in its respective jurisdiction versus those that do not. I limited the sample to all 126 local/municipal police departments serving jurisdictions with 5,000+ residents from the 2019 American Community Survey estimates. Here is the big takeaway:
Of the 126 departments, 267,710 of all stops were of residents in the jurisdictions where they were stopped (38.8%) and 422,351 stops were made on non-residents (61.2%). When examining residents versus non-residents at the department-level (i.e.., agency units of analysis), an average of two-thirds of all traffic stops were conducted on motorists who did not reside in the municipalities they were stopped in. Such figures create apple-to-oranges comparisons with population-based benchmarks from the US Census Bureau, and likely render the calculation of racial/ethnic disparities as unreliable in many cases. We must “proceed with caution” in any of these analyses, and I hope more states follow Missouri’s lead by differentiating whether stopped motorists live in the jurisdiction in which they are stopped.