Can Machine Learning Bring More Diversity – and Maybe Some New Thinking and Insights Too – to the U.S. Sentencing Commission?
I had the great honor of serving on the U.S. Sentencing Commission over much of the last two years under the leadership of Commission Chair Judge Carleton W. Reeves. Judge Reeves is a gracious man and a strong leader. He speaks eloquently and lovingly about almost everything, including his home state of Mississippi and hometown of Yazoo City, the federal judiciary and its role in bringing about justice, and more recently, federal sentencing law and policy. He does this despite the fact he has deep – and justified – reservations about aspects of each of them.
In talks he has given about federal sentencing, Judge Reeves regularly – and proudly – points to the diversity of the Commission as one of its great assets. In an article written with his Counsel, Con Reynolds, and which will be published in the February 2025 issue of the Federal Sentencing Reporter, Reeves notes that federal sentencing policy, especially after the Supreme Court’s decision in Booker v. United States, requires legitimacy to be effective. The Demand for Democracy in Sentencing, available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4884638. Judges, prosecutors, defense attorneys, probation officers, and the general public must see the federal sentencing guidelines as legitimate and wise, Reeves posits, if the Sentencing Reform Act is to achieve its promise.
And Reeves argues that to achieve legitimacy, federal sentencing policy must be democratically developed. In the article, he and Reynolds lay out various ways the Sentencing Commission can realize its “democratic promise.” Among those is through the diversity of its membership. Reeves observes in the article that “[s]ome commissioners are men. Most are women. They are Black, White, Asian, and Latino. Some were born into citizenship. One applied for and earned that citizenship. Some have served as prosecutors. Some have served as public defenders.” He notes that the commissioners have seen the federal criminal justice system from the vantage point of attorneys, policy makers, advocates, and as relatives of crime victims and those convicted and imprisoned. Reeves and Reynolds conclude –
“It is difficult to overestimate the impact of this diversity on the agency’s work. Just look at the ways in which the Commission operates at public hearings. The questions Commissioners ask witnesses reflect a remarkable range of backgrounds, perspectives, and experiences. That diversity lets us solicit better reflections and input from our witnesses. And, more importantly, it lets Commissioners’ internal deliberations produce the same kinds of outputs.”
Another Dimension to Diversity
I deeply admire both Judge Reeves and the genuine diversity of the Commission that brings a freshness to the agency and a better understanding of the impact its work. There is at least one aspect of the Commission’s membership, though, that is not particularly diverse and that leads to some staleness of thought. Law school is the formal foundational education that each and every one of the commissioners share. And law practice is the foundational professional experience they all share too. While there are important skills and wisdom that come from these experiences, there also genuine and important gaps in knowledge left at the Commission by virtue of these shared and limited experiences.
While working on crime policy at the Justice Department, I saw these knowledge and skills gaps in plain sight there too, and yet so often missed or ignored. The leadership of the Department of Justice has always been comprised of lawyers: Attorneys General of one sort or another, with titles tied to variations and combinations of Deputy, Associate, Assistant, and Acting. And as is wont to be, these very smart lawyers hire other very smart lawyers to assist them. The result is Department leadership offices stocked with an abundance of great writers and great doctrinal legal scholars, and a scarcity of the many other skills necessary for great policy making.
These gaps manifest themselves, at the Department and at the Commission, in many ways and not infrequently. Among them are the inability to properly diagnose policy problems and then creatively develop solutions to those problems. Even developing hypotheses about what the problems might be can be beyond the grasp of many lawyers. We don’t have training in policy and data research and analysis, and so we are often stuck in a tired and confined set of ideas about what works to improve public safety and what is working and what is problematic in the delivery of justice. We hear the old song from some, over and again, that law enforcement and prison work to reduce crime and that the answer to too much crime – or more severe crime – is more prison and more law enforcement. And we hear a different old song over and again from others that treatment is what works. Similarly, on the delivery of justice, we instinctively and without much reflection focus our concern around unwarranted disparities in criminal justice on race, ethnicity, and gender bias. But is there more?
Learning from Economists and Machines
I’ve been thinking through these issues and whether my hunches are right that there may be different and hidden diagnoses and answers to some of our policy issues with the help of several articles by Jens Ludwig and Sendhil Mullainathan, PhD economists at the University of Chicago. I happen to have some economist relatives, some of whom kindly send me articles from economics journals that touch on crime and sentencing. As I read the articles, they strike me as on a different level of sophistication and usefulness than what is often found in law reviews. Economists and their PhD students seem, to the untrained eye, to be doing among the best criminology, for what they do is dive deeply into datasets of various kinds and then often reveal from their analyses new insights. Criminal justice is awash in datasets – often incomplete and siloed, certainly – and these datasets are ready fodder for doctoral candidates and PhDs alike.
Ludwig and Mullainathan recently published an article in the Quarterly Journal of Economics, Machine Learning as a Tool for Hypothesis Generation. Ludwig is the Edwin A. and Betty L. Bergman Distinguished Service Professor at the University of Chicago’s Harris School of Public Policy. He is also the Pritzker Director of the University of Chicago’s Crime Lab, co-director of the Education Lab, and co-director of the National Bureau of Economic Research’s working group on the economics of crime. His current work focuses on how behavioral science and data science can help solve social problems. Mullainathan is the Roman Family University Professor of Computation and Behavioral Science at Chicago’s Booth School of Business. His current research uses machine learning to understand complex problems in human behavior and social policy.
The article suggests that even at the stage of developing a hypothesis about what works and what is broken in criminal justice and sentencing, as we lawyers may be creatively limited, machine learning algorithms can help identify new patterns that we might never see. Ludwig and Mullainathan illustrate this by looking at judges’ decisions about whom to detain pretrial and whom to release on bond. They put together various intersecting datasets from the courts and law enforcement – including mug shots of arrestees – and discover that a single previously unidentified factor, a factor sitting in plain sight, emerges as having a large explanatory power for judges’ detention decisions: the defendant’s face. Their discovery is not about the age or race or skin color of the defendant, for when they control for age, gender, race, skin color, and even the facial features suggested by previous psychology research, “none of these factors (individually or jointly) meaningfully diminishes the algorithm’s predictive power.” The algorithm and their research methods suggest that defendants with “well-groomed” and “heavy or full” faces are treated much more favorably than others. These features are, of course, unrelated to the legally relevant factors for the detention decision and are thus important to identify to help judges and policy makers ensure just and equal application of the law.
Of course, hypothesis generation is not hypothesis testing. Each follows its own logic, and each requires different methods and approaches. In their article, Ludwig and Mullainathan make clear that “[w]hat is needed to creatively produce new hypotheses is different from what is needed to carefully test a given hypothesis. Testing is about the curation of data, an effort to compare comparable subsets from the universe of all observations.”
Beyond diversity, one of the Commission’s other great assets – perhaps its greatest asset – is its enormous repository of presentence reports, plea agreement, judgment and commitment orders, and other court documents. One of its great powers is its ability to seek data and the assistance of other federal criminal justice agencies. Section 995(c) of Title 28, United States Code, mandates that “[u]pon the request of the Commission, each Federal agency is authorized and directed to make its services, equipment, personnel, facilities, and information available to the greatest practicable extent to the Commission in the execution of its functions.”
Imagine the possibilities if the Commission obtained data from federal law enforcement, federal prosecutors, the Bureau of Prisons, and federal probation, and then set machine learning to work. And imagine if the Commission would recognize and tap into – on the Commission itself or otherwise – the expertise of behavioral scientists, economists, psychologists, and data scientists.
When Judge Marvin Frankel first advocated for a “Commission on Sentencing” in his classic work, Criminal Sentences: Law Without Order, he envisioned a commission that would include “lawyers, judges, penologists, and criminologists ... sociologists, psychologists, business people, artists, and, lastly for emphasis, former or present prison inmates.” We need to remind ourselves of this wisdom from Judge Frankel now and then; that people who look at the world differently from us have much to contribute. Lawyers are great. A diverse set of lawyers are even better. But even more diversity at the Commission – by tapping into new people, new skills, new backgrounds, and machine learning too – might be better still and might bring with it some new thinking and insights and perhaps better sentencing policy.