On interpretable models

A black board with various mathematical symbols and equations
Photo by Thomas T / Unsplash

Hello, dear reader, and welcome to another issue of AI, Law, and Otter Things! It has been a bit more than a week since our last issue, and in the meantime the European Commission has opened a call for evidence meant to support its windmill-hunting effortsproject of simplifying the digital acquis. My colleagues working on cookies and online platforms have been shocked with the push against the ePrivacy directive, while the AI Act part has quite a few sweet nothings. So, for the time being, I will not add anything on it.

Today's issue, instead, comes back to my work on AI transparency. As I mentioned before, this is no longer a research priority for me, as my work moves more and more towards more abstract questions of regulation. Still, I find myself coming to the topic from time to time, not only because of its saliency on AI debates but also because it remains relevant for some of my more applied work. In this context, I want to revisit some thoughts on inherently interpretable models that I had developed for an abandoned project that is now being revived. After that, of course, you will find the usual content: some reading recommendations, academic opportunities, and a cute otter.

By the way, if you are at the ELU-S conference in Prague next week, make sure to say hi!

Model interpretability: what is it good for?

When one talks about technical transparency in AI models, there is an almost knee-jerk tendency of equating it with explainable AI models. However, the kind of information such models provide about opaque AI systems does not always line up with what the law expects when it demands transparency, especially when one considers that explanations are subject to manipulation precisely in the kinds of adversarial contexts where they would be the most useful. Faced with those limitations of explanation, scholars and activisms sometimes propose an elegant solution: if we cannot make all algorithms transparent, then we should prohibit the use of opaque techniques, at least in some contexts.

This restrictive approach would allow an extensive form of transparency. Whereas any transparency offered by explainable AI is mediated by how the explanation model is constructed, a restrictive approach would result in inherently interpretable models. That is, an AI system can only be used (at least in some contexts) if external observers can make sense of the inner logic that guides its operation. This promise of direct visibility would thus produce a form of transparency that is closer to the core meaning of that term, removing barriers to the scrutiny of those systems.

However, a few factors might limit the potential of inherently interpretable models as a tool for keeping algorithmic power in check. First, one might point out that the various forms of human oversight currently proposed in law often reduce humans to mere rubberstampers of automated decisions, as they do not equip human overseers with the knowledge and conditions they would need for properly inspecting and potentially overriding automated decisions. Transparency is necessary but not sufficient for control of how AI systems operate.

In addition, transparency is in the eye of the beholder. A system that a technical expert might deem perfectly interpretable might not be much clearer to the untrained eye than a complex neural network. This has been suggested, for exampe, by the empirical study of the accuracy-explainability trade-off carried out by Andrew Bell and co-authors some years ago. So, even the clearest interpretable models might be of little value for citizens trying to enforce their individual rights in contexts of automation, or for human workers trying to control systems that are formally under their responsibility. Such pointlessness might become even more pressing in cases where humans have only a limited time to respond to AI outputs, as they will likely not have the time to make sense of techniques that they could understand given time. These constraints suggest that interpretability is of limited use for some of the roles that the law expects AI transparency to play.

Nonetheless, inherently interpretable models can still be valuable in some contexts. An obligation that certain models must be interpretable unless there are good reasons otherwise might force an actor that is developing or implementing an AI system to consider whether the purported gains from an opaque model are indeed relevant in a given context. For example, Paul Ohm argues that many AI applications, especially in the public sector, do not benefit that much from the gains in performance promised by opaque models. Here, the main contribution of transparency mandates comes from tinkering with the trade-offs that organizations consider when they make use of AI technologies.

Mandated interpretability can also be useful for ex post scrutiny of algorithmic decisions. Even if individuals might lack the resources to interpret the models in real-time, entities such as civil society organizations, journalists, and the courts might be able to tap into technical expertise and take the time to make sense of what is going on within a model. In doing so, they might identify issues with actually-existing systems, a task that might lead to fixing those systems and/or to the ascription of liability from any harms caused by those issues. So long as we do not expect it to be a silver bullet, the mandated use of inherently interpretable models can indeed contribute to control over AI-powered practices.

Things you might be interested in

I finally got to watch Parlement, a comedy TV series set in the European Parliament. Being halfway through the second season, I have enjoyed it very much so far, and the show also offers me the opportunity to practice my French, as its dialogue shifts between English, French, and German (with some other languages appearing from time to time). In Luxembourg, you can watch it on Netflix, so that might be an option in other countries, too.

As for academic stuff, here are some materials I've come across recently that might be of interest to you:

Opportunities

The research seminar FAIRNESS IN DIGITAL CONTRACT LAW: EUROPEAN AND GLOBAL COMPARATIVE PERSPECTIVES will take place at the Jagiellonian University (Krakow, Poland) on 4-5 December 2025. Abstracts are due by 26 September 2025.

The BRIDGES (Brazilian Research, International Dialogue, and Global Exchange Symposium) symposium, convened by Brazilian professors based in Canada and the US, offers Brazilian early-career scholars the opportunity to get feedback on works in progress, with a view to submitting them to English-language law journals. Abstracts are due by 15 October 2025.

The School of Communication and Culture at Aarhus University is looking for a fixed-term (2 years) Assistant Professor in Cybersecurity with a Focus on Human Practices, Values and Behaviours. Applications are due by 9 October 2025, for starting on January 2026.

The research project 'Artificial Secrecy? Taking Transparency in EU Digital and Data Regulation Seriously' at the University of Amsterdam is hiring a PhD candidate and a postdoc to research the balance between transparency rules and confidentiality in regulated digital technologies and data in the EU. In both cases, applications are due by 17 October.

The IE Law School in Madrid is looking for 4 tenure-track Assistant Professors in Law. They are hiring broadly, but particularly interested in Private Law (Contracts, Torts, Property, Family Law, Conflicts of Law, etc.); Commercial, Corporate, and Finance Law; IP Law; Digital & Tech Law; Human Rights; and Environmental & Climate Change Law. Applications are due by 31 October 2025, for a start in 1 September 2026.

Finally, the otter

a small otter laying on top of a rock
"I have a cunning plan" - Photo by Hoyoun Lee / Unsplash

Hope you found something interesting above, and please consider subscribing if you haven’t done so already:

Subscribe

Thanks for your attention! Do not hesitate to hit “reply” to this email or contact me elsewhere to discuss some topic I raise in the newsletter. Likewise, let me know if there is a job opening, event, or publication that might be of interest to me or to the readers of this newsletter. Hope to see you next time!