By Danny Muchoki – Founder SRA.Online.
I’ve spent a lot of time incorporating A.I. (i.e. a language model) into a HIPAA Security Rule compliance app. The experience taught me two things:
- Language models are powerful, but fundamentally limited.
- We’ve known about these fundamental limits for thousands of years.
The bottom line? As powerful as this new technology seems, without careful thought you risk losing sight of your organization’s main strategic goals.
To understand how this can happen, consider the story of Croesus.
Long ago, a rich and powerful king named Croesus [kree-sus] went to the oracle at Delphi. Croesus asked the oracle whether he would win a war against the Persian king, Cyrus the Great. The oracle told Croesus that, if he went to war, he would “destroy a great empire.” Hearing the oracle, Croesus went to war but lost. It apparently never occurred to Croesus that the great empire he would destroy would be his own.
ChatGPT is like the modern Delphic oracle for many of us. Imagine that Croesus asked ChatGPT the exact same question. It’s likely that he would have received a similarly ambiguous answer. You’ve probably had the same experience with several language models.
So, what’s the lesson here for compliance professionals?
I think it’s this: we tend to be a lot like Croesus. We tend to frame new challenges in narrow ways. But when we do that, we often lose sight of our organization’s strategic goals.
Narrowly framing a question seems natural. It seems almost scientific. We aim to avoid ambiguity (or to find certainty) by focusing on a few key explanatory variables. After all, we only have so much time in the day and only so many dollars in the bank. The consequences of compliance failures can be steep. A great organization could be destroyed by loss of revenue, ruinous fines, destroyed reputations, loss of trust, or key personnel spending time in jail.
Language models work by finding patterns in language. Lots of compliance work involves finding patterns in organizations. Thanks to language models, we can now find patterns in organizational policies and procedures on a massive scale. Language models are a powerful resource which we should use as compliance professionals.
As we go finding patterns, let’s remember that ambiguity is a feature of language. Yet language models have no consciousness or intent. Language models do not know the use and purpose of ambiguity. This is where language models are fundamentally limited.
Whether or not the story of Croesus is completely true, look at how the story presents ambiguity.
First, the oracle answered Croesus’ question on going to war. Croesus’ question did not consider other options (i.e he was trapped in tunnel vision). Croesus framed the question narrowly, so the oracle gave a narrow answer. In language model terms, Croesus restricted the context window.
Second, the oracle never specified which great empire would be destroyed. This means Croesus did not have (or did not communicate) a clear understanding of his realm’s stature, resources, and objectives. Many organizations are in a similar position.
New technologies, risks, and challenges often focus our attention. When facing a new challenge, it’s natural to home in on the few variables we believe are important. But that focus can quickly turn into tunnel vision. We, like Croesus, often lose our ability to see the bigger strategic picture.
Imagine that Croesus asked a different/better question: “What is the best way I can defeat the Persians?” Or “Given the resources of my realm, what are the chances we can defeat the Persians?”
Even with these “better” questions, the limitations of language models quickly appear.
Imagine that the oracle/ChatGPT told Croesus to appoint a qualified security official to face the Persians. In fact, the HIPAA regulations tell healthcare organizations to appoint qualified security officials. That sounds like unambiguous solid advice, right?
Until you ask – what does “qualified” mean?
Does Croesus’ security official need to speak Old Persian? Would it be helpful if this security official knew the Persian king personally? Does the official need to have experience in warfare, or specific experience fighting the Persians, or be a good negotiator who aims to avoid war?
Which of these characteristics matters more than others? How many candidates have these qualifications? Given all these constraints, is this practical advice? Or is it not much different from a machine-generated hallucination?
Next time you reach for your favorite language model, remember the story of Croesus. Do not lose sight of your strategic goal. That means it’s up to you to bring your context to bear. Add context! Broaden your horizons and your vertices! These new thinking machines can find many patterns. Used properly, they can also be useful in nudging us out of the tunnel vision that often comes from facing new challenges.
At the same time, it’s important to be honest with yourself. Do you have enough context to frame the question? If not, perhaps consulting the oracle would be premature.
As powerful they may be and become, language models cannot decide which contexts, patterns, and perspectives matter. Through our queries, we tell A.I. which patterns to look for. We decide which patterns matter and which don’t.
We decide.
At least for now.