Would We Accept Bureaucracy If It Was Impersonal?

In a prior post, we talked about the absurdity of two of the assumptions underpinning Weberian bureaucracy theory. These assert the notion that bureaucracy is an impersonal machine in which experts practiced their craft unimpeded by politics or differences of opinion as they built careers around their expanding skillset in an ideal, self-energized administrative organism.

In a tautology fit only for an academic, bureaucracy is defined to be the paradigm application of unbiased expertise, ergo bureaucracy is the epitome of robust productivity, unencumbered by human frailty.

(We might add that the proponents of bureaucratic purity may be victims of the No True Scotsman fallacy. Examples purporting to describe bureaucracy in action are not actually bureaucracy. True bureaucracy has never been put into practice.)

When we start talking about removing the human from a mechanical process, we mean robots. In a software context, we refer to machine learning. Increasingly, this means artificial intelligence.

In ancient history, this might have meant expert systems in which we specify a set of rules and people walk through the decision tree automatically. There is no discretion. Answer some questions and you get to an answer.

“In artificial intelligence, an expert system is a computer system emulating the decision-making ability of a human expert. Expert systems are designed to solve complex problems by reasoning through bodies of knowledge, represented mainly as if–then rules rather than through conventional procedural code.^[2] The first expert systems were created in the 1970s and then proliferated in the 1980s.^[3] Expert systems were among the first truly successful forms of artificial intelligence (AI) software. An expert system is divided into two subsystems: the inference engine and the knowledge base. The knowledge base represents facts and rules. The inference engine applies the rules to the known facts to deduce new facts. Inference engines can also include explanation and debugging abilities.”

When we see “if-then rules,” this suggests a flow chart. The system asks questions. The user provides answers. Based upon the response at any given node, the system will ask another question, obtaining incremental information. This continues until the system feels it has sufficient information to make a prediction or a recommendation. For example, the knowledge base could be a set of medical textbooks. The inference engine is a set of logical rules.

Here's another example:

Here, we’re trying to figure out what kind of animal we’re looking at.

There is a knowledge base that contains information about animals. There is a global database that stores information related to the specific animal that we seek to identify. We save the current state of the inference exercise in the global database; it is wiped clean every time we have a new animal to identify. As we ask questions and get answers (does it fly?), this information is added to the global database. The inference engine ploughs through the knowledge database, updating the state of the global database with new confirmed information and asking questions determined by this state, until we reach our goal of identifying the animal in question, e.g., a cheetah.

An expert system here employs a simplistic inference engine consisting of a series of path-dependent if-then questions. If we answer that the animal is a mammal, then it only queries the parts of the knowledge base that relate to mammals. Eventually, we have narrowed the search enough to obtain our goal.

When we look at products like OpenAI, we see that these tools are the most sophisticated inference engines man has developed to date. Trained on datasets of inconceivable scale, these inference engines know how to pose the dynamic questions and bring with them their own general knowledge base. If these is some general content online on which these engines have been trained such as Wikipedia and the answer relies in that knowledge base, then their non-linear, multidimensional thinking can generate a passable answer. If they lack the general content in their as-delivered knowledge base, then they’ll make something up that sounds reasonable, but is potentially very wrong. The kids refer to this as hallucinating.

If you can combine these state-of-the-art inference engines with the best domain-specific knowledge bases, now you’re cooking with gas. This is called retrieval-augmented generation.

What’s more, the inference engines don’t look and feel like clumsy flowcharts. Their user experience is fantastic. It still sounds artificial but in many cases it is hard to distinguish between the machine and the man. It feels like you’re speaking with a human being.

Now that we have dynamic inference engines that can interact with users in an organic-seeming manner, can we realize the impersonal potential that Weber envisaged?

There are (at least) two problems.

One, the inference engine is the product of training on massive datasets. This means that the biases (both computational and ugly) in the datasets become embedded in their synthetic genome. We can imagine racist inference because this kind of bad behavior is here today. Trying to make socially unbiased machines that can act as bureaucratic agents is not a trivial exercise. This is in addition to computational inadequacies that lead to hallucination. Whoever  controls the training of these inference engines will have real power, replicated in hundreds of thousands of interactions with real consequence.

These overlords may seek to eliminate unacceptable bias.

Or they may attempt to control for incorrect political opinions. Democracy dies in the training set.

Two, people complain about bureaucracy when the bureaucrats are human. What are they going to say when there are machines producing decisions that affect their lives, potentially contrary to their personal interests? Is this consistent with a democracy? How would we appeal a decision that we didn’t like? Would it be to a higher-ranking robot overlord? How would we contest a deep learning model that internalized the racist assumptions of its developers? How would we deal with a large language model predicated upon a set of baseline political assumptions with which we disagreed or that have not been codified in legislation?

This last point is a tangible risk in the context of so-called effective altruism, a philosophy that appears to combine utilitarianism with technological proliferation at scale.

“For instance, some charities help 100 or even 1,000 times as many people as others, when given the same amount of resources.

“This means that by thinking carefully about the best ways to help, we can do far more to tackle the world’s biggest problems.”

It sounds great, as if there was a way to optimize the amount of good one can do. Underpinning this fantasy is the notion that tradeoffs don’t matter. Everything is an arbitrage, in which we can improve the world, without apparent cost. There are no difficult decisions to make.

In ordinary circumstances, this wouldn’t matter. But effective altruism is at the heart of some of the biggest technology food fights happening right now. It may have been central to the bizarre power struggle for the leadership of OpenAI. It was something that figured highly in the federal trial of Sam Bankman-Fried. Hey, if you’re going to save the world, there is no problem embezzling money and defrauding people because altruism.

“Some have argued that what Bankman-Fried seemed, in his enigmatic way, to be saying to Vox was not that his vow to uphold E.A.’s ideals was merely a cover story but something like the opposite: he was, in fact, so committed to the greatest good for the greatest number that he was unwilling to observe the kinds of everyday ethical niceties that hedge naïve utilitarian calculations. “

The problem with philosophy (or religion) is that you can use it to justify anything, good or bad. The problem with people that do things in complex systems is that they cannot understand the full impact of their actions, even more so at scale.

What does this mean for our predictions of the future of bureaucracy?

Theory: replacing humans in the bureaucratic process with generative AI agents has the potential to introduce massive efficiencies. However, it may be impossible to render bureaucracy impersonal in the Weberian ideal because of biases embedded, by coincidence or design, in the inference engine. Human control is paramount. Impersonality is impossible.

AI governance may turn out to be the most important question of our lifetimes.