The Chinese Social Credit System
Q: Should we be worried about the Chinese Social Credit System?
Building a road through high mountains is not an inherently bad idea but you are always a few inches away from a dangerous drop. Who is building it? Is it being built safely and wisely? Does it understand the real situation in the mountains?
We need not worry about the general premise. Instead we need to start thinking about which particular parts of the situation create more or less worrying outcomes…
NEW & WEIRD?
Social credit systems are not unique in Chinese history nor are they unique to the Chinese. Many groups — from the US Army to the Big Banks — have employed systems of credits and demerits. Shore leave? Return home? Get a mortgage to buy a home? Perform your civic duty by purchasing 9 sandwiches… and you earn a FREE SANDWHICH!
Consider the American medical system. If citizens do enough dollar-making duty, they earn the bonus reward of health for themselves and their family. We do not ordinarily think about our system in these terms but there are many ways of looking at the situation in which we are not a “free society” standing in contrast to China’s “authoritarian oppression.” It is a lot murkier than that. And we have to be suspicious of our own cultural responses. Everyone knows that a person who rejects their own aggression or eroticism or hierarchical judgments will tend to subconsciously project an amplified version of those qualities onto others. So we would not be surprised if people who are only half-conscious of being embedded in a problematic, uncertain and manipulative social credit system are very likely to get angry and anxious about the apparent credit system of the foreign Other.
We are already swimming in schemes in which putatively “good behavior” earns rewards. These are highly effective so we evolve to select them. Gaining and exchanging symbolic points is one of the simplest ways to non-violently encourage better behavior (as most parents know too well) and it is also one of the easiest ways to establish “flow states” in video games.
Credit systems are ancient, normal, popular and effective. No worried. We should, however, be highly concerned about the specifics of how these schemes are implemented. What is implied in the framework of the evaluations being used to decide how credits are assigned? What kinds of corruption could affect the system over time?
The Chinese Social Credit System is supposed to optimize for a variable translated as trustworthiness. That’s not quite the same as optimizing for the intersection of the good, true and beautiful but it’s not terrible. You can certainly make a strong argument that it is better to have a nation-run credit system which challenges corporate corruption and incentivize decent, healthy and patriotic public behavior is better than none which empowers a conflicted flotilla of semi-private credit systems aimed at exploited citizens through deception, debt, addictive carcinogenic foods and producing an anti-national morass of mistrust and erratic behavior.
Perhaps game-ifying the election with points will increase the number of voters? Perhaps if you are one the precarious and malnourished millions of people in “the West,” you would relish a change to gain more security, better food, health care by helping your neighbors, obeying traffic laws & cleaning your yard?
THE REAL PROBLEMS
Now that doesn’t mean the Chinese system is a good thing. It may in fact be (as everyone pedantically notes) “Orwellian” or even “like an episode of Black Mirror.”
The issue for me is not whether we should have mass surveillance and credit systems (we already have those) nor is it whether trust and good behavior are things that we could and should encourage.
Here are the issues:
Do we trust the Chinese State (or any state) to decide what is actually good?
Is the system opt-in or mandatory?
Is the focus on gaining benefits — or on demerits, shaming and suppressive social stress?
Is it secure from future attempts to hack it?
I do not know enough of the socio-technical details to make a plausible determine whether the Chinese Social Credit System is robust against future drift and grift that could hack it another direction.
I also think it’s fairly self-evident (practically, not just morally) that systems are relatively fragile if no one would choose them. Opt-in solutions are superior wherever they are possible. And, frankly, if a system is not useful and interesting enough for people do adopt freely then it is likely not a very good system for them. We adopt things quickly if they are easy to use and provide immediate practical or emotional benefit. And the designers/administrators of systems are usually more responsive, intelligent and decent if they are oriented toward providing happy outcomes. So from my perspective, social credit system should lean heavily toward providing opportunities rather than threatening shame and demerits for failure to comply.
And that leads back to my primary concern: who decides?
WHO ARE THEY & DO THEY KNOW WHAT IS GOOD?
Who decides what is trustworthy (the variable that the Chinese government claims to be optimizing for)? Which perspective is play? What orientation determines the evaluations about healthy, normal and productive behavior?
There are many famous (almost certainly false) tales about an legendary encounter between the Chinese social-harmony philosopher Confucius and trans-rational, naturalistic sage Lao-Tse. One such tale ends with Confucius returning to the city from a wilderness meeting with the Taoist mystic. It says that Confucius then forbid any of his students to ever speak with the wily, complex Lao-Tse.
“That is a not a man,” he tells them, “it is a dragon.”
So we could start by asking: Which China is determining the parameters of trustworthy behavior? Is this to be a Confucian ranking or a Taoist approach? And these questions are just the tip of an iceberg. We must get much better at making similar inquiries about every private and public institution that proposes to deploy the (potentially good idea) of game-ifying our civic well-being.
These deeper inquiries cluster around (at least) three crucial issues:
Use of Improper variables.
Lack of understanding about normative deviation.
Conflation of “distress” with “problem”.
VARIABLES
How do you set up your computer to turn the mass surveillance data into a reasonably good profile of a citizen? Should you be kicked off Twitter and Youtube for discussing the Lab Leak Hypothesis? Perhaps that not yet enough information to determine whether you are a sane, interested citizen or a rabid, socially-toxic, anti-science quasi-terrorist?
Consider the “AQAL model” proposed by the notoriously bald integral philosopher Ken Wilber. This model suggests that in order to represent a person you would need to evaluate their oscillating levels of complexity along at least half a dozen major developmental lines (as mediated by their typological style) expressed simultaneously but variably in their subjective responses, external behaviors and qualitatively-evaluated intersubjective relationships — not forgetting to correct for shadows and pathologies in which aspects of the self can be appear as the opposite of their true nature — and keeping mind that all of this evolves over time as your sensibilities and capacities continue to change (and therefore must not be cmopeltely pegged to your past behavior).
Wilber may or may not be correct about the details, of course, but we can certainly be forgiven for thinking that something like degree of complexity would be the minimum necessary matrix of variables for providing a viable profile a human being.
Now… which (and how many) variables are being used to represent a “person” in the Chinese system? Which ones are representing you, right now, to Google or Facebook or Homeland Security or the KGB or the private Israeli security firm that hacked your phone before you bought it?
One of the legitimate reasons that profiling, surveillance and ranking freaks us out is that our species has a long history of evaluating each other using a very small number of clumsy external variables. Try to estimate the number of times people have, essentially, given other people demerits based on the trivial issues of skin pigmentation, genitalia or place of birth. It’s a nightmare.
Martin Luther King exhorted us to judge people not by the color of the skin but by the content of their character. Very well! But what is the minimum set of factors and frameworks necessary to quantify the content of a character without disastrously omitting some key human variables?
Unless there is an open discussion and transparency around the implied intersection of variables being deployed, then we should definitely be worried about any ranking system…
NORMATIVE DEVIATION
In the 1950s there were two major scientific studies performed on human sexuality: the Kinsey Report and the work of Masters & Johnson.
Masses of previously concealed information were anonymously made public. The results shocked the normative sensibilities and surface discourse of the United States. Sexual behaviors that had previously been shamed, defamed & punished, both by law and public sentiment, were suddenly exposed as standard behaviors. Activities and predilections that were formerly regarded as unfortunate deviations from normalcy (e.g. sodomy, polyamory, cunnilingus, kink, inter-racial sexuality, cosplay, etc.) turned out to be so prevalent that they might as well be considered mainstream.
There is a non-trivial danger that the normal diversity of human behaviour can be interpreted (even for long periods of time) as something worth shaming and forbidding. We can only imagine how immoral and obnoxious it would be give to offer social credits to young homosexuals in order to entice them into psychochemical “rehabilitation camps.”
So an important question to ask about social credit system concerns how well they anticipate and appreciated normal deviance. Most of humanity’s great cultural heroes — artists, inventors, esoteric adepts, philosophers, writers, visionaries — are people who exhibit a higher statistical probability of experimenting with unconventional or socially denigrated phenomena.
To gain personal insight and creative freedom (which are important nutrients in the body of a robust, evolving society) individuals may feel the need to challenge their own anxieties around taboos and/or to temporarily explore many differently “charged” genres of experience. How else are they expected to gain a more comprehensive and integrative feeling for human existence?
For a social credit system to be healthy and productive in the long term, it would have to have
(a) some powerful mechanism to prevent itself from mistaking current social norms around shameful, perverse or deviant behaviors from being conflated with the natural diversity of healthy human existence.
(b) some significant allowance for creative experimenters to engage in intentional perversion and/or strategic taboo violation.
DISTRESS/DISASTER CONFLATION
One of proposed reasons that information about the emergence of COVID-19 was delayed, obstructed and misguided in China resides in the interface between governmental layers. The administration of Wuhan, apparently, did not want to present bad news to the national government.
This kind of face-saving is not only part of the legendary social morality of “the Far East” but also something that each human being understands. The children in my own house constantly struggle to avoid admitting any responsibility for flatulence.
My boyhood hero, the great philosopher & sci-fi writer Robert Anton Wilson, invented Celine’s Law — the principle that you always, instinctively, deceive any “superior.”
If someone can reward or punish you (and, through you, your loved ones) based on their response to your communication then you really have no choice but to skew that communication in order to leave out any worrying or failure-implying information. Of course this compounds. The further up the hierarchy you go, the lower is the level of accurate and actual understanding of the world.
The reason we have the phrase “don’t shoot the messenger” is because human beings tend to conflate the distressing information with the actual disaster. It takes special work to tease your reactions apart from realities.
Consider a scenario in which a woman is wearing partly exposed clothing which provokes, in public, an agitated inner feeling of arousal on the part of some men. Should the Law require her to be concealed in anti-sexual garments (even to the point of blaming her negligence for any sexual assaults)? Or should the law require that men, in public, learn to tolerate their distressed arousals? Which legal approach is likely, over time, to produce more sexually civilized culture exhibiting a greater sense of public trust and sophistication? Which one is likely to maintain high level of fragile distrust concealed beneath a desperate attempt to control others?
So here’s an outstanding trust problem for the architects of social credit:
What is your plan for distinguishing between “upsetting effects” and “negative behavior?”
If the provincial government of Wuhan does not trust the federal government of China to be happy to hear true bad news then it will have not choice but to conceal important information and project positive news. Anyone who has been exposed to the insipidly boastful and endlessly “successful” news from North Korea can readily understand how affirmative public communication often coincides with the worst possible social outcomes.
Every authoritarian and totalitarian system seems, over time, to exaggerate the ordinary human bullshit and contract around a fragile, anxiety-based system of public positivity & face-saving. It confused the healthy acknowledgement of failure and upset with the active causing of problems. The danger is that people loose “public credit” by bringing actual reality into acknowledgement. That’s not a recipe for a long-range, harmonious and trustworthy culture.
So you cannot have a healthy social credit system unless you have a growth mindset approach in which failures and loss-of-face are regarded positively as part of learning and growing. Ultimately, you need to figure out how to socially reward people for bringing forward upsetting information that challenges the status quo narrative…
CONCLUSION
We all face the emergence of social credit systems. It is not just a Chinese experiment. It is not just a potential over-reach of the American government. It is present, ongoing and everywhere. Such systems can be more or less comprehensive, integrated and humane. They can be good or bad depending on many factors.
Instead of worrying that they might come into existence we should be deploying all of that concern to distinguish what makes a good, empowering, healthy, non-suppressive version rather than the opposite….