Technology is Not Values Neutral: Ending the Reign of Nihilistic Design
We fail to take tech seriously when we do not grasp its full impact on humans | Jun 26, 2022 | 25 Min Read
©Adaptive Cultures
Verified facts can be used to support erroneous conclusions. Here is how we can put an end to that.
Fact-checking has become popularized as the definitive process for certifying truth in the media. This has occurred in response to the proliferation of a wide variety of internet subcultures, often based largely upon misinformation. Propaganda and bad faith communication are all too common, making the checking of facts an important part of sensemaking.
While fact-checking is necessary, it is often not enough to provide the whole picture. Under current conditions of escalating culture and information war, facts themselves have become weapons.[1] Neither propaganda nor bad faith communication require the speaking of falsehoods.[2] It is often more effective to mislead and misinform through a strategic use of verified facts. The ability to critique and correct for the misuse of facts in public culture is an essential component of the democratic way of life.
Unfortunately, today it is standard practice for both institutions and individuals from all sectors of society to offer strategically cherry-picked and decontextualized facts,[3] set within a predetermined emotional or ethical frame.[4] This way of using facts is an effective tool to bring some people towards previously unappealing conclusions. It also provides rhetorical ammunition to those already predisposed to drawing these conclusions. While honestly passing the scrutiny of the fact-checkers, such an approach is nevertheless far from entirely truthful.
Verified facts are collected as ammunition for culture war, rather than for the sake of gaining a comprehensive understanding.
In today’s so-called “post-truth” digital media landscapes, the practice of weaponizing facts has become widespread, microtargeted, and optimized for psychological impact. The normalization of mishandling facts threatens to undermine people’s sense of living in a shared reality. For some, it goes so far as to undermine the idea that reality can be known at all.
Democratic forms of government are now being undermined by the mishandling and misrepresentation of “facts.” Stopping our descent into a “fact-blind culture” requires a new approach to the way we pay attention to and talk about “the facts.” For those seeking to improve the state of the epistemic commons, and address 21st-century challenges to sensemaking, there is no way forward that does not involve fundamental upgrades to how “facts” are handled in public discourse.[5]
There is a growing body of literature on fact-checking as a media practice.[6][7][8][9] The fields of epistemology and the philosophy of science now have sub-branches seeking to address the crisis concerning “facts” in public culture. A thriving international movement of fact-checking is leading to the establishment of many new organizations aimed at certifying the truth. The details of these efforts can be found elsewhere.[10]
Despite often earnest effort, the recent growth of fact-checking is not making the situation obviously better. Some argue that more fact-checking is in fact making things worse. How can that be?
The answer is that fact-checking—the verification of specific claims—does nothing to address the three primary ways in which facts can be used to mislead (see the box below). Because fact-checking offers official verification, it permits easier use of facts to mislead and misinform. This sounds counterintuitive. But the more accepted a fact is, the greater its effect when it is made part of a misleading campaign.
©Adaptive Cultures
Decontextualizing and recontextualizing. When a fact is presented without the necessary context, or in a misleading context, resulting in an intended misunderstanding of its significance.
You are told something is a fact (e.g., the findings from a study) without being told the methods used, measurement errors, limitations of observations, etc. Typically, without understanding these issues, the fact seems important and impressive. Once the fact is understood in the context of possible measurement errors and inherent limitations of methodology, it appears tentative and potentially questionable.
Cherry-picking and limiting focus. When several related facts (or even a whole data set) are presented alone, as if they provide a comprehensive view of the relevant subject – when in reality there are other relevant facts that are not mentioned.
You are told that a great number of studies show the same thing, while never hearing about the studies done on relevant alternative hypotheses. Typically, without knowing more, it appears there is overwhelming consensus, while after knowing more, it appears there is reasonable dissensus and open-endedness.
Reinterpreting and pre-framing meaning. When a set of verified facts is presented as already determined to be part of an emotional frame. The fact is not presented as being interpretable according to many other possible valid frames.
You are told, or it is strongly implied, that the results of certain studies should be understood as “hopeful,” “groundbreaking,” “frightening,” or “damning,” etc., without being presented any alternative frame or perspective. Typically, value-laden and emotionally manipulative language creates the impression that it is inevitable to draw only certain conclusions, while the presentation of multiple perspectives and frames results in more diverse kinds of meaning-making.
A misleading campaign of facts runs according to some combination of the three primary ways outlined in the box above. Information campaigns that are factually truthful but nevertheless misleading are the stuff of classic propaganda, as we have documented in our recent series on the problem of modern propaganda.[11] For decades there has been cumulative innovation in the industry of public relations.[12] Techniques for misleading with facts have been continually and scientifically advanced.
Facts become weapons for use in politically charged discourses in which winning is more important than accurately representing larger and more complex truths.
Today, the strategic misuse of facts is becoming a common practice employed by everyday citizens on social media. Many people post to their social media feeds only those facts they endorse, which support their existing beliefs and ideologies. Verified facts are collected as ammunition for culture war, rather than for the sake of gaining a comprehensive understanding. Microtargeting then caters to these preferences, ensuring that there is a steady supply of cherry-picked facts on offer. The resultant filter bubbles and algorithmic radicalization have been discussed in our related paper on 21st-century information warfare.[13]
The algorithmic radicalization prevalent on social media does not require “fake news.” Because facts can be used to mislead, extreme polarization and ideological identity capture can occur when individuals engage with information that is factual. This is possible when facts are taken out of context, cherry-picked, and emotionally loaded. Facts become weapons for use in politically charged discourses in which winning is more important than accurately representing larger and more complex truths. This debases the usefulness of “facts” and fact-stating discourses, which is to debase a necessary component of adequate public sensemaking.[14]
But what would happen if we decided to slow down and think about the facts together? What if we really wanted to understand what was going on in a way that accounted for all the facts and their various frames and interpretations?
The rivalrous “checking” of facts must give way to a more collaborative mutual understanding of facts. With a focus on education, this approach requires that individuals seek earnestly to evaluate the complexity of factual claims. Working together, individuals engage in a collaborative process to understand the implied significance and meaning of the facts in question, including all the associated complexity and nuance. There are four essential ways of understanding facts (see Box 2). Understanding facts requires a process that transcends but includes the familiar process of fact-checking, adding considerations of context, representativeness, and framing.
Interpreting a fact involves values and judgments not determined by the fact itself.
Checking and Verification. If a factual claim is made, determine the methods used to justify it. Upon understanding the fact’s origin, find the clearest ways of stating what has been found to be true. When reading about new scientific findings made public by the media, one should ask questions like:
Has a reliable source been cited to support the fact?
Has the fact been corroborated by multiple independent sources?
Is it being stated in a way that is as clear and as close to empirical as possible?
Caveats and Context. After checking the fact, consider the limits of the observations and methods involved, comparing them to other possible observations and methods. Qualifying the fact as “contextually true” gives a deeper understanding of its validity, showing more clearly just what is known and what is not. When engaging with the actual studies and evidence referenced in media stories, one should ask questions like:
What is important to know about the contexts in which the fact has been validated?
What limitations do the relevant methods, observations, and claims have?
How much will the fact hold true beyond the context it was validated in?
Representation and Comprehensiveness. Once the fact is placed in context, expand that context to include the broader field of issues. Seek to gather as comprehensive a network of information as possible, including all relevant knowledge with a direct bearing on the significance of the fact. Understanding a fact requires having a representative view of the full state of knowledge on relevant issues. After reviewing the research studies cited by the media, and thereby validating and contextualizing the fact in question, you can further ask:
What additional facts must be considered to fully represent the meaning of the fact in question?
What is the full inventory of adjacent and related knowledge with regards to the issue?
Perspectives and Framing. Finally, after gaining a representative picture of the fact in context, reflect on its meaning by seeking to view it from several relevant frameworks and worldviews. Taking up multiple perspectives and engaging with the emotional dynamics of understanding the fact allows for a considered approach to communication and education. Beyond any fact are the values that frame it; therefore, one must ask:
In what ways can the fact be framed emotionally and taken personally by different types of people?
What are the implications of the fact, according to various relevant political ideologies?
What are the various reasonable perspectives on the meaning of the fact?
©Adaptive Cultures
Beyond simply verifying a fact, it must be placed in context and positioned relative to all other closely relevant facts. This includes gathering facts about the methods used to generate the fact in question. Verifying one fact requires verifying many others, while working towards presenting as comprehensive a network of related truths as possible. The emotional impact of any given fact is always complex. Interpreting a fact involves values and judgments not determined by the fact itself. Verification of factuality is only the beginning of a larger process of meaning-making, which involves considerations that cannot be reduced to specific debates about the “facts.”
Our task is to create new processes for determining what counts as a shared, socially meaningful, mutually understood “truth.”
Misleading with facts can only be done when attention is not paid to all four ways involved in the comprehensive understanding of facts. Fact-checking as currently practiced typically only focuses on one of the four. Educational efforts aimed at improving public discourse must consider more than how to detect deceptions and lies. There is a great deal more to understanding a fact than knowing if it is true. And there is a great deal more to understanding complex realities than agreeing on a set of facts.
The point of this article is not that fact-checking is bad, but that it is necessary yet partial. As it stands, it is inadequate as a response to information war—but this does not mean it should be abandoned. The future of our civil society and public sphere depends upon drastically upgrading current approaches to dealing with “facts.” Our task is to create new processes for determining what counts as a shared, socially meaningful, mutually understood “truth.” Obviously, this requires more than making sure that every fact is checked.
It is possible to expand our approaches to dealing with facts in public discourse in ways that include more complexity, nuance, and perspective-taking. A start would be to have fact-checking sites and discussions informed by the models offered above, instead of being constrained to only “checking.” Until such steps are taken to improve public culture, it will remain as easy to mislead with facts as it is to manipulate through deception—perhaps even easier.
The stakes are high when it comes to the future of “facts.” As has been made clear: the mishandling of facts eventually breaks public sensemaking, and the breaking of public sensemaking eventually breaks society. The clock is ticking. As more and more “facts” pile up, our culture nevertheless gets farther and farther away from reality. Information warfare is now systematically and rapidly undermining the possibility of coherent public fact-stating discourses. This leads to a situation where political and public relations campaigns begin to operate more explicitly and self-consciously outside the truth. Of course, “facts” remain important—especially if they are officially verified—because they can be used as ammunition. But the larger, more complex truth is lost, accepted as a casualty of culture war.
This kind of cynical, post-truth culture is antithetical to democratic ways of life. But the solution is not to create centralized “Truth Committees.” These would serve as the official legitimators of censorship, becoming the ultimate authorities on shared social reality. Open societies are defined, in part, by the free flow of reliable facts through public culture. They are different thereby from societies that route information through narrow channels and give over individual judgment to the dictates of authorities. Responsibility for the integrity of public fact-stating discourses should be distributed throughout civil society. The movement around the advancing field of fact-checking should not seek to consolidate power, but to distribute it.
There is no technical “fix” or simple solution for improving the overall tenor and complexity of public communication about facts. There are, however, possibilities for digital technologies to enable educational initiatives of profound depth at massive scale. The same technologies that are now being used to mislead us with facts can be used to help us piece all the facts together and place them in the right context. The now crucial nexus of digital technologies, education, and politics can be reconfigured to allow for widespread learning and mutual understanding. Even though these facts are clear, there is, as always, the question of what we choose to do with them.
Agent provocateur translates to “inciting incident” in French. It is used to reference individuals who attempt to persuade another individual or group to partake in a crime or rash behavior or to implicate them in such acts. This is done to defame, delegitimize, or criminalize the target. For example, starting a conflict at a peaceful protest or attempting to implicate a political figure in a crime.
Ideological polarization is generated as a side-effect of content recommendation algorithms optimizing for user engagement and advertising revenues. These algorithms will upregulate content that reinforces existing views and filters out countervailing information because this has been proven to drive time on-site. The result is an increasingly polarized perspective founded on a biased information landscape.
To “cherry pick” when making an argument is to selectively present evidence that supports one’s position or desired outcome, while ignoring or omitting any contradicting evidence.
A general term for collective resources in which every participant of the collective has an equal interest. Prominent examples are air, nature, culture, and the quality of our shared sensemaking basis or information commons.
The cognitive bias of 1) exclusively seeking or recalling evidence in support of one's current beliefs or values, 2) interpreting ambiguous information in favor of one’s beliefs or values, and 3) ignoring any contrary information. This bias is especially strong when the issues in question are particularly important to one's identity.
In science and history, consilience is the principle that evidence from independent, unrelated sources can “converge” on strong conclusions. That is, when multiple sources of evidence are in agreement, the conclusion can be very strong even when none of the individual sources of evidence is significantly so on its own.
While “The Enlightenment” was a specific instantiation of cultural enlightenment in 18th-century Europe, cultural enlightenment is a more general process that has occurred multiple times in history, in many different cultures. When a culture goes through a period of increasing reflectivity on itself it is undergoing cultural enlightenment. This period of reflectivity brings about the awareness required for a culture to reimagine its institutions from a new perspective. Similarly, “The Renaissance” refers to a specific period in Europe while the process of a cultural renaissance has occurred elsewhere. A cultural renaissance is more general than (and may precede) an enlightenment, as it describes a period of renewed interest in a particular topic.
A deep fake is a digitally-altered (via AI) recording of a person for the purpose of political propaganda, sexual objectification, defamation, or parody. They are progressively becoming more indistinguishable from reality to an untrained eye.
Empiricism is a philosophical theory that states that knowledge is derived from sensory experiences and relies heavily on scientific evidence to arrive at a body of truth. English philosopher John Locke proposed that rather than being born with innate ideas or principles, man’s life begins as a “blank slate” and only through his senses is he able to develop his mind and understand the world.
An orientation towards a reality that is neither epistemic nihilism nor epistemic hubris. As opposed to an ethos of knowing, it is an ethos of learning, which The Consilience Project suggests is needed for grappling with the unique challenges of 21st-century sensemaking. This ethos implies curiosity and a motivation to pursue further learning, embracing facts and truth where these are possible to attain, but always remaining open to further learning—refusing to commit to absolutism or fundamentalism.
This form of nihilism is a diffuse and usually subconscious feeling that it is impossible to really know anything, because, for example, “the science is too complex” or “there is fake news everywhere.” Without a shared ability to make sense of the world as a means to inform our choices, we are left with only the game of power. Claims of “truth” are seen as unwarranted or intentional manipulations, as weaponized or not earnestly believed in.
Epistemology is the philosophical study of knowing and the nature of knowledge. It deals with questions such as “how does one know?” and “what is knowing, known, and knowledge?”. Epistemology is considered one of the four main branches of philosophy, along with ethics, logic, and metaphysics.
Derived from a Greek word meaning custom, habit, or character; The set of ideals or customs which lay the foundations around which a group of people coheres. This includes the set of values upon which a culture derives its ethical principles.
A category of risk that denotes the complete and total elimination of humanity or the planet. Example: Earth killer asteroid impacts
Discourse oriented towards mutual understanding and coordinated action, with the result of increasing the faith that participants have in the value of communicating. The goal of good faith communication is not to reach a consensus, but to make it possible for all parties to change positions, learn, and continue productive, ongoing interaction.
Processes that occupy vast expanses of both time and space, defying the more traditional sense of an "object" as a thing that can be singled out. The concept, introduced by Timothy Morton, invites us to conceive of processes that are difficult to measure, always around us, globally distributed and only observed in pieces. Examples include climate change, ocean pollution, the Internet, and global nuclear armaments and related risks.
Information warfare is a primary aspect of fourth- and fifth-generation warfare. It can be thought of as war with bits and memes instead of guns and bombs. Examples of information warfare include psychological operations like disinformation, propaganda, or manufactured media, or non-kinetic interference in an enemy's communication capacity or quality.
Refers to the foundational process of education which underlies and enables societal and cultural cohesion across generations by passing down values, capacities, knowledge, and personality types.
False or misleading information, irrespective of the intent to mislead. Within the category of misinformation, disinformation is a term used to refer to misinformation with intent. In news media, the public generally expects a higher standard for journalistic integrity and editorial safeguards against misinformation; in this context, misinformation is often referred to as “fake news”.
A prevailing school of economic thought that emphasizes the government's role in controlling the supply of money circulating in an economy as the primary determinant of economic growth. This involves central banks using various methods of increasing or decreasing the money supply of their currency (e.g., altering interest rates).
A form of rivalry between nation-states or conflicting groups, by which tactical aims are realized through means other than direct physical violence. Examples include election meddling, blackmailing politicians, or information warfare.
Open societies promote the free exchange of information and public discourse, as well as democratic governance based on the participation of the people in shared choices about their social futures. Unlike the tight control over communications and suppression of dissenting views that characterize closed societies, open societies promote transparent governance and embrace good-faith public scrutiny.
The theory and practice of teaching and learning, and how this process influences, and is influenced by, the social, political, and psychological development of learners.
The ability of an individual or institutional entity to deny knowing about unethical or illegal activities because there is no evidence to the contrary or no such information has been provided.
First coined by philosopher Jürgen Habermas, the term refers to the collective common spaces where people come together to publicly articulate matters of mutual interest for members of society. By extension, the related theory suggests that impartial, representative governance relies on the capacity of the public sphere to facilitate healthy debate.
The word itself is French for rebirth, and this meaning is maintained across its many purposes. The term is commonly used with reference to the European Renaissance, a period of European cultural, artistic, political, and economic renewal following the middle ages. The term can refer to other periods of great social change, such as the Bengal Renaissance (beginning in late 18th century India).
A term proposed by sociologists to characterize emergent properties of social systems after the Second World War. Risk societies are increasingly preoccupied with securing the future against widespread and unpredictable risks. Grappling with these risks differentiate risk societies from modern societies, given these risks are the byproduct of modernity’s scientific, industrial, and economic advances. This preoccupation with risk is stimulating a feedback loop and a series of changes in political, cultural, and technological aspects of society.
Sensationalism is a tactic often used in mass media and journalism in which news stories are explicitly chosen and worded to excite the greatest number of readers or viewers, typically at the expense of accuracy. This may be achieved by exaggeration, omission of facts and information, and/or deliberate obstruction of the truth to spark controversy.
A theory stating that individuals are willing to sacrifice some of their freedom and agree to state authority under certain legal rules, in exchange for the protection of their remaining rights, provided the rest of society adheres to the same rules of engagement. This model of political philosophy originated during the Age of Enlightenment from theorists including, but not limited to John Locke, Thomas Hobbes, and Jean-Jacques Rousseau. It was revived in the 20th century by John Rawls and is used as the basis for modern democratic theory.
Autopoiesis from the Greek αὐτo- (auto-) 'self', and ποίησις (poiesis) 'creation, production'—is a term coined in biology that refers to a system’s capability for reproducing and maintaining itself by metabolizing energy to create its own parts, and eventually new emergent components. All living systems are autopoietic. Societal Autopoiesis is an extension of the biological term, making reference to the process by which a society maintains its capacity to perpetuate and adapt while experiencing relative continuity of shared identity.
Used as part of propaganda or advertising campaigns, these are brief, highly-reductive, and definitive-sounding phrases that stop further questioning of ideas. Often used in contexts in which social approval requires unreflective use of the cliché, which can result in confusion at the individual and collective level. Examples include all advertising jingles and catchphrases, and certain political slogans.
A proposition or a state of affairs is impossible to be verified, or proven to be true. A further distinction is that a state of affairs can be unverifiable at this time, for example, due to constraints in our technical capacity, or a state of affairs can be unverifiable in principle, which means that there is no possible way to verify the claim.
Discussion
Thank you for being part of the Consilience Project Community.
0 Comments