Callin Discussion Today at 3:00 EST: On the "Top 50 Organizations to Know" in the Censorship-Industrial Complex

On the release of Racket's "Top 50" census of "anti-disinformation" organizations, why the general public will have to keep fighting to keep track of the exploding censorship bureaucracy

Visual representations of “misinformation,” 2016-2023

First of all, after months of work by a team of ten, Racket has released a huge taxonomic survey of the “Top 50 Organizations to Know in the Censorship-Industrial Complex.” This is the result of both traditional research and journalism, and a painstaking effort at following leads in the #TwitterFiles. If you’re interested in talking with me and with some of the list’s authors, Racket is hosting a Callin discussion today at 3:00 p.m. EST. To attend, you just have to download the app, and follow the link here.

About this project, a few additional notes:

A million years ago, when dinosaurs roamed the earth and I briefly worked for First Look Media, I had a strange conversation with the firm’s owner, eBay founder and billionaire Pierre Omidyar. Pierre at the time — this was 2014 — was being hyped in progressive-leaning outlets like New York magazine as a progressive savior, the leader of an “insurgency,” presumably against a corrupt, militaristic political establishment. He’d pledged to spend a whopping $250 million on new press ventures and through such gifts would re-orient the information landscape, putting people like Glenn Greenwald and Laura Poitras in charge of a true alternative to outlets like CNN, FOX, and MSNBC, which were still choking on the WMD fiasco that sent America to war in error.  

Omidyar’s Greenwald-Poitras project, which would be called The Intercept, was supposed to pound the powers that be with hardcore investigative reporting, using the leaked documents of Edward Snowden as leads. My sideshow contribution was to be more modest. I was hired to run a satire site, based on the old Spy magazine, whose ostensible purpose would be to train audiences to take a more sarcastic/skeptical view of media, making episodes like the WMD mess less likely.

Pierre and his eBay staffers periodically arrived at the Intercept’s editorial offices armed with bundles of Moneyball-type questions about the news business, which were received much as one might expect a baseball operations office to respond to new theories of lineup construction, sent in by the insurance or pharmaceutical company that just bought the Mets or Red Sox.

For instance, the eBay crew then was caught up in gadget concepts like UpWorthy, which used quantitative analysis to generate click-successful headlines. We were asked a million questions about site functionality, infographics strategies, etc. In hindsight, Alex Pareene and I should have just started writing and publishing stuff the first day.

Omidyar however saw the tech industry as a potential savior of modern society. He was focused on developing technical adds to the media concept. Having made a vast fortune with a digital auctioneering program that allowed people on opposite sides of the earth to sell bits and bobs to one another, he imagined a similar electronic widget could be discovered that would solve the ills of the press. One idea I remember emerging from the eBay team involved placing socially beneficial news messages in video game landscapes.

When he asked about my plan, I offered that in a future media landscape in which many editorial decisions would be made by algorithms, audiences would crave the unmistakably human. Therefore a humor outlet should thrive, because humor can’t be easily synthesized.

“But how does it work?” he asked. 

“Well,” I shrugged, “you just have to hire funny writers.”

Omidyar frowned. If a media concept could not be done at scale, he found it boring and probably unworkable. He soon also became disenchanted with Greenwald's concept of reporting — essentially, just follow leads wherever in old-school human fashion — and before we knew it, his $250 million pledge was gone, replaced by a threadbare Intercept that was financially starved and ignored by management. 

Ten years later, on the heels of a different paradigm-shifting document dump called the Twitter Files, Omidyar is one of the key figures populating what Michael Shellenberger cannily termed the Censorship-Industrial Complex. What Pierre did not spend on “insurgent” news alternatives back in 2014, he now very much does spend on “anti-disinformation” groups like Meedan and the Oxford Internet Institute. Omidyar’s brief exposure to human reporting and commentary seems to have convinced him to empty his pockets for an opposite venture, helping build a system to clamp down on dangerous human speech before it can be algorithmically reviewed. 

Left, then. Right, now.

In one study funded by Omidyar’s Oxford group, researchers defined “junk” news in part according to style:

These outlets use emotionally driven language with emotive expressions, hyperbole, ad hominem attacks, misleading headlines, excessive capitalization, unsafe generalizations and logical fallacies, moving images, and lots of pictures and mobilizing memes.

If you’re thinking those descriptions sound like techniques people have been using since the advent of the commercial press to generate attention — William Randolph Hearst helped build mass media with all-caps banner headlines — you’re not alone. Numerous outlets in the “Censorship-Industrial Complex” nonetheless identify bad media based on “emotive” content. One example of such targeting involves the US-funded, UK-based Global Disinformation Index, which (as reported by the Washington Examiner) separates news outlets into “quality” and “junk” categories and down-ranks the latter, so that platforms using their services will spend less on advertising with news sources deemed to be of poor quality by digital judges. 

The Censorship-Industrial Complex represents the realization of the dreams of a hundred Omidyars. These are the wealthy and influential citizens of Western countries — primarily, from Five Eyes nations like the U.S., U.K., and Canada — who’ve gotten together and poured both private and public money into a vast new enterprise designed to create machines to watch, analyze, and judge all Internet traffic. 

The text in the accompanying “Top 50” list, which represents a fraction of the work done by a large team over the course of the last months, has its roots in a dead end I hit in February 2023, while trying to work through #TwitterFiles documents. At one point I began seeing hints of a relationship between a U.S. government agency called the Global Engagement Center, and a host of ostensibly private “anti-disinformation” groups run out of think-tanks with names like the Alliance for Securing Democracy, the Atlantic Council, the Foundation for the Defense of Democracy, and others.

A pattern was emerging in the documents, under which Twitter would be pressured to remove or deamplify content from multiple institutional directions at once: from government agencies, NGOs, and commercial news media, seemingly acting in concert. Twitter’s senior executives might grouse at individual requests, but if these institutions achieved a unified critical mass of displeasure over this or that issue — Russian interference, vaccine disinformation, domestic extremism — the company would accede and accounts would usually be actioned. 

By the fall of 2020, Twitter itself was an enthusiastic partner of this new cartel, even serving as a social guide for new entrants (see the “Top 50” entry for Wikimedia, which Twitter formally introduced to an FBI contact for “disinformation” matters in October of that year). In the summer of 2020, Twitter joined up with Stanford’s Election Integrity Partnership, which was partnered with the Homeland Security sub-agency CISA to police election “misinformation.”

In the year that followed, it teamed up with six other major tech platforms in joining Stanford’s Virality Project, which helped guide vaccine messaging across the social media landscape. Vaccine messaging was guided on these platforms with the aid of multiple new “anti-disinfo” outlets, as well as new content-targeting wings of older institutions like the Anti-Defamation League. Later the company received even more intense and explicit messaging through both government and private agencies about the Ukraine conflict.

It soon became clear that Twitter’s internal correspondence was allowing us to see the emergence of a booming new intelligence/defense-based contracting community, one whose size, scope, and interrelatedness had not been studied at all, at least not with a critical eye. As we’d soon learn, some of the “anti-disinformation” groups had in fact attempted to conduct censuses of its own population, but with unclear success, and never with the idea of warning the public about its excesses.

It was at that moment, when it became obvious that Twitter was being besieged with requests for content moderation from more organizations than any one person could count, that we started this project. 

Racket hired a team to attempt a survey of organizations that were doing algorithmic language-policing. The group you see in the piece’s byline, which includes Andrew Lowenthal, Susan Schmidt, Tom Hyatt, Matt Farwell, Geneve Campbell, and @TechnoFog, among others — some reporters, some former NGO administrators, some independent researchers — was assembled to try to get a handle on just how many of these organizations were out there. We thought it would be manageable. Maybe, on the outside, we were looking at thirty, forty, at most fifty major groups. 

Quickly, it turned out that the “Censorship-Industrial Complex” is far bigger than any of us thought. What our team began calling “The List” has by now surged past 300 organizational names. Moreover, we know the real number is bigger because we’re now familiar with all the attemts of other agencies to perform such counts. In one “DisinfoCloud” digest, paid for by the U.S. State Department and sent fortuitously for us to find in Twitter’s in-box, there’s a report that “The Partnership for Countering Influence Operations (PCIO) created a catalog of 460 counter-influence operations initiatives.” The PCIO study reportedly built on the earlier work of groups like the Rand Corporation and the Credibility Coalition, the latter of which had already counted 250 names. 

From DisinfoCloud, January 15, 2021

Many of these groups are state-funded. Some are bankrolled by private outfits like Omidyar’s, or the Newmark Foundation of Craigslist founder Craig Newmark, or the Knight Foundation. Even a cursory review revealed that a true map of this “ecosystem” will take a long time to complete. Our guess is neither our efforts nor those of the PCIO or the Credibility Coalition have dented the true number. As proposals multiply for legislation like America’s RESTRICT Act and Europe’s Digital Services Act (which would create new obligations for platforms like Twitter to liaise with “trusted flaggers”), demand for new “anti-disinformation” organizations keeps rising, causing these groups to proliferate at a rate faster than its own counting mechanisms can track. 

Very little has been written about these organizations. In fact, there’s so little public information about many of these groups that the Twitter Files in many cases are the best sources we have on them. “The List” therefore contains a lot of new Twitter Files material, a lot of it useful mainly in explaining the relationships of groups to each other, or the government (“First Draft,” now called the Information Futures Lab at Brown University, is #1 on the list in part because it really is that influential, and in part because it was Twitter’s first choice when it decided to assemble a private Signal group of key “anti-disinformation” groups).

Shellenberger’s Public will likely pick up the baton, not only identifying and researching remaining groups, but finding ways to track the institutional and interpersonal connections between them, which we determined some time ago was a serious issue. The bonding agents for this political system are often buried in professional relationships between key figures, who as we saw graphically in the #TwitterFiles emails move freely from one institution to another, replicating their old work even if previous work has failed or been exposed in a negative way to the public. 

There’s perhaps no better example of this than an accidental disclosure in the Twitter Files: an email between Twitter executives detailing a call from the New America Foundation (which could be, but is not, on “The List”), explaining that former FBI agent Clint Watts, who’d worked on the infamous Hamilton 68 dashboard allegedly tracking Russian influence, was consulting on a new project for the group, purporting to track domestic extremism. 

Absent this email, in which Nick Pickles explains that in the call to New America he “highlighted the concerns… around credibility” that came up with Hamilton 68’s Russian-influence-spotting project, the public would have no idea that Hamilton had a domestic-facing relative. A quick glance at the respective products, however, shows that the two organizations were using basically identical techniques (New America has declined to comment publicly on this issue):

The wider “Report on the Censorship-Industrial Complex” has a number of features still on the way, including a historical timeline, a look back at the CIC’s role in helping create and market the Trump-Russia scandal, and a horrifying story of one person’s experience of what it’s like to deal with the “Complex” once it sets its hive-mind to erasing/neutralizing a single individual. “The List,” however, is this team’s largest collective effort, a starter kit for a criminally under-covered story. I couldn’t be more grateful to the authors listed above, and to the visualization work from the folks at mrmooremedia.com, for their hard work in helping the public begin to see the outlines of this camouflaged, extremely dangerous system. 

It’s not impossible that enhanced awareness of the problem will arouse either election-affecting outrage below or cause a change of thinking up above. Just this week Lawfare — perhaps the single most dependable, lockstep cheerleader for the intelligence community in the American media landscape — published an article suggesting a re-think of “anti-disinformation.” It’s incredible this realization took so long to reach these quarters, but still, wow:

It’s time to look at the problem differently. Those attempting to address the issue should move away from attempts to regulate disinformation and toward the ecology of the information environment more generally…

Lies and influence operations are part of the authoritarian playbook. The trouble is that control of the information space in the name of controlling disinformation is also part of the authoritarian playbook…

Maybe that light bulb will go off above enough other influential heads to stop this madness. Until then, it’s on us to educate ourselves and find ways to beat it back. For readers, “The List” may appear a daunting lift, but it’s worth it. The names in the piece may as well double as an introduction to the roster of a new self-styled global oligarchy.

Sooner or later, you’ll need to know these groups, as we can more or less guarantee: they’re already keeping tabs on you.