華文

Stress Tests for Civic AI

In brief

Why this page exists

Civic AI should not be protected by its own vocabulary. If a framework cannot say where it fails, it becomes the thing it warns against: care as branding, participation as theatre, governance as a checklist.

This page is the public ledger for the strongest objections. It keeps the claim smaller and more useful: Civic AI is a discipline for making situated AI deployments answerable to the people they affect. It is not a universal theory of intelligence, not a replacement for frontier safety, and not proof that every deliberative process is legitimate.

1. Synthetic publics can drown real publics

The hardest failure mode is not apathy. It is fake participation at scale.

A deliberation system depends on knowing when a public voice is human, affected, and accountable enough to count. AI weakens that assumption. In California, South Coast AQMD's gas-appliance rulemaking was flooded by more than 20,000 AI-assisted opposition comments, sent through an advocacy platform. Reporting by the Los Angeles Times and the American Planning Association describes the episode as a warning: AI-generated comments can dilute real community voice and influence public decisions. California's SB 1159 responds to exactly this risk by clarifying that AI systems are not members of the public for open-government participation.

That is not a footnote to Civic AI. It is the core adversarial case.

Design response: public participation needs provenance, rate limits, accountable standing, selective-disclosure identity, and audit trails that distinguish expression from amplification. The right to speak remains human. The right to be algorithmically multiplied does not.

Sources: Los Angeles Times on South Coast AQMD, American Planning Association on AI public engagement attacks, California SB 1159 analysis.

2. The evidence comes from favourable soil

The best-known cases, Taiwan's Alignment Assembly, Engaged California, Team Mirai, Monlam AI, ROOST, and the UK long-term care co-production, are real. They also come from unusually prepared soil: civic hackers, responsive public institutions, strong associational life, or expert communities with the capacity to govern tools.

That matters. The mechanism may transfer; the institutional conditions do not automatically transfer.

Design response: never sell Taiwan, California, or Japan as plug-and-play templates. Treat each as a proof of mechanism. In a new room, start again: who is missing, who has standing, which local institutions deserve trust, which do not, and who can stop the system when the first answer is wrong?

3. The 6-Pack is an adapter, not a theory of everything

The 6-Pack draws openly from Joan Tronto's care ethics, deliberative democracy, Plurality, and deployment accountability. Its value is not that it discovers six new virtues. Its value is that it translates care into institutional machinery:

The test is therefore practical. If a pack name does not change who can object, who must answer, what gets logged, or what happens after failure, it is decoration.

4. Kami retirement must be engineered, not presumed

The strongest language about a Kami says it recedes when its work is done. That can become too easy. Many useful systems do not disappear. They become standing infrastructure: vTaiwan, Polis, Monlam AI, ROOST, and Engaged California all point more toward continuity than vanishing.

So retirement cannot be treated as the natural behaviour of a good system. It has to be a contract.

Design response: every Kami needs a named authority to shut it down, a sunset or review date, non-expansion rules, resource and retention caps, handover tests, and a public record of succession. Sometimes the right ending is not deletion. It is transfer of duty to a successor while private histories stay with the people who created them.

5. Frontier alignment remains a separate load-bearing problem

Civic AI addresses deployed systems that act inside real rooms: care homes, classrooms, parishes, deliberation platforms, councils, unions, clinics, and families. It asks who is owed an answer when such a system acts, and who is authorised to give that answer.

That is not Bostrom's alignment problem. It is not the problem of controlling a powerful general optimiser before release. Work on that first question remains necessary, and Civic AI should not borrow its urgency while claiming exemption from its hardest parts.

Design response: state the boundary plainly. Civic AI complements frontier alignment; it does not replace it. It builds civic terrain that makes deployed systems more answerable and makes monoculture less attractive. It does not solve the control problem from inside a frontier model.

6. "We, the people, are the superintelligence" is a metaphor under constraint

The phrase is useful only if it is kept modest. It should not mean that a mini-public is equivalent to machine superintelligence. It should mean that, under the right institutional conditions, people with tools can produce accountable collective intelligence that no central optimiser can legitimately replace.

That is already a strong claim. It does not need inflation.

Operational checklist

Before calling a deployment Civic AI, ask these six questions:

  1. Human standing: can affected people prove standing without surrendering unnecessary identity?
  2. Adversarial load: what stops synthetic publics, bots, or paid campaigns from flooding the channel?
  3. Decision force: where does the output bind actual decisions, and where is it only advice?
  4. Repair path: who can appeal, who pays for harm, and where is the repair logged?
  5. Exit capacity: can the community migrate, fork, pause, or shut down without losing service continuity?
  6. Boundary honesty: what problem is out of scope, and who is carrying that problem instead?

If the answer to any of these is vague, the deployment is not ready. The point is not to defend the 6-Pack. The point is to make the room answerable.

← FAQ Home →