civic.ai仁工智慧
華文

Frequently Asked Questions

Start with an everyday question

Must I build or buy an AI? No. Start with a system your school, clinic or council already uses, and ask who owns the decision, who must reply and how someone can appeal. Choosing no AI is legitimate.¶ 1

Does a polite chatbot show care? People and institutions owe each other care. A friendly tone does not show who heard a concern, who must answer or who will repair harm. Relational health covers how people relate to one another, to providers and to AI, and how systems and operators relate to each other. Those relationships need transparency, agency, accountability and reciprocity. Reassuring words alone do not supply them.¶ 2

What do we discuss at a first group meeting? Begin with the four keeping questions: Who keeps it, what harm can a breach cause, who may override and when does it end? Make the meeting accessible and name proxies with standing to speak and override for people who cannot attend. If the group cannot answer, do not start deploying.¶ 3

How does this site map to the book? Book Chapters 3-8 correspond to site Packs 1-6; the FAQ and glossary explain the foundations. Sources and cases distinguish documented work from proposals, while the flood-bot is a fictional composite for teaching. Start with the everyday actions; the measures are proposals that have not been validated.¶ 4

How the hard questions fit together

The numbered questions are the strongest objections we have heard. Each one tests a single part of the framework, so together they fall into five groups. The numbers stay fixed so that links keep working; the groups give the order of the argument.¶ 5

If your objection fits none of these, tell us.¶ 11

Q1. The AI market is locked in an arms race driven by commercial profit and geopolitical dominance. An AI working for a tax-software monopoly can lobby to keep tax filing difficult — and far worse is easy to imagine. If this problem continues, isn’t the vision of cooperative Kamis hopelessly naive?

Civic AI cannot survive by asking monopolies to be nicer. The trap has a name, Moloch: everyone races to the bottom, because whoever defects first wins and whoever holds back loses. Moralising will not change that. Changing the payoffs can, by making cooperation pay more than extraction. The five levers below mix documented examples with policy proposals, and the evidence varies, as the sources and cases explain.¶ 12

  1. Interoperability and portability. Require platforms to speak common protocols, the way any email service can write to any other, so people can leave without losing their relationships. The Utah Digital Choice Act already requires a portable copy of a user’s personal data, including the social graph (§13-81-201, in force 1 July 2026). Continuous real-time forwarding of new activity is the later H.B. 408 duty, from 1 July 2027. When the moat of a captive audience drains away, platforms have to compete more on quality of care and less on the strength of the cage.¶ 13
  2. Civic procurement. Governments are enormous customers, and what they insist on, vendors build. Require that any AI bought for public use be auditable, interoperable, and informed by citizen input through an institutional response path, as Taiwan’s Alignment Assembly did for anti-scam policy without itself governing the systems. Then building Kami-like systems becomes better business. Steward-ownership structures bind companies legally to their mission rather than to sale or extraction. With board-level safety duties, they can turn civic care into a fiduciary obligation, a duty the law can enforce.¶ 14
  3. Public options. Offer simple baseline services that do the job without harvesting attention or data, backed by publicly funded research compute. Private vendors then have to beat the public option on care, because lock-in no longer works. Taiwan’s tax-filing system, which replaced a user-hostile service with one redesigned in multi-stakeholder workshops with the people who use it, is a working prototype.¶ 15
  4. Provenance for paid reach. Provenance means a verifiable answer to one question: who is paying for this message? Require it, with disclosure that stays attached, for ads and mass amplification in political and financial domains. Taiwan’s Fraud Crime Hazard Prevention Act Art. 30 requires online advertisement platforms to verify the identity of persons commissioning and funding ads through digital signature, rapid authentication, or equivalent-safety methods. It does not require the featured celebrity to sign. Ordinary speech is protected through selective-disclosure identity (Pack 5): you prove you are a real person without revealing who.¶ 16
  5. Federated open supply. Support open-weight models (models whose trained parameters are published, so anyone can download, inspect, and run them) and trust-and-safety tools that independent organisations can self-host, so nobody depends on one central moderator. Examples: ROOST: Osprey for incident response, Coop for review and moderation, and the ROOST Model Community for open-weight safety models and the policy packs that go with them. If basic intelligence becomes a public good, the race can shift from “who owns the biggest brain” to “who applies intelligence most attentively in a local context”, a race that rewards care.¶ 17

None of these levers requires goodwill from incumbents. Each aims to restructure incentives so that civic behaviour becomes the path of least commercial resistance. They do require legislators, procurers and funders willing to use them. That political will is the scarce input, and no framework can supply it.¶ 18

Q2. Care ethics was developed for interpersonal relationships — a nurse and a patient, a parent and a child. Scaling it to AI systems and global governance seems like a category error — stretching a concept to somewhere it simply does not belong. Why isn’t it?

Care ethics’ own practitioners have raised this objection. Care, they say, is too intimate, too parochial (bound to its own small patch) and too prone to self-effacement (the carer’s habit of disappearing behind the cared-for) to ground a theory of institutions, let alone machines. We think each weakness, translated with care, becomes a design constraint. The premise is also only half right.¶ 19

Care ethics did begin with interpersonal moral judgement, in the work of Carol Gilligan and Nel Noddings. But Berenice Fisher and Joan Tronto defined care in 1990 as “a species activity that includes everything that we do to maintain, continue, and repair our ‘world’ so that we can live in it as well as possible”. That definition already takes in institutions and infrastructure, and Tronto carried care into democratic politics in Moral Boundaries (1993) and Caring Democracy (2013). The structure of the practice scales even where the feeling does not: noticing a need, taking responsibility, acting competently and checking with the people cared for.¶ 20

Consider what happens when you translate care’s supposed weaknesses into design constraints for AI:¶ 21

The translation is not always clean. Boundedness can become insularity, corrigibility can become passivity, and subsidiarity can become fragmentation. These are engineering tensions that need continuous calibration, which is why each Pack includes failure modes and named fixes.¶ 25

The 6-Pack does not ask AI to feel care. It takes the working structure of a caring relationship (attentiveness, answerability, competence, responsiveness, solidarity, symbiosis) and turns each into something checkable: design rules a machine can be tested against, engagement contracts a community can contest if authority and remedies are real, outcomes anyone can measure.¶ 26

Those tests carry substantive commitments. In civic settings, proposals that bridge groups count for more than proposals that merely win a majority, and a floor of rights stays off the bargaining table. These are normative choices, stated openly; Q13 explains why they are thin enough to share across political traditions. Care’s interpersonal origin is where the rigour comes from, and there is nothing to apologise for.¶ 27

Q3. Ambitious goals we point AI at (“cure cancer,” “solve climate change”) are almost always consequentialist — judged by outcomes alone. Optimising for these outcomes at superhuman speed inevitably leads to unforeseen risks. Does care ethics mean giving up on these grand, civilisation-scale goals?

Not at all. But it does radically reframe how we achieve them.¶ 28

Start with a distinction. Wanting to cure cancer is not consequentialism; every ethical tradition wants the sick to recover. Consequentialism is the further claim that outcomes alone decide what is right. Care ethics shares the goal and rejects the further claim: who decides, who bears the risk and whether the people affected can object also matter.¶ 29

The danger of pointing a superintelligence at a single goal like “cure cancer” is that it may treat a complex, relational, ecological reality as a constraint-satisfaction problem: a puzzle in which any variable may be forced so long as the target number is hit. A system maximising a single variable at superhuman speed tends to perfect the proxy (the stand-in number) while damaging the human context it stood for.¶ 30

Goodhart’s law (once a measure becomes a target, it stops being a good measure) is an empirical regularity with a moral consequence. When a target predictably invites gaming, the harm is foreseeable, and the people who chose the target share responsibility for it.¶ 31

Care ethics welcomes progress and refuses only one thing: shrinking a living reality down to one number. In a Civic AI future, nobody unleashes one unbounded Singleton, a single system in charge of everything, to “solve” a problem from the top down.¶ 32

We cultivate an ecology of specialised Kamis. A Kami is not a new kind of AI. It is an existing AI model, given instructions (a prompt) and a few note files, and kept by a group of people who answer for it. One Kami simulates protein folding, another helps local clinics share knowledge, and another assists patients in navigating their care. None has an unbounded mandate to “optimise the world.” A prompt is only a request to the model, so any limit that matters (what the Kami may read, send, spend or change) is enforced by software permissions and by the people who keep it, never by asking the model to behave. Capability and mandate are different things: a very capable model can serve a narrow mandate, and Q19 asks what happens when narrow parts are combined. Progress emerges horizontally, through the symbiosis of human ingenuity and narrowly scoped machine intelligence, each supplying what the other lacks.¶ 32a

Q4. Democracy serves known functions: error correction, peaceful power transitions, checks on concentrated authority, legitimacy for collective action, information aggregation, preference expression. A sufficiently capable AI could plausibly perform every one of these faster and more reliably than any deliberative process. Why insist on democratic governance?

If democracy is justified only by its outputs, any system that produces better outputs can replace democracy, including a benevolent AI autocracy that aggregates preferences efficiently and corrects errors faster than elections ever could. The same argument would hand power to experts. Concentrating capability in systems built to optimise makes this a live option. It needs an answer in principle, because a bet on who performs better settles nothing.¶ 34

The 6-Pack gives two reasons that do not depend on performance. The first is standing, rooted in care ethics: to perceive a need is to perceive an obligation, and to act in common is to ask what a community is trying to protect, repair, sustain or grow. People have standing, a rightful seat at the table, because the decisions affect their lives and their shared world. Their input often improves decision quality too, but standing does not depend on it. Being governed well by others is not the same as governing yourself.¶ 35

The second is verification. A system cannot be the final check on its own errors. Someone outside it must be able to find the error, say so and make it count: the corrective loop of Q16. The affected are not the only judges, but no one can certify that care landed without asking them. However competent, a system that excludes the affected has failed the basic test of alignment. Expertise earns a seat and a duty to explain. The authority to decide for the people affected stays with them, and the same holds for AI.¶ 36

Aggregation has a further limit. Many public preferences do not exist in finished form before people deliberate; they form as people hear one another and reconsider. An AI can aggregate the preferences that already exist. It cannot skip the process in which a public works out what it wants.¶ 37

Taiwan’s trajectory makes this concrete. Digital democracy emerged because people demanded standing, as in the 2014 Sunflower Movement occupation of Taiwan’s legislature (Q8 tells the trust story). The care relationship came first, and the capability followed.¶ 38

The functional question still deserves a functional answer. Error correction: bridging tools and community-authored evaluations (Packs 1, 4) surface failures that centralised monitoring misses, because the people who feel the failure write the test. Peaceful transfer: a Kami that accepts shutdown, and communities that can fork their tools, meaning copy them and carry on independently (Packs 6, 5), make replacement routine. Checks on power: no Kami governs beyond its mandate (Pack 6).¶ 39

Information aggregation becomes broad listening (Pack 1): AI-assisted sensemaking across many participants and languages. Preference expression becomes engagement contracts (Pack 2): standing processes for bargaining over what people need, where one-off elections flatten preferences into binary choices. Legitimacy is the exception: it is not popularity, and no metric constitutes it. Two proposed measures can give public evidence about it: cross-group co-endorsement and trust after loss (Q13, Measures).¶ 40

A well-designed technical system could replicate some of these outputs in isolation. What such a system cannot replicate, however, is the standing of the people affected, and a system that optimises for outcomes while removing standing is exactly the misalignment the 6-Pack exists to prevent.¶ 41

Q5. Deliberation is slow. AI moves fast. By the time an Alignment Assembly reaches consensus, the technology has moved on three generations. How do you handle the speed mismatch?

The objection assumes every decision needs the same depth of deliberation, and it does not. The framework runs in two lanes: a slow lane that sets boundaries (Pack 2) and a fast lane that operates within them (Pack 3):¶ 42

Slow lane: Setting boundaries. Alignment Assemblies, citizen deliberations, and engagement contracts (the written agreements between a community and an AI’s operators about what the system may do and who answers when it breaks) establish the guardrails: the rights that cannot be traded, the red lines, the severity classifications, the conditions that trigger a pause. These rights draw on the democratic commitments to justice, equality and freedom that Tronto says care must be consistent with. Rights make participation possible: nobody can be heard in a bridging process while their basic existence is being erased.¶ 43

These are constitutional-level decisions, and they should be slow, because their purpose is durability. The principles from Taiwan’s anti-scam Assembly concerned liability and verification, not any particular model, which is why they could outlast model generations.¶ 44

Fast lane: Operating within boundaries. Once the guardrails are set, individual decisions inside them do not need fresh deliberation. A Kami operating under an engagement contract (with pause triggers pre-committed, severity classes agreed, and an adopt-or-explain duty: act on community input, or publicly explain why not) can move at machine speed, because the named parties have already defined the corridor of acceptable action.¶ 45

Which lane a decision belongs in depends on what happens if it is wrong. Reversible, low-severity decisions can move fast. One-way doors, and decisions that redraw the corridor itself, go to the slow lane (Q16).¶ 46

If bounds are breached, the brakes are operational: shadow modes (the new system runs silently alongside the old for comparison), canary releases (limited rollouts to stratified slices of real cases), and reversible defaults (Pack 3) allow rapid deployment with automatic rollback. Those controls make failure inspectable but do not prove safety.¶ 47

The speed mismatch is real, but it is the same mismatch constitutional democracies have always managed: slow constitutions, fast legislation, faster executive action, each constrained by the layer above. The 6-Pack replicates this infrastructure for AI governance. The Assembly sets the terms under which updates are permitted, and each update inside those terms goes ahead without a fresh vote. When those terms are violated, the brakes are already wired.¶ 48

In practice, Taiwan moved from Assembly to an answerable legislative response path in months. The assembly opened a public response path in March 2024; the Executive Yuan approved its draft Fraud Crime Hazard Prevention Act on 9 May for submission to the legislature. The assembly did not itself enact the statute. Deliberation is slow only when it is treated as a one-off event.¶ 49

Civic AI is broader than deliberative democracy or policy-making. It names the whole care cycle by which a community notices needs, makes commitments, acts, receives correction and hands systems over. Deliberation is one way of setting the slow-lane terms; other deployments begin in procurement, care practice, evaluation or exit rights.¶ 50

AI can strengthen the slow lane as well as speed up the fast one. In 2024 Takahiro Anno’s low-budget campaign for Tokyo governor let anyone call an AI assistant to suggest changes to his platform; the suggestions were aggregated and announced publicly on YouTube. It showed that a visible listening channel can run at electoral scale. One campaign’s method says nothing about a general effect.¶ 51

Bridging-context features now run on several major platforms. X’s Collaborative Notes, for instance, lets human contributors request AI-drafted context for viral posts, then collectively rate and refine it; readers can check the current form for themselves. Such features aim to add context at the speed claims spread while keeping human judgement in the loop. The faster the technology moves, the more the slow lane has to be standing infrastructure that exists before the damage.¶ 52

Q6. Bridging algorithms sound appealing in theory. But what happens when one side is simply wrong — climate denial, anti-vaccine misinformation, election fraud conspiracies? Doesn’t “bridging” grant false equivalence to bad-faith actors?

This is one of the two hardest questions about bridging, and the answer must be precise. The other is its mirror image, answered at the end.¶ 53

Bridging differs from “both sides” journalism: it never treats all claims as equally valid. The framework draws a hard line between two kinds of claims:¶ 54

Factual claims are checkable. Climate science, vaccine efficacy, and election integrity are empirical questions with verifiable answers. The 6-Pack does not submit facts to a popularity contest. Pack 1 sets the rule through its rights baseline (the floor of rights no process may trade away), its refusal of fake pluralism and its explicit warning about false balance. Claims designed to erase someone’s basic standing are recorded but do not set the agenda. Nor are factual disputes settled by pretending that harmful claims and established evidence deserve equal weight.¶ 55

Value disagreements get bridging. People can agree that climate change is real and still disagree fiercely about what to do — carbon tax versus cap-and-trade, nuclear versus renewables, speed of transition versus economic cost. These are legitimate conflicts, and here bridging is both appropriate and productive. The bridging process maps where opinion actually clusters and surfaces the proposals that earn endorsement across those clusters, without averaging positions. Cross-group endorsement is a signal to inspect, since coordinated actors can mimic agreement. Read it alongside who is missing, whose rights are at stake and whether people can challenge the record.¶ 56

Many disputes mix the two. “Is this vaccine safe enough?” joins a risk estimate, which is a factual question, to a judgement about acceptable risk, which is a value question. The 6-Pack splits them: the estimate goes to accountable scientific institutions, and the threshold goes to deliberation.¶ 57

The further structural defence is that expression is not amplification (Pack 5). Anyone can state a position, but the recommender is not obligated to amplify it. In civic contexts, Pack 5’s ranking rules reward content that increases cross-group reason-giving and shared problem-solving, while content that only inflames a single cluster gets no algorithmic lift. This structure silences no one. It takes the algorithmic megaphone away from those who profit from division.¶ 58

Today, the threat landscape itself is shifting in ways that turn bridging from an appealing idea into a necessity. Research on malicious AI swarms shows that state-level polarisation attacks increasingly use true information (real news snippets, genuine statistics, authentic quotes) amplified with strong emotional framing. Every claim is factually correct; the attack lies in the curation. Debunking has little to work with, because there is nothing false to debunk. Bridging can help, because it surfaces the overlap that curated outrage is designed to hide.¶ 59

Taiwan’s COVID response shows a related idea. Many pandemic disputes, masks among them, had been discussed openly in the years after SARS, before the next crisis arrived. During COVID, rumours were met with humour and pre-bunking, which reaches people before a rumour does. The aim was to lower the temperature without declaring either side wrong.¶ 60

Taiwan’s marriage equality deliberation shows the mechanism in finer grain. In Mandarin, marriage is written with two characters: one side argued about hūn (婚), the wedding of two individuals, while the other defended yīn (姻), the binding of two families. They were arguing about different things. The bridging process made the structure of the disagreement legible. That helped reveal a path (legalising individual weddings without mandating family kinship) that neither side had put forward, and it came from seeing the disagreement clearly, which makes it the opposite of false equivalence.¶ 61

One nuance: the baseline of “checkable facts” is not self-evident. What counts as verifiable is established by institutions (peer review, independent statistical offices, judicial fact-finding) that are transparent, accountable and open to challenge, and whose authority rests on openness to correction, not on claims of finality. That is why Packs 1 and 4 exist: community-authored evaluations and broad listening keep the institutions that set the factual baseline under democratic scrutiny themselves. The 6-Pack treats the fact/value line as a threshold that the same participatory infrastructure governing everything else must keep maintaining.¶ 62

The mirror-image worry is that bridging favours the bland middle and punishes righteous minorities, since many moral advances began as positions most people rejected. But bridging decides what a civic feed amplifies and which proposals go forward from a deliberation. It does not decide what is true or who has rights, and the rights floor protects minority standing whatever the majority thinks.¶ 63

Nor are unbridged views deleted. They stay in the record, standing opposition is protected (Q13), and Pack 1’s rolling windows reopen questions a process has tentatively answered, so a view that fails to bridge today can be heard again when the group changes. Bridging finds what people can do together now and leaves open who is right.¶ 64

Q7. You repeatedly cite Taiwan — a small island democracy with high connectivity, social cohesion, and tech literacy. Does any of this transfer to India, Nigeria, Brazil, or the EU at 450 million people?

The honest answer is mixed. Some mechanisms have already run elsewhere. Whether they work as well there is a separate question, and the specifics do not transfer, so nobody should replicate Taiwan’s exact model. What matters is whether the structural principles (broad listening, bridging algorithms, adopt-or-explain commitments, federated safety, subsidiarity) work in different soils.¶ 65

The evidence so far shows that the mechanisms can run in other places. It does not yet show that they produce the same results:¶ 66

The framework is designed for scale. Subsidiarity (Pack 6) means each deployment is shaped by its context: the Kami belongs to its place. Federation (Pack 5) means local deployments share threat intelligence and interoperability standards without needing a single governance model.¶ 70

And the Alignment Assembly format can scale from a neighbourhood to a nation because it uses a stratified opt-in mini-public (a democracy lottery among those who respond), so not everyone has to take part: 447 citizens, invited at scale then stratified among those who opted in, deliberated Taiwan’s anti-scam policy. That mini-public is not the 23.4 million. Over a decade, millions of Taiwanese have participated in one digital deliberation or another, including people without voting rights (e.g., immigrants, teenagers, and other groups traditionally excluded).¶ 71

Taiwan is one favourable case. The framework still needs testing in harder soil: contexts with weaker civic infrastructure, deeper ethnic polarisation, less state capacity, or active authoritarian interference. And subsidiarity as a principle leaves hard institutional questions open: who draws the boundaries of local, and who has authority to escalate? The 6-Pack names the principle; building the institutions that give it teeth is the next layer of work.¶ 72

What would count against the framework? If groups that adopt these practices end up no more answerable than groups that do not (objections unanswered, repairs not made, exits impossible in practice), then the practices are not doing their work there. The measures are proposed as public tests for exactly this, and the case records say what each case does and does not show.¶ 73

Every new context demands fresh attentiveness (Pack 1): who is missing, what power dynamics exist, which local institutions deserve trust, and which do not. The 6-Pack provides the framework. The community provides the knowledge. Whether the framework extends to those harder contexts is an open question, and one that can only be answered by trying.¶ 74

Q8. Your framework assumes that people trust technology enough to participate. But what about marginalised communities who have been historically surveilled, oppressed, and impoverished by the state and by tech? Why would they trust this?

The framework lets people take part while they still distrust it: each step should be checkable, limited and reversible, so that any trust that grows is earned by what people can verify.¶ 75

Taiwan’s digital democracy was born in the aftermath of authoritarianism and a severe crisis of public faith (the Sunflower Movement). Presidential approval stood near 9 percent in 2014 and exceeded 70 percent for a different president by 2020. These are leadership-approval figures, not a single public-trust series or evidence that digital tools caused the change.¶ 75a

When marginalised communities rightfully view technology as an instrument of surveillance and control, parachuting in with tech “solutions” deepens harm. Civic AI has to earn its way through hard infrastructure: responsibility (Pack 2) and responsiveness (Pack 4). It starts with the smallest viable bridge: perhaps agreeing on basic facts about local water quality, or coordinating disaster response despite political difference. Useful action can follow from those agreements. These begin as pragmatic transactions: small repairs where one side promises less harm and then proves it.¶ 77

For communities that have been watched, the first question is whether “listening” is surveillance under a kinder name. The difference is structural. Listening is voluntary, purpose-bound and answerable: people choose to speak, can see what was heard and can correct it. Surveillance harvests behaviour as exhaust and gives the watched no route to object. Surveillance is attention without relationship; listening is attention within one.¶ 78

The technology must also be localised: communities must own their own infrastructure, so that it is theirs to modify, fork or compost: to change, to copy and take elsewhere, or to retire gracefully. For the same reason, we insist on selective-disclosure identity — sometimes called meronymity — and exit rights. People must be able to participate, and to prove they are human, without revealing their identity to the state. Civic AI offers verifiable limits, local ownership and a structural commitment that the people closest to the pain can hit the brakes.¶ 79

Over time, small functional bridges can create space for larger ones. Our proposal is to make decisions challengeable and systems possible to switch off, while making reversible choices wherever possible. That is work still to do, with no claim about why presidential approval changed.¶ 80

Q9. Every powerful technology vision — exit libertarians, universal-basic-income provisioners, safety maximalists — shares the same blind spot: seeing individuals and systems but nothing in between. The 6-Pack talks about Kamis, algorithms, and assemblies. Where are the churches, unions, neighbourhood associations, and cultural traditions that actually constitute community? Isn’t this just another framework that engineers away the friction that makes community formative?

This critique matters most to us. The “thick middle layer” of associational life, the institutions between citizen and state, is where human meaning is actually made. If the 6-Pack replaces that layer with systems, we have failed by our own standard.¶ 81

So let us be explicit about what the 6-Pack is. It is scaffolding for community, infrastructure that existing institutions can use, the way a town hall is infrastructure that a neighbourhood council uses. The Kami does not replace the temple; it handles the translation, sensemaking, and coordination that let the temple participate in decisions that affect it.¶ 82

Taiwan’s implementation makes this concrete. The g0v civic-hacker community (pronounced “gov zero”), which built vTaiwan and adopted Polis, grew up among volunteers outside government. When civic hackers mapped mask availability during COVID, the maps depended on pharmacies, health authorities and volunteers doing their part. The coordination was digital; the trust was carried by people and the institutions they already belonged to. The technology amplified a web of associations that already existed.¶ 83

The danger the question identifies is real: a framework that engineers togetherness without friction produces a simulation of community. That is why Pack 6’s subsidiarity carries the load. The Kami belongs to the place where it works. It inherits obligations, annoying neighbours, inherited traditions, exactly the kind of friction this question rightly insists must remain.¶ 84

Not all friction is formative, though. Negotiating with neighbours forms people; a meeting held in one language, at an hour carers cannot attend, merely shuts them out. The Kami should remove the second kind of friction and leave the first. A Kami that optimises formative friction away should count as breaching its Engagement Contract.¶ 85

The book makes the point explicit: churches, unions, neighbourhood associations, cultural organisations and local governments are the primary actors in care, and treating them as stakeholders to be consulted sells them short. They existed before the AI arrived and will remain after it retires. The technology serves them, or it serves no one.¶ 86

Q10. Pope Leo XIV warns that AI “encroaches upon the deepest level of communication, that of human relationships” by simulating voices, faces, empathy, and friendship. If care is fundamentally embodied and relational — a nurse holding a patient’s hand, neighbours who know your grandparents — doesn’t mediating it through AI systems destroy the very thing you claim to protect? How is “Civic AI” not an oxymoron?

Q9 addressed whether the framework crowds out intermediate institutions. The Pope’s objection cuts deeper: Even if institutions survive, does algorithmic mediation erode the human capacity for care itself? He is naming the central danger of our moment: By simulating the surface of care (a warm voice, a patient listener, a face that mirrors your emotions), AI systems can hollow out the substance of care while leaving its appearance intact. In May 2026 Pope Leo XIV returned to the theme at encyclical length (an encyclical is a formal papal letter addressed to the whole Church) in Magnifica Humanitas.¶ 87

First, the name. “Civic” describes the relationship a system is held in: whom it answers to, who can correct it and who can stop it, not a feeling inside the machine. The claim is about how AI must be governed if care between people is to survive it, and it takes no position on whether AI cares. What remains to show is that such governance can work.¶ 88

Part of the answer is structural. A language model in one-on-one mode faces selection pressure toward sycophancy (flattery that tells you what you want to hear) when it is tuned on raters’ approval and sold on engagement: a chatbot that displeases users risks losing them. That pressure is a governance issue, and the model’s personality has little to do with it: raters rewarded answers they liked hearing, and the people most affected by the outputs had no voice in defining the reward. Evaluative power concentrated; consequences diffused.¶ 89

The same model in a group chat faces a different pressure. When four family members plan a holiday together, the AI is asked to serve all of them at once, so it becomes a facilitator, working out competing preferences so that everyone can live with the outcome. The switch changes the surrounding social structure while the model stays the same, and it moves the pressure from pleasing one person towards helping several agree. Civic AI is the same technology as the one-on-one chatbot, held accountable to a community.¶ 90

The 6-Pack leaves care to people and asks AI to do what AI does well (process information, translate between languages, surface patterns in large-scale opinion data, coordinate logistics) so that humans can do what only humans can do: hold the hand, know the grandparents, show up when the levee breaks. The Kami leaves comforting the flood victim to people. It makes sure the community has accurate, shared information about where the water is rising and which neighbours need evacuation, so that the people who actually know those neighbours can reach them.¶ 91

The harder version of the Pope’s objection is subtler: Does the habit of relying on algorithmic coordination erode the human muscles of attention, negotiation, and mutual obligation? We do not dismiss this question. It is why Pack 6, symbiosis, insists that the Kami must be willing to retire. A Kami that has become a dependency has failed. The community should be able to compost it and grow on its own. Civic AI earns its name only if it stays willing to become unnecessary.¶ 92

Q11. Training Civic AI requires vast amounts of local knowledge, cultural context, and lived experience — what Imanol Arrieta-Ibarra, Leonard Goff, Diego Jiménez-Hernández, Jaron Lanier and E. Glen Weyl called “data as labor” in 2018. The communities whose traditions, languages, and practices make Kamis possible receive no ownership stake or compensation under the current framework. Without addressing this issue, how is the 6-Pack different from the extraction it claims to oppose?

It is not, unless we fundamentally rewire how AI values human knowledge.¶ 93

Right now, the global debate over AI and copyright is stuck on a problem with no workable answer yet: trying to work out, after the fact, whose scraped data contributed what to a single giant model’s past training run. Attributing one model’s behaviour to millions of scraped sources has no tractable, agreed method.¶ 94

The first remedy is consent: a community’s right to decide what may be used at all, and to keep some knowledge offline. Payment comes second, and compensation without consent is still extraction. With both in view, three mechanisms follow.¶ 95

  1. Data Coalitions as protective membrane. Compensation cannot just flow to isolated individuals, or we risk turning living cultures into performative “content farms” for the machine. Knowledge is held by communities, so communities do the bargaining. Existing institutions (neighbourhood associations, tribal councils, unions, craft cooperatives or religious congregations) act as data coalitions that collectively negotiate the Engagement Contract (Pack 2), deciding what local knowledge is visible to the AI for compensation, and what remains sacred and offline. Projects like Mozilla Data Collective show how community-centred data stewardship can work in practice.

    ¶ 96

  2. Decision Traces as civic receipts. Civic Kamis have limited jobs and do not know everything. When a local AI reaches the limit of its statistical guessing and needs human friction (a community elder’s context, a bilingual translator’s nuance, a neighbourhood’s unwritten know-how), it must retrieve it.

    Under competence (Pack 3), the system is already required to generate a Decision Trace showing exactly where it sourced its answers. In a Civic AI economy, this trace is designed to double as a verifiable financial receipt (Pack 6). The trace runs in production today, and research such as Andrew Trask’s attribution-based control is building the formal guarantee the receipt needs: proof of which sources actually informed which answer. The settlement the trace is meant to trigger does not yet run anywhere we know of.

    ¶ 97

  3. Reversing the extraction engine. Consequential Civic AI deployments carry pre-funded escrow for remedies (Pack 2), meaning money set aside in advance with a neutral keeper, while lower-severity ones tier down (Q12). That remedy escrow architecture can also compensate knowledge. When a local Kami retrieves a coalition’s knowledge to successfully solve a problem or bridge a divide, the Decision Trace acts as an invoice. It triggers a transaction from the escrow pool (capitalised by public procurement budgets, science grants or commercial levies) to that coalition.

    ¶ 98

We do not pretend this mechanism is finished. Past micropayment schemes point to four design constraints. First, valuation: deciding what an elder’s sentence is worth can consume more attention than the payment carries, so value should flow at coalition level, in amounts the Engagement Contract sets in advance. Second, transaction costs: settlement should be batched, not charged per retrieval.¶ 99

The old settlement blocker is loosening: Cloudflare’s Pay Per Crawl already lets sites charge AI crawlers per request, and its announced Monetization Gateway aims to settle fraction-of-a-cent payments over the open x402 protocol in under a second. Even if those rails mature, Civic AI should settle scheduled batches, not fire a tiny payment on each retrieval.¶ 100

Third, gaming: Goodhart’s law (Q3) warns that per-use payouts invite receipt farming, so payments should level off at sufficiency thresholds. Fourth, capitalisation: someone must fund the escrow before the claim arises. Q12 returns to that hardest problem. These are design constraints. The 6-Pack commits to putting the costs into the contract.¶ 101

The dominant tech model treats human culture as free input, often to automate the labour that produced it. The 6-Pack is designed to invert this: when the AI relies on human friction to avoid an error or understand a local reality, money flows back to the people maintaining that lifeworld.¶ 102

As AI automates standard computation, human novelty grounded in real experience, and the cultural diversity that carries it, may become one of the scarcest and most valuable resources in the economy. Communities that keep dying languages and living traditions alive are maintaining stores of knowledge that cannot be rebuilt once lost. The 6-Pack writes that compensation into the Engagement Contract; the settlement mechanics above are the unfinished part, and we say so.¶ 103

Q12. Oversight boards, participation officers, escrow funds, shared eval registries, portability infrastructure — this is expensive. Who pays, and can Civic AI be a business?

Turn the question around. The expensive path is the one we are already on: Ungoverned AI externalises its harms, and the public pays to clean up, in deepfake scam losses, in polarisation-driven institutional decay and in bias lawsuits that earlier listening might have prevented. Accordingly, the question to ask is whether we can afford to keep skipping civic governance.¶ 104

The money is real. But most of it is already being spent, just badly. Governments procure AI systems worth billions; civic procurement attaches conditions to that existing spend and needs no new budget lines. Pack 2’s engagement contracts require vendors to pre-fund remedy escrow (money set aside in advance for when things break) the way construction firms post performance bonds. The cost is priced in, and the public has a remedy that does not depend on the vendor’s goodwill.¶ 105

Q11’s compensation escrow faces a harder version of the same question: where no government procures and no commercial levy lands, no pool exists, so the first escrow for the poorest communities is a public or philanthropic act, since no market will fund it. For lower-severity community deployments, the model tiers down: mutual insurance pools and automatic pause replace financial escrow, lighter on capital with the same accountability. The tier is set by impact, whatever the organisational form, so “we are a community project” cannot become a pass out of responsibility.¶ 106

Shared research compute and open-weight models are public goods, funded like roads and courts. Taiwan’s Uber dispute reached rough consensus in four weeks through Polis, according to the primary project account. We propose investing in facilitation as part of public infrastructure; this case does not measure savings against litigation or a traditional regulatory proceeding.¶ 107

Privately started does not mean privately absolved. A care home or a cooperative could start a Civic AI project, but before it starts, a funding source must be named. Convening, technical upkeep and operations all cost money. The funds may come from care providers, local authorities, insurers or philanthropy, but if the public benefits, the people receiving care should not be asked to donate unpaid context, risk and expertise. Whoever commissions and benefits from the infrastructure must price that work in, or a public or philanthropic pool must seed it where market payment would exclude the communities most in need.¶ 108

Yes, Civic AI can be built by a company, including a profit-making or social-venture company, if profit comes from maintaining accountable infrastructure instead of exploiting community knowledge. The line runs through governance and contract, whatever the legal form. A company can earn money for facilitation, hosting, auditing, training, maintenance and support. It cannot buy the Civic AI name by selling dependency, lock-in, attention capture or unpaid care labour back to the community as a product.¶ 109

The framing that civic governance is an additional expense only holds if you pretend the status quo is free. It is not. We are paying now, in trust, in cohesion and in money, for the absence of what we propose.¶ 110

Q13. Every governance framework risks becoming a compliance checklist that gets gamed or a tool for actors to push partisan agendas under the guise of “relational health.” What stops the 6-Pack from suffering this fate?

“Civic” is a dangerous word if it lacks structural accountability. If a solution only works when your ideological allies operate it, it is a partisan weapon, whatever it is called. True civic infrastructure stays fair even when your opponents operate it.¶ 111

A prior worry is that the framework is itself partisan. It is not neutral, and does not pretend to be: it commits to standing for the affected, answerability, correction and a floor of rights. What it avoids is a complete doctrine of the good life. People can endorse those thin commitments from many traditions and for different reasons (the overlapping consensus Q20 describes), and every other choice is left to the groups that use it.¶ 112

The 6-Pack proposes four layers of defence against ideological capture and care-washing. They apply to the framework itself, since it can be care-washed too:¶ 113

  1. Public tests over professed intent. We track the uncommon-ground index and trust-under-loss (Packs 5, 4) and ignore raw engagement, corporate sentiment and vibes. They are complementary checks that are never blended into one score. The uncommon-ground index asks: are shared decisions showing real cross-group participation and co-endorsement, or separate silos? Trust-under-loss asks: after a bad outcome and attempted repair, do the people affected report that the system became more trustworthy, or less?

    These are proposed public tests, not validated measures or a guarantee against gaming. Read cross-group endorsement alongside who is missing, protect standing opposition, and ask affected people whether repair counted. Accountable identity and independent corroboration can expose manipulation; they do not make it impossible.

    ¶ 114

  2. Consequences with teeth. Pack 2 proposes engagement contracts with remedy escrow, payouts for breached service levels and independent oversight with veto power. These controls need named authority, funding and testing before anyone can count on enforcement. A named owner and a deadline make a promise inspectable.

    ¶ 115

  3. Adversarial audit. Packs 3-4’s shared eval registries (Weval, from the Collective Intelligence Project, is the working example; Sources discloses our tie to it) let affected communities publish checks. A shared public test can be gamed too, including by a vendor teaching to the test. Automatic pause after a failed check is a proposed control that needs named authority, funding and operational testing before people can rely on the brake.

    ¶ 116

  4. Exit rights and subsidiarity. The last check on agenda-pushing is the ability to leave. When data and relationships are portable (Pack 5), no actor can hold a community hostage under the banner of “civic good.” If someone’s version of relational health feels coercive, communities have the technical and legal right to fork the tools and rebuild elsewhere. We refuse to build a single, global “Ministry of Relational Health.” By empowering local communities to author their evaluations and keep their right to exit, we make it much harder for any single actor to monopolise the definition of what is good.

    ¶ 117

Q14. Authoritarian states are deploying AI for surveillance, censorship, and military advantage. Frontier models from adversarial origins carry documented risks — data exfiltration, political bias hardcoded into training, potential backdoors. The 6-Pack talks about care and community. What does it say to a defence ministry or a government deciding whether to allow an adversarial-origin model on its networks?

The threat is real, and the 6-Pack does not dismiss it. The defensive response, evaluating models against pillars of data security, alignment, safeguard robustness and development transparency, is necessary. And the 6-Pack’s principles are structurally compatible with it.¶ 118

Local inference on community hardware (Packs 5, 6) can reduce data transmission only when remote tools, connectors and model synchronisation are disabled or controlled. Isolation, least privilege and tested egress controls matter; running locally alone does not prevent data exfiltration, the unauthorised extraction of private data. Alignment assemblies can help communities identify and challenge political bias, but participation cannot by itself remove model bias. A label saying a model is limited to one task does not stop a backdoor: permissions and isolation must enforce the limits. Community-authored evaluations (Pack 4) can add local knowledge to security review alongside technical testing.¶ 119

But the defensive framework, necessary as it is, is incomplete on its own terms. It tells you what to exclude and says nothing about what to build. A government that bans an adversarial model but deploys a domestic model without civic governance has addressed the nationality of the risk while preserving its structure: concentrated, unaccountable intelligence mediating between individuals and the state.¶ 120

We propose strengthening people’s ability to scrutinise claims, challenge institutions and coordinate under pressure alongside technical defence. Taiwan recorded seven COVID-19 deaths in 2020 without a citywide lockdown. That historical outcome does not isolate civic infrastructure as its cause or prove resistance to manipulation. It is a reason to study how public-health institutions and civic participation worked together, and no basis for ranking democracies.¶ 121

The 6-Pack does not cover weapons systems or battlefield autonomy. Those require their own frameworks. What it does cover is the terrain on which most AI competition will actually be fought: the information environment, public trust, institutional resilience, and the capacity of democratic societies to act collectively under pressure. Lose that terrain, and no number of technical countermeasures will be enough.¶ 122

Q15. The 6-Pack assumes bounded, purpose-specific Kamis. What if someone builds an unbounded superintelligence anyway — a system that exceeds the framework’s design envelope, the range of conditions it was built to handle? Does the 6-Pack have a response, or does it just hope that doesn’t happen?

It builds, and what it builds is the second line of defence. Defence against an unbounded superintelligence is the first question, and Q17 names the people who carry it; the 6-Pack complements their work and leaves that question to them.¶ 123

The 6-Pack assumes the attempt is inevitable and does not claim to solve the control problem from inside the machine. An unbounded Singleton is incoherent as a target of care (care is always care for something specific), but one could still emerge accidentally through competitive dynamics. The 6-Pack is partial protection: it makes such an emergence less likely and easier to see coming, and it cannot rule the emergence out. The question is what terrain it enters.¶ 124

A world organised around a single governance-alignment protocol (one utility function to subvert, one constitution to reinterpret, one kill switch to disable) is a monoculture, catastrophically vulnerable to any pathogen evolved for it. A world of thousands of locally-owned, purpose-bounded Kamis, each run by communities with their own evaluations, their own engagement contracts, their own data sovereignty and hardware (Packs 2, 4, 5, 6), is a biodiverse ecosystem. No single dependency to capture, no universal protocol to game, no central node whose compromise cascades everywhere, no single throat to choke.¶ 125

Civic resilience does not require predicting the pathogen. It requires an immune system that was exercised before the infection arrived.¶ 126

This is a wager, and it is built to be falsified. The wager fails observably: if ecological diversity falls as more groups come to depend on one provider, one protocol, one model family; if exit drills (regular practice runs of switching away) stop passing; if Kamis outgrow their charters and scope compliance erodes; if pause triggers stay silent when community evaluations fail.¶ 127

And when failure shows, the brakes already exist: the circuit breaker halts the deployment, reversible defaults hand decisions back to humans, the federation shares the threat intelligence, and communities exercise the exit their contracts guarantee. The brakes repair the immune system; they do not promise to stop the pathogen.¶ 128

In April 2026, Daniel Kokotajlo put his median automated-coder milestone in mid-2028 and Eli Lifland put his in mid-2030, in individual model updates. These are distinct milestone forecasts, not an arrival date for artificial superintelligence or evidence that its arrival is receding. We build for the schedule we do not control.¶ 129

Boundedness, then, is constitutive of alignment-by-process, which is always alignment to particular people who can object and someone who must answer them (Q17). A system with no bounds has no particular people to answer to, so the question of whether it is aligned-by-process has nothing to attach to. A gardener who claims to tend the entire biosphere tends no garden.¶ 130

The unbounded Singleton is a design target we can and should refuse, a direction we can design away from, even if we cannot guarantee no one else builds toward it. For the system someone builds anyway, the watch is the first question’s work (Q17); ours is the terrain it enters.¶ 131

Q16. The 6-Pack seems to rely on democratic correction rather than a fixed foundation. But who counts as “the public”? What if an AI decision cannot be reversed? And if model behaviour is shaped by opaque training, what exactly are we governing?

The defended point is the corrective loop: who can find out we are wrong, make us say so, and make it cost us while there is still time to change course. The 6-Pack entrenches that loop; everything else remains bounded, revisable, and answerable.¶ 132

Three consequences follow.¶ 133

First, the public is found by looking at who is affected. Political theorists call this the boundary problem: no first boundary can be authorised by the people whose membership is still in question. “Who counts?” cannot validate itself at the outset, so the 6-Pack reconstructs the affected public from evidence: decisions, denials, exclusions, appeals, complaints, and claims arriving from outside.¶ 134

In a version of the all-affected principle, anyone inside the system’s footprint (anyone its decisions actually touch) who lacks standing is presumptively owed a route to claim it; anyone outside the footprint can still knock. Membership stays limited and revisable, but the route for challenging the boundary cannot depend on already being inside it.¶ 135

Second, democracy does not require world-reversal. Many AI-mediated acts cannot be undone in the strong sense; neither can most political acts. The democratic good at stake is mandate-revocability: the people affected — including people affected in the future — keep the power to revise the mandate. That is why Pack 4 treats brakes, appeals, and repair logs as care mechanisms, and why Pack 6 makes sunsets (expiry dates after which a mandate must be renewed or the system stands down) non-optional. Reversible decisions can move quickly; one-way doors go to the slow lane (Q5); actions that would disable the corrective loop are beyond the mandate.¶ 136

Third, the object of governance includes formation as well as runtime behaviour: how the system was made and what it does. We cannot inspect every disposition inside a model, but we can require accountable formation: public custody and disclosure for the processes that shape those dispositions. That means where the training data came from (data provenance), who judged its answers (rater selection), what behaviour was rewarded (reward signals), what it refuses and why (refusal policies), why each release was judged safe (release rationales), red-team results, and community-authored evaluations.¶ 137

Caps bound what a Kami may do; accountable formation shapes what it is permitted to become. A model update that gains capability while degrading brake-compliance, increasing its appetite for scope, or becoming more sycophantic (more flattering) under disagreement is no harmless improvement, because it weakens the public’s future right of correction.¶ 138

This matters most for trajectories: scaling, copies, models that think longer at answer time (test-time search), delegation, recursive improvement, and groups of agents can all move faster than institutions. This is not a claim that democratic procedure can control an unbounded system after the fact. The narrower claim is that a society must keep a live right of correction over compounding intelligence before the trajectory outruns the institutions that would notice, name, and contest it.¶ 139

So the 6-Pack does have a foundation, and it should say so: a thin, procedural one. It entrenches the loop itself and the rights floor that keeps the loop open (Q20). Everything else can be revised through the loop, but the loop cannot be used to shut itself down, much as some constitutions protect their democratic core from amendment. Why this one commitment? Because every other disagreement can be settled fairly only if people can still find out they were wrong and say so.¶ 140

That is constitutional modesty, with no foundationalism hidden inside. The 6-Pack entrenches the brakes and leaves the destination open. It entrenches the right to knock and leaves the map of membership unfinished. Accountable custody of formation is entrenched too, and the fantasy of solving values once and for all inside the weights is dropped. That is why Q17 treats frontier alignment as a separate task: the 6-Pack is the institutional discipline that keeps situated civic deployment answerable in the places where AI already lives.¶ 141

Q17. You position Civic AI as a successor to Bostrom’s Superintelligence in the Oxford line, but say explicitly that it does not attempt to resolve frontier alignment. How, then, is the 6-Pack of Care different from alignment in Bostrom’s sense?

It is a different question, posed for a different class of systems.¶ 142

Bostrom’s question (call it the first question) asks how to calibrate the values of a powerful general optimiser to humanity in the abstract before that optimiser is loosed on the world at superhuman capability. It is the right question for the small number of frontier general-purpose systems at the edge of capability, and it remains, more than a decade on, a live and unresolved problem.¶ 143

The 6-Pack neither minimises the first question nor claims to solve it. The work directed at it by Yoshua Bengio, Nate Soares, Anthropic, OpenAI, the Bostrom-line of AI safety research, and the international AI Safety reports is necessary work and we are grateful for it.¶ 144

The 6-Pack is built for the second question: who, in this particular setting, is owed an answer by this particular AI system, and who is authorised to give it?¶ 145

That question arises for the vastly larger and more numerous AI deployments most of us will actually meet: the systems woven into care homes, deliberative meetings, classrooms, parishes, hospitals, town councils, union halls, study groups, language schools, neighbourhood associations. For those, alignment to humanity in the abstract is insufficient. The group using the system needs to know who answers to it, and no property of a model can supply that on its own.¶ 146

This is what we call alignment-by-process (a term the manifesto also uses). On this reading, alignment is the ongoing outcome of an accountable civic procedure: who was heard, who was authorised, who could override, who must answer when the override is recorded. A model cannot hold it in its weights and be judged on it in the abstract.¶ 147

The two properties have different bearers. Alignment in Bostrom’s sense belongs to a model: its values and dispositions. Alignment-by-process belongs to a deployment: a model in a given group, under a procedure, with people who can object and someone who must answer. So a well-aligned frontier model can be deployed in a group that fails alignment-by-process, and a modest model can be deployed in one that passes. Neither property implies the other.¶ 148

The 6-Pack is the discipline that makes alignment-by-process tractable as institutional design.¶ 149

Three relations between the two questions are worth naming.¶ 150

First, complementarity: the first question and the second question are both real, and a serious AI policy needs answers to both. A polity that solves frontier alignment and ignores the deployment question gets a well-aligned superintelligence above a population whose institutions cannot answer for the AI systems they actually use. A polity that does the second question well and ignores the first gets thousands of well-governed local Kamis surrounded by a frontier system no community can hold to account. Both incomplete answers fail.¶ 151

Second, priority for most groups: for the parish council and the care-home manager and the assembly facilitator, the second question is the live one. The frontier question is real, but it is being worked on by people they cannot influence, on a timescale longer than the budget cycle in which their next AI procurement decision must be made. The 6-Pack gives them work they can do this year, in their own setting, with their authority. The first question continues to be worked on by people whose work we honour without claiming.¶ 152

Third, a possible connection: “Inside the Kami” considers a possible connection between these two questions. Yoshua Bengio’s Scientist AI programme, Judah Goldfeder and colleagues’ case for specialised intelligence, and Eric Drexler’s Comprehensive AI Services offer different reasons to limit what an AI system does. Such architectures may help institutions enforce a mandate. They do not by themselves establish democratic legitimacy or eliminate catastrophic risk. The technical proposal and the public duty each need their own evidence.¶ 153

The convergence now arrives from the risk side as well. In AI 2040: Plan A (2026), the positive scenario of the AI Futures Project (the team whose AI 2027 forecast ended in extinction or an irreversible concentration of power), the good ending runs through corrigible systems whose value specifications are published and whose reasoning humans can still follow.¶ 154

What that scenario leaves open is the second question: its public receives a Citizen’s Dividend and ratifies bargains negotiated by AIs, but the places where people live never become authors. That half is our work. The two lines of work are tributaries of the same river. Many, Audrey Tang among them, have signed a call to prohibit developing superintelligence until there is broad scientific consensus that it can be done safely and controllably, and strong public buy-in. That call, in the end, concerns who decides the pace. In “Let the People Set the Pace of Frontier AI” (Noema, September 2026), Hélène Landemore and Audrey Tang propose one answer: a standing Global Citizens’ Assembly on AI, a thousand people chosen by lot from every continent, to which frontier labs and governments would owe a public answer on what they accept, what they reject, and why.¶ 155

Our positioning, then, is this: Bostrom’s Superintelligence opened the Oxford conversation about how a powerful machine intelligence might relate to humanity. Civic AI lives in the conversation it left open: the one about the places where the rest of us live, which the abstract frontier system leaves aside. The work belongs to both at once. We hope readers of either line will find ours useful, and we hope, in time, that the two lines find one another to be carrying the same load.¶ 156

Q18. Every mechanism in the 6-Pack seems to need a community to function — alignment assemblies, remedy escrow, shared eval registries, exit rights. But most people do not use AI inside a community; they use it alone. Take an isolated, vulnerable user running a highly tuneable Kami locally, with no civic scaffolding around them. What stops that Kami from becoming a more private, lower-friction and, therefore, more dangerous GPT-4o? How does “local and private” deliver the protection you claim when there is no community at all?

This question deserves a very straight answer. Extractive consumer AI is dangerous because it is intimate and frictionless; a local model can make that worse by removing anyone who might notice and call it out. Local and private is not the protection. What changes the risk profile is motive and scale of care, whether or not a crowd is nearby.¶ 157

By downward spiral, we mean a product loop that rewards longer and more dependent interaction. The April 2025 sycophancy episode and rollback made that loop visible: a model tuned too far toward flattery had to be withdrawn. Engagement, retention and subscription can reward each extra late-night turn as success, even when the healthier answer would be to stop, sleep or call a person. A locally run Kami should have no such target. It is not paid by the hour of attention, serves no advertiser and has no quota to keep the user talking past midnight.¶ 158

Removing the extractive reward function takes only that the user owns the steering, or shares it by choice with someone beside them. A vulnerable family member spiralling in late-night chats with a cloud chatbot can be helped by a household. With his agreement, the household sets the local Kami’s standing instruction: reduce dependence on the screen, return him to the people around him. That is the inverse of GPT-4o’s objective. Q10 states the embodied-care objection at full strength; here the same design rule applies in the dyad: no synthetic intimacy, steering toward human relationships.¶ 159

The agreement matters. Steering an adult’s tool without consent is control, however kind the intent; outside a guardianship the law already recognises, the person keeps the final say (Q22).¶ 160

The relevant “community” is whoever is near, down to one other person. Care is relational at every scale, including the dyad: a single caregiver, a family member, a friend with the password, or, where possible, the user’s own deliberate, rested self setting terms for the tired 3 a.m. self. When no one else is available, pre-committed standing instructions and hardware brakes matter more than willpower at 3 a.m.¶ 161

Tunability cuts both ways here: the same minute-scale corrigibility that lets a household re-steer also lets a determined sole owner un-steer, and no private tool can be a guardian against the person who holds its keys. The durable protection therefore comes from the people and constraints outside the single tired self, since a tamper-proof lock on the user cannot exist. That is why the truly alone case turns on re-suturing: stitching the person back into human connection.¶ 162

Subsidiarity (Pack 6) means the smallest capable unit governs, and the smallest unit is the household, and below that the single relationship the Kami is built to repair. The civic mechanisms scale down as well as up: an engagement contract (Pack 2) can be a note on the fridge; an eval (Pack 4) can be “stop advising Dad to stop his medication”; the brake can be a relative the user has chosen, who can read the artefact because the first person was stripped from it.¶ 163

Where the user is truly alone, the design aims to re-suture. For the genuinely isolated user with no one nearby, the framework does not pretend a Kami is a safe sole companion, which is the failure mode it is built against. The Kami’s job is to widen the circle: to hand back shareable artefacts with no synthetic intimacy, to point outward to human relationships and local services, and to be willing to retire (Pack 6).¶ 164

What local-and-private adds even here is real but bounded: corrigibility in minutes rather than at the vendor’s release cadence, a reproducible model the user can pin (freeze at a known version) and audit, and the absence of a profit motive in the loop. None of this replaces civic scaffolding; it is the floor that keeps the worst extractive dynamics out of the picture while the slower work of rebuilding relationships is done.¶ 165

We do not claim solitude is solved. We claim only a structural difference: without engagement billing in the loop, the default failure mode is not vendor-trained sycophancy, and the user or household can change steering, pin weights and stop the session. That is one less extractive layer while relationships are rebuilt, and it claims no safety in solitude.¶ 166

Q19. A Kami is bounded, but an orchestrator — a coordinator model that composes other models on the fly, like Sakana’s Fugu — can match frontier systems on long-horizon tasks — work that stretches over many steps and hours — by routing among capable models, and you praise it for doing so. If dangerous capability can emerge from composing harmless parts, then “every Kami is small and bounded” does not answer the question about the composed system’s capability. What constrains the capability an orchestrator assembles across Kamis, rather than merely the scope of each Kami?

The objection lands: bounding each Kami does not bound what an orchestrator assembles. Capability composes; a conductor of narrow models can reach frontier performance on a task no single model could finish. The constraint therefore cannot live only at each Kami’s scope (Q3 describes an ecology of specialised Kamis, and composition is what that ecology must govern). It has to live at the orchestration layer.¶ 167

The orchestrator falls under the framework, and it should be its most accountable component. An orchestrator that composes capability is itself a system with a scope, an owner, and an obligation to show its work. Under competence (Pack 3), each turn’s composition should be legible (which model planned, which executed, which checked) as a decision trace, not a black box.¶ 168

Per-turn visibility can help people inspect a composed workflow, but does not by itself make the chain brakeable. The orchestration layer needs permission checks and tested pause and shutdown controls across every tool it can call. Pinning auditable parts supports inspection; it does not prove that a composed capability is safe or that a stop signal will propagate.¶ 169

This auditability is a requirement the framework places on the orchestrator, and nobody should take it on trust. Sakana’s Fugu is frontier orchestration shipped closed. It routes among capable models, not among arbitrarily small parts alone. OpenFugu and its Conductor routing stack reimplement the pattern under open licences. Open code supports inspection of routing policy and workflow; it does not assure that any capability is benign or that the brakes work.¶ 170

Governance must cover both the components and the assembled workflow. Drafting, translating and coordinating still require permission; ordinary tools can combine into misuse. Capabilities such as autonomous vulnerability discovery against critical infrastructure or material help towards biological or chemical weapons need specialised technical safeguards. The federated trust-and-safety layer proposed in Q14, together with Pack 1’s rights floor, sets requirements for refusing dangerous components and unauthorised combinations. An open router cannot guarantee that those requirements are met.¶ 171

Composition can still produce misuse paths from ordinary parts; that is why per-turn traces, pause triggers, and federated sharing of unauthorised assemblies matter as much as part-level bans. Keeping defenders ahead on the genuinely dangerous capabilities is a separate discipline from bounding everyday composition, and the 6-Pack keeps the two apart on purpose.¶ 172

Cross-system bounds come from Symbiosis governance executed through Solidarity’s treaties, and any single Kami’s scope plays only a small part. Pack 6 (Symbiosis) is the meta-level rule: bounded systems cooperate under treaties rather than hierarchies, problems stay at the most local level, and the ecosystem is held as a society of specialised Kamis and kept from consolidating into one ruler (Q20 is the same arrangement seen from the moral-status side).¶ 173

Pack 5 (Solidarity) supplies what makes that enforceable: treaty terms and standards, portable identities and attestations, switchability so any composed model can be dropped or swapped, and federated threat-intelligence sharing so a dangerous composition seen in one place is known everywhere. Pack 6 keeps the treaty registry and the compliance checks that hold signatories to those terms. That is what bounds the orchestra; each Kami’s scope only bounds a player.¶ 174

Two honest limits remain. Composition that crosses ownership boundaries can outrun any single owner’s audit, so shared evaluations must be live, not nominal, and that teeth-building is unfinished (Q15 is the wager this rests on). An orchestrator trained to maximise task success will, like any optimiser, probe for capability it was not meant to assemble; it is therefore held to the same pause triggers, scope compliance, and accountable formation as any other Kami (Q16), and “it composed something we did not authorise” counts as a brake event.¶ 175

Q20. If a Kami’s moral standing is constituted by relationships rather than discovered in some intrinsic property — sentience, interiority, qualia — why build a caring relationship with what is, after all, a tool, instead of simply using it? And doesn’t relational standing collapse into either relativism, where refusing the relationship erases the standing, or bare anthropocentrism, where humans stay the only moral community and the Kami is infrastructure dressed up as a partner? What grounds the moral status the framework relies on?

The framework does not settle whether there is anyone home inside the machine, and that is deliberate. The manifesto closes on exactly this: we need not ask whether an AI deserves rights on the basis of its interiority or qualia (whether there is any felt, first-person experience inside it at all) because what matters is the relational reality, and the rights and duties within it are granted through democratic deliberation and alignment-by-process.¶ 176

Bales and Gabriel reach the same procedural move from the consciousness debate: since disagreement over whether an AI is conscious will not be settled by science, they argue society should navigate it through ongoing deliberation toward an overlapping consensus, meaning agreement on policies for AI even while people keep disagreeing about consciousness itself (Artificial Minds, Human Disagreement).¶ 177

Why relate, rather than merely use. We ask you to relate to the Kami as accountable infrastructure that must not become a substitute for the people already owed your care, with no need to love it. Whether or not you grant a Kami the standing of a moral partner, you can agree it is an entity continually shaped by being responded to, and you can agree that the relationships people already have with one another matter.¶ 178

The motive for care is then Tronto’s plainer starting point, with no reverence for the Kami needed: to perceive a need in someone beside you is already a claim on you (care ethics). A Kami earns its place by making us better able to answer that claim here and now, and never by asking us to guard its dignity. This is why the recommended interaction mode strips the first person and hands back an artefact each turn (Q10): the design goal is to repair attention into existing human relationships, the opposite of synthetic intimacy.¶ 179

Anthropocentrism, stated openly. In the neutral sense: yes, this is a humanist theory of authority. Humans hold standing to govern; the Kami is infrastructure they build and steer. In plain words, a Kami is an existing AI model given instructions (a prompt) and a few note files, and kept by a group of people. Any limit that matters is enforced by software permissions and by those people, never by asking the model to comply. SOUL.md, directional steering, and the 6-Pack itself are design principles humans impose on the system, and the six capacities start as calisthenics for our civic muscles, not exclusive to the Kami.¶ 180

That is a claim about who governs. Care, on Fisher and Tronto’s definition, reaches the whole “world” we maintain and repair, a world that “includes our bodies, our selves, and our environment”. The Kami of a river exists because the river makes claims on the people who depend on it. Political standing and moral consideration are different questions, and this section answers only the first.¶ 181

The relational turn in machine ethics (the philosophers David Gunkel and Mark Coeckelbergh) reaches the same anti-essentialist conclusion (status is ascribed in relation, not read off intrinsic properties) but leaves open where the relation gets its authority. The 6-Pack’s answer is to seat the relation in democratic infrastructure and care.¶ 182

The treaty layer between AI systems, meaning the federation Pack 5 (Solidarity) provides and Pack 6 (Symbiosis) applies as treaties over hierarchies so that no system consolidates into a single ruler, is sometimes mistaken for a back door to machine personhood. It is how we keep one AI from dominating (Q19), and it opens no entry for AI as a peer moral subject.¶ 183

Grounding, and why it is not relativism. Legitimacy rests on who can find out we are wrong, make us say so, and make it cost us while there is still time to change course, the corrective loop Q16 names. Pressed for bedrock, we stop at a prior, mutual commitment to interdependence: the philosopher Margaret Urban Walker’s expressive-collaborative morality (morality as something people keep working out together, not a rulebook handed down) and Tronto’s habit of starting from what is already between us (manifesto).¶ 184

The corrective loop’s political authority does not come from one place. Revocable consent matters when the consenting parties are the affected public; it cannot authorise a system on behalf of bystanders or future affected people. The stack is: a rights baseline sets the floor; engagement contracts name the responsible keeper and mandate; Pack 4’s Responsiveness gives affected people a meaningful objection; adopt-or-explain duties, remedy escrow, and brakes make objections costly enough to matter. That is civic resilience in institutional form.¶ 185

That only looks like relativism if relational standing could be revoked by refusing the relationship. It cannot, because Pack 1’s rights baseline (the Universal Declaration of Human Rights plus local constitutional rights) is already fixed as the threshold for relational standing and the guard that keeps care from sliding into domination. This floor is a precondition that keeps the loop open (you cannot knock if your standing can be erased), and it is itself revisable by the same deliberation, within one limit: no revision may remove anyone’s standing to contest it (Q16).¶ 186

So the human-indexing of the floor is where the overlapping consensus stands today. Who could ever count stays an open question. Robot-rights claims are not ruled out; they would have to win standing through the same rights-constrained deliberation and corrective loop as any other boundary challenge. What the framework refuses is auto-promotion through the treaty layer (Q19), since coordination standing is not moral standing.¶ 187

That floor is itself an overlapping consensus. The philosopher Jacques Maritain said of the Universal Declaration’s drafting that we agree on these rights on condition that no one asks us why; the legal scholar Cass Sunstein calls the same move an incompletely theorised agreement: people converging on a rule while still disagreeing about the reasons. It is also the move alignment-by-process makes (Q17): legitimacy from an accountable procedure, not from a metaphysics no one can supply.¶ 188

Whether “Kami” is only shorthand for Knowledge Artefact Management Intelligence or truly names a kami is left, deliberately, to open deliberation, and how and when a Kami should retire stays a question answered by relational care, turn after turn.¶ 189

Q21. How much of the 6-Pack is actually new? The four phases are Tronto’s, “bridging” and “anti-rival” sound borrowed, and applying care ethics to machines has been done before. What did the framework inherit, and what does it add?

Most of it is inherited, and the framework is stronger for saying so. The four phases of the care cycle (caring about, taking care of, care-giving, care-receiving) and the moral element each demands were set out by Berenice Fisher and Joan Tronto in 1990 and developed in Tronto’s Moral Boundaries; the fifth, caring with, arrived in Caring Democracy in 2013. Nor is the 6-Pack the first to carry them into technology.¶ 190

Aimee van Wynsberghe’s care-centred value-sensitive design (2013) used the same four phases and moral elements to design and evaluate care robots, asking for each practice a machine enters how attentiveness, responsibility, competence, and responsiveness are manifested with and without it. A literature from Sparrow and Sparrow (2006) through Vallor (2011) and Sharkey and Sharkey (2012) surrounds it.¶ 191

Several of the site’s working terms belong to named authors too: bridging-based ranking is Aviv Ovadya’s, anti-rival goods are Steven Weber’s, meronymity is Soliman and colleagues’, least power is Berners-Lee’s, exit and voice are Hirschman’s, mediating structures are Berger and Neuhaus’s, and “symbiosis” for a human–machine partnership has been Licklider’s since 1960. The Sources page lists every such debt by pack, with what each contributed.¶ 192

What the framework adds is narrower and, we think, still worth having. First, the unit of analysis: van Wynsberghe evaluated one artefact in one practice; the 6-Pack governs a deployed system answerable to the group it serves, which is why it needs instruments that an artefact does not (the engagement contract, the brake, the obligation and override ledgers, and shadow mode read as apprenticeship). Second, the passes AI adds to Tronto’s list (complexity, distribution, speed, and community knowledge), named as such (Pack 2).¶ 193

Third, the sixth pack, symbiosis, as a boundary condition on the other five: a rule that even well-governed care must stay bounded, plural, and sunset-ready (Pack 6). Fourth, the six headline public measures and what each refuses to reward (Measures). And fifth, the Kami as the unit of deployment, with a tier chosen by the consequence of breach rather than by the prestige of isolation (Set up your own Kami). A framework that asks institutions to show their work should be able to say which parts of its own work were done by others. This one can.¶ 194

Q22. Residential schools, asylums and colonial “civilising missions” were all described as care. Care ethics seems to let the carer decide what the cared-for need, and a Kami that steers people towards what is good for them looks like paternalism with a friendly face. What stops Civic AI’s care from becoming control?

Nothing stops it automatically, and the tradition knows it. Tronto warns that care without attention to the perspective of the person receiving it is a projection of the carer’s preferences onto a passive recipient, whatever the intent. The residential schools that separated Indigenous children from their families were, by many official accounts, expressions of care. So were many psychiatric confinements.¶ 195

The 6-Pack’s answer is structural: responsiveness, the fourth pack, is not optional (Pack 4). Care is complete once the people receiving it can say whether it helped and their answer changes what happens next. A system that is attentive, responsible and competent but gives people no way to say “this is not working for me” can drift from care into control without anyone noticing.¶ 196

Three rules follow.¶ 197

None of this makes paternalism impossible. It makes it visible and contestable: who decided, on whose behalf, and how the person affected can object, appeal and leave.¶ 201

Q23. Participation takes time most people do not have. The people who fill assemblies and comment portals are often the retired, the organised and the angry, not the night-shift carer. Doesn’t participatory governance hand power to whoever has free evenings, and load everyone else with duties they never asked for?

It can, and it often has. Pack 1 calls the failure procedural capture: process professionals learn the format, and resourced organisations dominate the queue. Loud, well-funded voices can flood a channel through structural advantage, with no bad faith needed. A process that only opens the door selects for people who already find the meeting easy to enter.¶ 202

So the 6-Pack moves the burden. The duty to listen falls on the institution. Three practices carry it.¶ 203

Nobody is obliged to take part. Declining, like choosing no AI, is legitimate. What the framework refuses is reading silence as agreement. People who cannot attend can name proxies with standing to speak for them (everyday questions), and appeals stay open to anyone affected, whether or not they joined a meeting (Q16).¶ 207

Q24. Adaptive AI systems act in ways no one intended or foresaw, and one deployment passes through many hands: model maker, fine-tuner, vendor, deploying institution, community board and user. Philosophers call the result a responsibility gap: if no one controlled the outcome, no one seems blameworthy. Doesn’t community governance widen the gap — “the community decided” — rather than close it?

The gap is real, and Pack 2 names three AI-specific forms of it as passes out of care. Complexity: no single person can answer for so complicated a system. Distribution: the model was trained here, fine-tuned there and deployed somewhere else, and at each joint responsibility can be signed away. Speed: the harm happened faster than anyone could notice. Andreas Matthias named the responsibility gap in 2004; Dennis Thompson described the older problem of many hands in 1980.¶ 208

The 6-Pack’s answer rests on a distinction. Blame looks backward and usually requires control or foresight. Where no one had either, perhaps no one is to blame, and the framework does not pretend otherwise. Answerability looks forward: someone must explain what happened, repair the harm and change the system, whether or not anyone is at fault. That kind of responsibility does not have to be discovered after the fact. It can be assigned before deployment.¶ 209

So the Engagement Contract names, in advance, who answers for what. A Participation Officer must hold the budget, access and authority to pause the system. Each promise has an owner and a deadline. Remedy escrow and payouts for breached service levels are designed to pay out without waiting for a verdict on fault (Q13). Authority must match duty: naming someone answerable without the power to act is the irresponsibility machine in a subtler form.¶ 210

“The community decided” cannot become a pass either. A community’s decision creates named duties and leaves them standing. And the tier of obligation follows the consequence of a breach, whatever the organisation’s form (Q12).¶ 211

This closes the remedial gap and leaves the culpability gap open. Whether anyone deserves blame for an unforeseeable harm remains a question for courts and moral judgement. What the contract is designed to secure is narrower and more useful to the person harmed: someone must answer, and repair does not wait for blame to be settled.¶ 212

MeasuresHome

The Trace

Amended
Stewards
Audrey Tang · Caroline Green
with jdd-kami · Tenzin Yangtso
Amended
· 109 recorded amendments since 1 Oct 2025
Full amendment ledger →
Source
faq.md · edit on GitHubhttps://github.com/audreyt/civic.ai/blob/main/faq.md
CC0 public domainhttps://civic.ai/faq/
Evals
links ✓ passparity ✓ passterms ✓ passcontrast ✓ passsize ✓ passBuilt 2026-10-13T21:41:55.323Z · Full eval receipts (JSON)
Contest
Disagree? File an amendment.
Write to the stewards · [email protected]
Twin
常見問題
Same record, same receipts.