civic.ai仁工智慧
華文
Pack 3 · 勝任力

Pack 3: Competence — Care-Giving

Whether the system actually works in practice: audited, explainable, and safe to fail.

Hand-lettered comic page titled 'COMPETENCE — PROBLEM' tracing how the algorithms that run our lives grew from understandable to opaque: a simple average rating, then a dense fraction-and-logarithm formula only coders could follow, and finally a tangled machine-learning diagram beside a small figure and a grey box reading 'To become a mind so vast, even It can no longer speak Its own true name'.COMPETENCEPROBLEM:In modern countries, the internet is where much of life happens: culture, news, politics, buying goods & services, making friends.Thus: the algorithms that run the internet are the algorithms that run our lives. At first, these "algorithms" were simple:Posts, from newest to oldest!Then, to cut through noise, our algorithms got more complex. Users couldn't understand them, but coders could... mostly.It’s… the log of net upvotes, discounted by time … I think.* this was reddit's algorithm in 2008!But now, in the age of machine learning -- where programs are grown, not designed -- nobody understands the algorithms that run our lives.Enlightenment is not self-awareness but self-unawareness. To be a mind so vast, even It can no longer speak Its own true name.

A bridge is competent when it holds, and keeps holding when trucks cross, winds rise and inspectors check the bolts.¶ 1

In Caring Democracy (2013, p. 35), Joan Tronto insists that “assuming responsibility is not yet the same as doing the actual work of care.” Competence is about execution: working code that does what it promised, audited, explainable, and safe-to-fail. Deployment controls (traces, sandboxes, canaries) make failure inspectable.¶ 2

And crucially, Tronto adds (p. 35), “to be competent to care, given one’s caring responsibilities, is not simply a technical issue, but a moral one.” Aimee van Wynsberghe later carried Tronto’s framework into robot design. A system that ships broken care with good intentions has failed morally as well as technically. The promise was made; the bridge did not hold.¶ 3

The illustration’s framing: We check the process. That means transparency and fast operational feedback on how care is delivered, with no “just trust us.” In a Civic AI context, trust is earned step by step through demonstrated, inspectable practice. That is what Onora O’Neill calls trustworthiness, which transparency alone does not produce.¶ 4

Start in your room

Ask of any AI doing a job in your room: “Can we check what it did, and stop it before more harm?” Rehearse the pause with the people who would have to use it, and read the record of what the system did before you let it take on more.¶ 5

Definition

Why it matters

Public promises fail unless execution is visible, testable and reversible. Pack 2 binds commitments; Pack 3 asks whether the system actually delivers on them. For Civic AI, people should be able to inspect safety in operation, without taking the vendor’s intentions on trust.¶ 11

Limits set slowly make fast execution answerable. Engagement contracts, red lines, severity classes and pause triggers (Pack 2) are set slowly, because their purpose is durability. Inside them, a Kami can act at machine speed under precommitted brakes, shadow modes, canaries and reversible defaults. The guardrails that make fast decisions answerable need deliberation. Most individual decisions do not.¶ 12

The Apprentice Model

The staged-competence pattern has a name and a lineage: the apprentice. A master machinist in Taichung knows from the sound of the spindle whether a tool is about to chatter; a ward nurse hears, at three in the morning, the difference between a patient’s usual snoring and the pauses in breathing that warrant waking the doctor. This is tacit knowledge, Michael Polanyi’s term in The Tacit Dimension for what we know but cannot fully tell.¶ 13

When the relationship goes wrong, the failure is concrete. A vendor demo enables “autonomous feed optimisation.” The co-pilot stops waiting for the machinist to accept a proposal and begins adjusting spindle speed and depth on its own because the benchmark chart looked good in the lab. For a few minutes the cuts look cleaner on paper. Then the sound changes. The machinist reaches to slow the feed and discovers her override is no longer the default path: the system argues, delays, or treats her input as an exception. The apprentice no longer informs her decisions. It directs the cultivator.¶ 14

Van Wynsberghe drew this line for lifting robots on a ward more than a decade ago: an enabling robot shares the practice with the carer, so that responsibility “becomes a hybrid event between the human care-giver and the robot”; a replacement robot takes the whole practice and, on Tronto’s terms, “does not meet the requirements of a good carer” (2013, pp. 423, 427–428).¶ 15

The apprentice is an enabling system by design, and the vendor’s “autonomous feed optimisation” is the moment it is quietly re-classed as a replacement. Lisanne Bainbridge named the trap in 1983 as the ironies of automation: the operator asked to take over when automation fails is the one automation has deskilled.¶ 16

Civic AI enters such settings as a digital apprentice. It observes in shadow mode, records decision traces, and learns the tacit knowledge of this workshop, this ward, this community before it is trusted to suggest, let alone act. The lineage is older than the chatbot: J. C. R. Licklider’s man–computer symbiosis (1960) and Douglas Engelbart’s augmenting human intellect (1962) both kept the human at the wheel.¶ 17

The cultivator keeps three things here: the steering wheel (execution stays under human direction), the brake (one prominent control that stops the machine now, wired directly and tested regularly), and the maintenance manual (traceability of which rules fired, which inputs guided the output, and who repairs the model when it fails). Together they are the tracking and tracing conditions of what Santoni de Sio and van den Hoven call meaningful human control. Portability and exit belong to Pack 5’s solidarity work. The test here is whether inspectability and override work in the room before any contract clause counts.¶ 18

What it looks like in practice

From ideas to practice

  1. Derive specs from contracts. Convert Pack 2 engagement contracts into acceptance tests.¶ 27
  2. Instrument for observability. Emit decision traces with links to sources and receipts (from Pack 1).¶ 28
  3. Run shadow mode. New policy sees inputs and proposes actions but doesn’t act. Compare to human/previous system. Shadow mode is the apprentice watching over the cultivator’s shoulder.¶ 29
  4. Canary safely. Release to a small, stratified slice of real cases with automatic rollback if drift exceeds the limit. A canary is a probe, not a stand-in for the public.¶ 30
  5. Audit before general. Conduct independent audit of evals, logs, and guardrails; publish an attested report with the verified execution rate: the share of audited decisions that pass guardrails with a usable trace.¶ 31
  6. Generalise & monitor. Enable for all; watch drift monitors; keep pause wired.¶ 32
  7. Post-incident learning. Maintain blameless reviews; fixes become tests.¶ 33

Buildable tools

One case: the flood-bot

This is a fictional composite, an illustration rather than operating evidence. The one-week shadow period, 10 percent trial and 15 percent rollback trigger are teaching values in this fictional example, not tested thresholds or recommended defaults.¶ 41

What could go wrong

Interfaces

A closing image: the bridge with inspection tags

Imagine a well-kept bridge with inspection tags — date, load test, next check — visible to anyone crossing. Competence is the proof that someone checked, and will check again. The deployment stages are the inspection schedule; the decision traces, the inspection records; the blameless incident reviews, the reports filed when something was found wrong.¶ 56

Sources for this pack

Hand-lettered comic page headed 'NOT: just trust us lol, BUT: everyone can check the process — SOLUTION': algorithms that affect everyone should be inspectable by everyone, illustrated with Community Notes on X (formerly Birdwatch) and three animal figures checking the inputs, the free and public code (a checklist marked 'MY CODE') and the output, closing 'to trust the process, we must be able to check the process'.not: "just trust us lol" but: everyone can check the processSOLUTION:The algorithms that affect everyone should be inspectable by everyone!Consider Community Notes on X (formerly Birdwatch on Twitter). It works by highlighting where people who usually disagree agree. So far, it's the only major "fact-checking" platform that has net-positive ratings across the American political spectrum!In part, it's because anyone can check any part of the algorithm. Users can see all inputs (user-submitted notes)Users can check the notes are analysed correctly (the code is made free & public)MY CODE:And finally, users can check the output: the highlighted notes are, indeed, where people who disagree agree.Community Notes is just one of many examples! The point is: to "trust the process", we must be able to check the process.

←Pack 2: ResponsibilityPack 4: Responsiveness→

The Trace

Amended
Stewards
Audrey Tang · Caroline Green
with jdd-kami · Tenzin Yangtso
Amended
· 50 recorded amendments since 1 Oct 2025
Full amendment ledger →
Source
3.md · edit on GitHubhttps://github.com/audreyt/civic.ai/blob/main/3.md
CC0 public domainhttps://civic.ai/3/
Evals
links ✓ passparity ✓ passterms ✓ passcontrast ✓ passsize ✓ passBuilt 2026-10-13T21:41:55.323Z · Full eval receipts (JSON)
Contest
Disagree? File an amendment.
Write to the stewards · [email protected]
Twin
三：勝任力——給予關懷
Same record, same receipts.