Before You Deploy: Eight Questions for AI Builders
Nana Banana Generated
A continuation of My Data Manifesto.
We all use AI systems now. Most of us build with them, write with them, or make decisions informed by them. But very few of us stop to ask what those systems carry. Not what we put in, but what was already there.
Documents hold language patterns their authors never intended. Systems encode assumptions their developers never examined. The distance between the person who builds a tool and the person affected by it is almost always wider than anyone in the room realizes.
This was written for social innovators, the people building tools in education, health, housing, justice, and community development. But it should be applied across domains. Social innovators are often the first to center the people a system touches. Engineers and developers bring the architecture. Both need to pause before they ship.
This is an invitation to reflect on how you build, how you use, and what you carry forward when you deploy.
Eight questions to sit with.
1. Who are you building for, and who are you building on?
Name the people your system serves. Now name the people whose data trained it. Are they the same population? If not, what assumptions are you importing from one group and applying to another?
If you can't answer this, you don't know your own system.
2. Who has the least power in this interaction?
Every AI system operates inside a power dynamic. Someone has less information, less access, less recourse. Identify that person. Now ask: does your system make their position better or worse?
A system that benefits the institution while burdening the individual it claims to serve is not innovation. It is automation of an existing inequity.
3. What does your system do when it doesn't know?
If your system generates an answer every time, regardless of confidence, it is not helpful. It is dangerous. Silence is not failure. Fabrication is.
Define the conditions under which your system should refuse to answer. If you haven't defined them, your system will fill every gap with plausible noise, and the person on the other end won't know the difference.
4. Can the person it affects understand what it is doing?
If a parent, patient, applicant, or student cannot understand your system's output without professional translation, you have not built a tool. You have built a gate.
Complexity that serves the builder but obscures the impact is not a design constraint. It is a design choice. Make a different one.
5. What are you refusing to collect?
The question is not what data you can collect. It's what data you won't. Every boundary you draw before deployment is an act of respect. Every boundary you fail to draw becomes an extraction you can't undo.
If a family uploads their child's educational record to your system, that is not a data point. It is a trust transfer. Build like you understand the difference.
6. What breaks at scale?
A flawed system tested on 50 people is a prototype. A flawed system deployed to 50,000 is an institution. Scale does not fix bias. It compounds it.
Before you celebrate growth, ask: what error rate am I comfortable with at this volume? Who accumulates the consequences of that error rate? Is it you, or is it them?
7. Whose average are you designing for?
The "average user" is a statistical ghost. Cognitive science tells us that variability within individuals often exceeds variability between groups. If your system is optimized for the mean, it is systematically excluding the margins.
Design for the full distribution. When you build for the person the system was never designed for, you make it more resilient for everyone.
8. What would this system look like if you were on the other side of it?
If you were the parent receiving this output. The patient reading this recommendation. The applicant filtered by this score. Would you trust it? Would you understand it? Would you feel like you mattered in its design?
If the answer is no, you are not done.
Now Create Your Own Data Manifesto
Take your answers above and use them. Copy the prompt below, along with what you wrote, and paste it into whatever language model you use. Claude. ChatGPT. Gemini. It does not matter.
Review what comes back. Edit it. Cut what feels generic. Rewrite what feels borrowed.
It should sound like you. Not like a model.
The PromptYou are helping me write a personal data manifesto. A manifesto is not a policy document or a marketing page. It is a public declaration of principles that will guide how I build, deploy, and think about AI systems.Below are my answers to nine questions. Based on these, write a manifesto in my voice. It should be direct, grounded, and free of jargon. It can name harms, but it must also point toward what I am building for. Use short sections with clear headings.Keep it under 800 words. Do not use bullet points. Do not use corporate language. Write it like I would say it out loud to someone I respect.The Questions1. What moment or experience brought you to this work? Tell the story concisely.[Your answer]2. What identity or identities shape how you see this work? (Examples: parent, engineer, teacher, clinician, someone who was failed by a system.)[Your answer]3. What is one thing most people misunderstand about how AI systems actually work?[Your answer]4. Who is most vulnerable to harm in the systems you are building for or thinking about?[Your answer]5. What is one thing you refuse to do with data, even if it is legal or common?[Your answer]6. What should a system do when it does not know the answer?[Your answer]7. What does success look like for you, if not scale or downloads?[Your answer]8. Name one measurable outcome that will tell you whether you are living up to this manifesto. It could be a refusal rate, a readability threshold, a disparity gap between users, or something else entirely. What number would reassure you? What number would worry you? If you do not have a number yet, name what you would need to measure to find one.[Your answer]9. Finish this sentence: "I build because…"[Your answer]Write the manifesto now.