AI governance for an SME is a short list of working rules, not a framework binder. Know which AI tools your people use, decide what data may go into them, name one person who is accountable, require human review of anything that reaches a client, and keep a record you can show when someone asks. If you serve clients in the EU, add a handful of dates from the EU AI Act. This guide covers all of it, in the order we would do it.
TL;DR
- Start with an inventory of every AI tool in use, including AI built into software you already pay for, then assign one accountable owner.
- Write a one-page acceptable-use policy, an approved-tools list and a data-classification table before you write anything longer.
- Shadow AI (staff using unapproved tools) is the most common gap. You can’t govern what you can’t see.
- The EU AI Act reaches US firms that serve EU clients or use AI whose output is used in the EU. Stand-alone high-risk obligations now start on 2 December 2027, but AI literacy and transparency duties already apply.
- Regulated work (finance, health, legal, tax) needs controls that produce evidence: logs, review records and data that stays under your control.
Table of Contents
- What AI governance means when you have 20 people, not 20,000
- What goes wrong without it
- The first five moves, in order
- A starting policy table
- Governing AI for service firms that handle client data
- The EU AI Act for service firms: what applies and when
- Our perspective
- Where we can help
- FAQ
- Sources
What AI governance means when you have 20 people, not 20,000
Large-company governance means committees, model risk teams and audit cycles. A small firm needs the same outcomes with far less machinery. The Australian Institute of Company Directors describes SME governance as proportionate steps: understand how AI is used, align it with strategy and values, identify and mitigate risks, assign accountability, develop policies, and make sure data governance, privacy and security are fit for purpose.
The reason to do it is simple. Australia’s National AI Centre points out that an organization remains accountable for how and where it uses AI, and that AI’s complexity can leave gaps in who is responsible for outcomes. Buying a tool does not buy you out of the consequences. NAB’s small-business workbook says the same thing in practical terms: if AI produces an incorrect claim and you share it with customers, you remain responsible, and AI output can also raise intellectual-property issues.
Those sources are Australian, but the logic is not local. Accountability for what you deliver to a client does not depend on which tool wrote the first draft.
What goes wrong without it
The risks are mundane, which is why they get missed.
- Confidential data in the wrong place. An associate pastes a client contract into a free chatbot to summarize it. A bookkeeper uploads a spreadsheet with payroll data. The AICD notes that lax data security can expose trade secrets or customer data, while unreviewed outputs can cause inaccuracies, compliance violations, contract breaches and reputational damage.
- Shadow AI. A 2026 report from the Financial Stability Board says unauthorized AI use at work reduces visibility into data flows, security controls and adherence to company policy. The same report names weak governance and thin senior oversight as critical vulnerabilities in financial institutions, because they produce fragmented AI adoption nobody can monitor.
- Unreviewed deliverables. Confident but wrong output reaches a client because nobody was assigned to check it.
- No evidence. When a client’s security questionnaire or a regulator asks how you control AI, there is nothing to hand over.
None of this requires a bad actor. It requires only a busy team and no written rules.
The first five moves, in order
The AICD checklist and the National AI Centre guidance converge on a short sequence. Here is how we would run it in a small firm.
1. Build the inventory
List every AI tool, who uses it, what data goes in and what comes out. Include AI embedded in products you already have: your email suite, CRM, accounting package, ERP or cybersecurity software. The AICD specifically recommends including AI embedded in existing technology products and updating the list as use changes. Ask staff directly, and make it safe to answer honestly. An amnesty works better than an audit.
2. Name an accountable owner
One person, with authority, who answers for AI use. In a 15-person firm that is usually a partner, the COO or the head of IT. The National AI Centre guidance, as reproduced in NAB’s workbook, calls for an accountable person and an AI register. Accountability should sit with leadership, and ongoing oversight needs a regular slot in management meetings. For firms running AI through their ERP, this piece on AI oversight in ERP systems shows what that responsibility looks like in day-to-day operations.
3. Write three short documents
Start with one page each:
- Acceptable-use policy: what staff may and may not do with AI.
- Approved-tools list: which tools, on which plan, for which tasks.
- Data-classification table: public, internal, confidential and regulated data, with a yes or no for each tool.
The AICD advises developing an AI policy that lines up with your existing privacy, data, cybersecurity and procurement policies rather than sitting beside them. Do that, and you avoid a second rulebook nobody reads.
4. Require human review where it counts
The NAB workbook says to keep skilled human control over important customer, financial, legal, safety and people-related decisions. Turn that into a rule: any AI-assisted output that leaves the firm is reviewed by a named professional, and the review is recorded. A checkbox in your workflow tool is enough.
5. Set a review rhythm
Put AI on the agenda quarterly. Check the inventory, look at incidents and near misses, and update the approved list. Add basic training. Staff who understand why the rules exist follow them.
A starting policy table
| Data type | Free consumer AI tools | Approved business-grade tool | Private or self-hosted model |
|---|---|---|---|
| Public marketing content | Allowed | Allowed | Allowed |
| Internal procedures | Not allowed | Allowed | Allowed |
| Client confidential data | Not allowed | Only if contract terms and settings are verified | Allowed, with logging |
| Regulated data (health, tax, financial identifiers) | Not allowed | Only after a documented review | Preferred, with access controls and audit trail |
Treat this as a template to adapt, not a standard. Your contracts and your regulators decide the real answers.
Governing AI for service firms that handle client data
A service firm’s product is trust. Treat each AI workflow as a controlled use case: understand what data it draws on and how it is stored, used and accessed, confirm privacy and security protections, and consider the people affected. That is the AICD’s approach, and it scales down well.
Automattic’s agency checklist adds practices that fit any professional-services firm: classify data, keep client data out of free-tier consumer tools, require human review of AI-assisted deliverables, and state AI involvement in the statement of work. The last point is cheap and builds client confidence. Tell clients where AI is used and who checks it.
If you work in a regulated field, the controls have to go further. Our own pages cover the specifics: HIPAA-compliant AI for health organizations, traceability checks for tax preparation under Circular 230, and an ISO 27001 checklist for AI. If you want a formal management-system route, our ISO/IEC 42001 roadmap and the standards-mapped governance guide lay out a longer program.
The EU AI Act for service firms: what applies and when
US firms often assume the EU AI Act is someone else’s problem. If you have EU clients, EU staff or deliver work used in the EU, check. The Act separates providers (who build or place AI on the market) from deployers (who use it). A typical service firm using third-party tools is a deployer. A firm that builds its own AI product for clients may be a provider, with heavier duties.
The timeline moved in 2026. The Digital Omnibus on AI, Regulation (EU) 2026/1744, entered into force on 27 July 2026 and pushed stand-alone high-risk obligations, such as AI used in hiring or credit scoring, from 2 August 2026 to 2 December 2027. AI embedded in products under EU safety law now has until 2 August 2028. Prohibited practices still apply, and Article 50 transparency duties such as chatbot disclosure generally applied from 2 August 2026. Generative AI systems already on the market before that date have until 2 December 2026 for machine-readable marking.
| Obligation | Date it applies |
|---|---|
| Prohibited practices | 2 February 2025 |
| AI literacy | 2 February 2025 |
| Transparency (chatbot disclosure, content labels) | 2 August 2026 |
| Stand-alone high-risk systems | 2 December 2027 |
| High-risk AI in regulated products | 2 August 2028 |
The duty that most small firms overlook is AI literacy. Article 4 requires providers and deployers to take measures to support AI literacy among their staff and others using AI on their behalf. A short internal training and a record of who attended is a sensible response.
The rules continue to change, so confirm current dates with the European Commission before you rely on any of them. For a three-step plan, see our guide to EU AI Act compliance for service firms. It is not legal advice. If you might fall into a high-risk category, such as screening candidates or assessing creditworthiness, talk to counsel with EU AI Act experience.
Our perspective
Most governance advice for small firms stops at policy. We think that is the weak point. A policy tells staff what to do, but it cannot stop a free chatbot from receiving a client file, and it cannot produce an audit trail after the fact.
We come from regulated industries: ISO 27001, pharma, finance. In those settings, getting AI wrong is not an option, and the lesson is that controls belong in the system, not just in the handbook. Private or self-hosted model deployment keeps client data under your control. Logging gives you evidence without extra work from staff. Human review built into the workflow means it happens every time.
That is also why we think the strongest governance position for a service firm is owning the system rather than renting a stack of tools. When you own the workflow, the data path and the audit trail, governance becomes a property of how you deliver, not a project you run next to it. Start with the five moves above. Then decide which workflows are worth building properly.
— Matevz
Where we can help
If your firm sells expertise and you want governed AI inside your delivery, that is what we build. We work with established service businesses in regulated sectors: law, accounting, healthcare, finance, insurance and others. Our model is to invest engineering into the firm, automate its delivery and co-build an AI-native product with the people who hold the domain knowledge. The systems run on private or self-hosted LLMs, so client data stays under your control, with compliance, security and audit trails designed in from the start.
See how Partnership and Venture modes work, or explore the AI OS and its compliance-focused modules. For the security side of the build, our lifecycle security architecture and zero trust for AI guides show the layers we put in place.
FAQ
Do small businesses really need AI governance?
Yes, because accountability does not shrink with headcount. You remain responsible for what you publish or deliver to clients, whether a person or a model drafted it. A small firm can meet the need with an inventory, a one-page policy, an owner and a review step.
What is shadow AI and why does it matter?
Shadow AI is employees using AI tools for work without approval. The Financial Stability Board links it to reduced visibility into data flows and security controls. The fix is an approved-tools list that is good enough that staff don’t go around it, plus a safe way to report what they already use.
Does the EU AI Act apply to a US service firm?
It can, if you place AI on the EU market or if the output of your AI is used in the EU. A firm that only uses third-party tools is usually a deployer, with duties such as AI literacy and transparency. Check your situation with EU-qualified counsel, as the dates shifted in 2026.
What should we do first this month?
Run the inventory and name the accountable owner. Those two steps take days, and everything else depends on them. Then write the three one-page documents.
Sources
- AI Governance Checklist for SME and NFP Directors (AICD)
- Guidance for AI adoption: foundations (National AI Centre)
- AI for Business workbook (NAB)
- Sound Practices for Responsible Adoption of Artificial Intelligence (Financial Stability Board)
- AI Governance Checklist for Agencies (Automattic)
- EU AI Act deadlines after the Digital Omnibus
- Article 4: AI literacy
