To build an internal AI tool, pick one workflow with a measurable outcome. Write a plain-language spec before you touch a prompt. Give the model strict boundaries and a human escape hatch. Build in small phases, test against realistic data, and pilot with the people who will use it. The model is the easy part. The spec, the permissions and the testing are what decide whether the tool survives its first month.
This guide is for owners and operations leaders at service firms: law, accounting, healthcare, insurance, consulting, property management. The steps below work whether you build with your own engineers, with coding agents, or with an outside partner.
TL;DR
- Start with one narrow task and a success condition you can measure, not a general assistant.
- Write the spec first: data model, roles, rules and worked examples, in your firm’s own vocabulary.
- Treat the LLM as a tool-using component with least-privilege access, and route risky actions to a person.
- Build in three to five phases, with a reset-to-seed-data button so every phase gets the same acceptance test.
- In regulated work, decide data location, audit trail and ownership before you pick a model.
Table of Contents
- What an internal AI tool actually is
- Step 1: Choose one workflow and define done
- Step 2: Write the spec before the prompt
- Step 3: Decide the architecture and the boundaries
- Step 4: Build in phases, with a reset button
- Step 5: Keep a human in the loop where it counts
- Step 6: Govern it from day one
- Step 7: Pilot with the people who will use it
- Build, buy or bring a partner
- Our perspective
- How we can help
- A pre-build checklist
- FAQ
- Sources
What an internal AI tool actually is
An internal AI tool is software your own staff use to get work done, with a model doing part of the reasoning. It can be a document-review queue for a law firm, a bookkeeping intake workflow for an accounting practice, or a claims-triage screen for a broker. It is not a public chatbot, and it is not a ChatGPT tab your team pastes client files into.
There are two broad shapes:
- A chat or search layer over your documents. It answers questions and drafts text. Low effort, limited impact.
- An agent inside a workflow. It reads inputs, calls defined tools, writes to your systems, and hands off to a person at set points. Higher effort, and where the cost savings come from.
McKinsey-based analysis describes an “agentic gap”: most organizations stop at AI as a chat tool, while high performers use agents to automate multistep workflows. The same analysis says agents need persistent memory, defined access to internal data and a clearly defined task with measurable success conditions. This guide is mostly about the second shape, because that is where you need an actual method.
Step 1: Choose one workflow and define done
Most failed projects start with an ambition like “help with client service.” Start with a task a new hire could be trained on in a week, such as triaging inbound support tickets. The agent guides from JetBrains say the same: agents that work in production start with a narrow, measurable task, and broad autonomy is something a narrow agent earns after it proves itself. IBM’s guide says the process should begin by defining the agent’s purpose and identifying the tools and information it needs.
Write the task as one sentence, then add three numbers:
- Volume. How many of these does your team handle per week?
- Time per item today. Measure it with a stopwatch for a week, not from memory.
- Acceptable error rate. What share of outputs may need correction before a reviewer loses trust?
Good candidates in a service firm:
- First-pass review of contracts against a clause checklist
- Extracting figures from client bank statements into a bookkeeping ledger
- Pre-filling KYC forms from onboarding documents
- Checking a tender response against a requirements list
- Drafting a lease abstract from a signed lease
Bad candidates: anything where nobody can say what a correct answer looks like.
Also write down what the tool will not do. Personal Software Factory’s guide recommends deciding what you will leave out. That decision keeps the scope from growing every time someone sees a demo.
Step 2: Write the spec before the prompt
The most common failure is treating the build as prompt engineering instead of software engineering. The same guide argues that an internal tool needs a formal spec in plain language. It should define the data model, role-based access and state transitions before any code is generated or prompts are written.
Your spec should cover:
- Entities. Matter, client, invoice, claim, tenant, whatever your firm actually calls them.
- Roles. Who can see, edit, approve and override. If you have partners, associates and paralegals, the tool needs those three roles, not “admin” and “user.”
- States. For example: received, extracted, needs review, approved, filed. Every item sits in exactly one state, and the spec says which moves are allowed.
- Rules with worked examples. Don’t write “flag unusual invoices.” Write “flag any invoice more than 15% above the vendor’s trailing average, and here are three real examples.”
Use your domain’s words. The guide’s example is “approver” versus “manager”: generic product language makes the tool fail to map to how work really happens. A spec written by the person who does the job beats one written by an engineer who has never seen a closing binder.
For firms that bill by the hour, this is also where you capture partner judgment: the unwritten rules about what gets escalated. Our write-ups on encoding partner judgment in an accounting firm and on law firm automation with human review and ethics controls go deeper on that.
Step 3: Decide the architecture and the boundaries
OpenAI’s practical guide to agents describes an agent as three core parts: the model, the tools it can call, and the instructions that guide it. For an internal tool, the tools are the part to design most carefully. Each tool is a door into your systems.
Give the model the narrowest keys
The OWASP Top 10 for LLM Applications 2025 lists prompt injection and sensitive information disclosure as its top two risks. Prompt injection is a fundamental risk because models do not reliably separate your instructions from untrusted content, such as a PDF a client emailed in. The mitigations in the document are practical:
- Sanitize data and validate inputs.
- Apply least privilege, with the application holding its own API tokens, and handle sensitive functions in code instead of handing them to the model.
- Filter and check outputs, for example by testing whether an answer is grounded in the retrieved sources.
In practice: the model should be able to propose “approve this invoice,” but your code should decide whether that action is permitted for that user.
Use an outbox for anything with side effects
For approval and finance workflows, the Personal Software Factory guide recommends an outbox pattern. Notifications and external API calls are written to a database table first and processed afterward. This lets you test notification logic without real Slack or email, and lets you retry after a failure without losing the transaction’s state. If you have ever had an automation send the same email twice, you know why this matters.
Decide where the data lives
This is the question regulated firms must answer first. Options include a vendor’s hosted API with a zero-retention agreement, a private deployment in your own cloud tenancy, or a self-hosted open model. Choose based on what your client contracts, your regulator and your insurer allow. If client data cannot leave your control, say so in the spec so nobody quietly wires in a public endpoint later. Our pieces on zero trust for AI in regulated teams and a seven-layer AI security architecture cover this in detail.
Step 4: Build in phases, with a reset button
Do not ask a coding agent or a developer for the whole system in one go. Build in three to five phases, and run acceptance checks after each. A typical sequence:
| Phase | What gets built | Acceptance check |
|---|---|---|
| 1 | Data model, roles, empty screens | Each role sees only what it should |
| 2 | The core workflow with no AI, manual inputs | An item moves through every state |
| 3 | The model step, behind a feature flag | Outputs match worked examples from the spec |
| 4 | Outbox, notifications, audit log | Failures retry cleanly, every action is logged |
| 5 | Reporting and pilot polish | Pilot users finish a real task unaided |
Building the workflow without AI first (phase 2) is deliberate. If the process is wrong, you find out before model behavior muddies the picture.
Build the reset-to-seed-data button
The Personal Software Factory guide calls the reset-to-seed-data button the most underused part of internal tool development. Build a way to restore the database to a known, realistic state: edge-case users, an approver who is out of office, transactions sitting exactly on every rule boundary. Then every phase gets the same acceptance run, and testing stops being a manual, error-prone chore.
For an AI step, extend the idea: keep a fixed set of 30 to 50 real, anonymized examples with known correct answers. Re-run it after every prompt or model change. It is your regression suite, and it is the only honest way to know whether a change made things better.
Step 5: Keep a human in the loop where it counts
OpenAI’s guide lists conditions where the agent should hand control back to a person: when it is uncertain, when it exceeds retry limits, and when it attempts a high-risk action such as a financial disbursement or an access change. The McKinsey-based analysis above also raises the question of organizational policy on human-in-the-loop quality control.
A workable rule set for a service firm:
- Always review anything that leaves the firm: filings, client letters, payments.
- Review by confidence. Below a threshold you set from your test set, the item goes to a person.
- Sample the rest. Have a reviewer check a random slice of auto-approved items each week and track the correction rate.
The design question is whether review is built into the system or bolted on after mistakes. We wrote about that difference in constitutive versus corrective human in the loop.
Step 6: Govern it from day one
Governance is easiest when it is part of the design. The NIST Generative AI Profile (AI 600-1) emphasizes building trustworthiness in early and updating risk tiers to cover generative AI risks, such as malicious use and dependencies between the AI system and other IT systems. It treats this as part of core oversight, not a separate hurdle.
For a single internal tool, keep it lightweight:
- An owner named for the tool, and a named reviewer for outputs
- A one-page register of what data it touches and which model it calls
- An audit trail of inputs, outputs, approvals and overrides
- A written procedure for pausing the tool
If you need a format for the risk register itself, these AI risk management registers show how enterprise teams structure them for a board. When you need to go further, our guides on a NIST AI RMF evidence pack and an ISO/IEC 42001 implementation roadmap cover the formal route. Check with your compliance counsel or your regulator’s guidance before putting client data through any new system. Requirements differ by profession and by state.
Step 7: Pilot with the people who will use it
Benmore’s guide advises starting with a specific workflow, making access rules explicit, and piloting with the users themselves. Take that literally:
- Pick three to five users, including one skeptic.
- Run the tool in parallel with the old process for two to four weeks.
- Compare time per item and correction rate against the numbers from Step 1.
- Collect the failures. Add each one to your seed data and test set.
- Expand only when the numbers hold for two consecutive weeks.
If the pilot users go back to the spreadsheet, the tool failed regardless of its accuracy. Find out why before you add features.
Build, buy or bring a partner
A fair comparison, by situation:
| Situation | Likely best route |
|---|---|
| Generic task (meeting notes, email drafting) | Buy an off-the-shelf tool |
| One workflow, an in-house engineer, low-risk data | Build in-house with coding agents |
| Regulated data, audit requirements, a workflow that is your competitive edge | Build with a specialist partner and own the result |
| You want to turn your delivery method into a product clients can buy | Co-build a product with a technical partner |
The dividing line is ownership. Renting a tool means your process lives in someone else’s roadmap and your data in someone else’s system. For commodity tasks that is fine. For the workflow that makes your firm different, it is a risk.
Our perspective
We see the same pattern in firms that come to us. They have deep expertise, a good client base, and a pile of manual delivery work. They have usually tried a chat tool or two. The tools helped individuals, but nothing changed in how the firm delivers.
Our view is that an internal AI tool should be a system you own, not a subscription you rent. That means your data stays under your control, with private or self-hosted models where the work demands it. It means audit trails exist because the architecture creates them, and the logic that encodes your experts’ judgment belongs to you. Our team comes from regulated settings, including ISO 27001, pharma and finance, where a wrong output has consequences. That shapes how we scope: narrow tasks, explicit review points, evidence by design.
The second thing we believe is that the best internal tool often becomes a product. A firm that has automated its delivery has also written down its method. With the right partner, that method can be sold to clients as software. That is why we work as an investor and builder, not as a vendor of one-off AI projects.
— Matevz
How we can help
If your firm is in a regulated field and you want to build a system you own, we do this in a few ways. Partnership mode and Venture mode cover different levels of involvement, and we also offer an AI-native grant and a funded build. Our approach puts engineering into an established service business on a revenue-share or equity basis, so you bring the expertise and clients and we automate delivery and co-build the product. We are EU-based and support private and self-hosted LLM deployment. See the AI OS or talk to us about a build.
A pre-build checklist
Before you commission or start anything, confirm you can answer yes to each:
- Can I describe the task in one sentence, with a measurable success condition?
- Do I know the current time per item and error rate?
- Is the spec written in my firm’s vocabulary, with roles and states?
- Have I decided where client data may be processed?
- Does every high-risk action route to a named human?
- Do I have a seed dataset and a test set of real examples?
- Is there a named owner, and a way to pause the tool?
- Are pilot users chosen, and is a comparison period planned?
If three or more are a no, spend a week on the spec. It is the cheapest week of the project.
FAQ
How long does it take to build an internal AI tool?
A narrow single-workflow tool built in three to five phases can reach a pilot far faster than a broad platform, but the time depends on integrations, data quality and review requirements. Allow the pilot period of two to four weeks on top of the build. Treat any fixed promise before the spec exists with suspicion.
Do I need engineers, or can I use a no-code tool?
For low-risk, internal-only tasks, no-code and AI coding agents can get a prototype running. Once the tool touches client data, approvals or payments, you need someone who owns permissions, logging and testing. That is engineering work, whoever does it.
How do I keep client data safe in an internal AI tool?
Start with least-privilege access, input validation and output checks, which are the mitigations in the OWASP LLM Top 10. Then decide where the model runs: a hosted API with contractual limits, a private cloud deployment or a self-hosted model. Confirm the choice with your compliance counsel.
What is the difference between an internal AI tool and an AI agent?
An internal tool is the whole application your staff use: screens, roles, data and logs. An agent is the component inside it that reasons and calls tools to complete steps. Most useful internal tools contain an agent, but the tool also needs everything around it.
Sources
- Personal Software Factory: How to build an internal tool with AI
- McKinsey State of AI 2025: What It Means for Engineering Leaders
- OpenAI: A Practical Guide to Building AI Agents
- OWASP Top 10 for LLM Applications 2025
- NIST Generative AI Profile (AI 600-1)
- IBM: How to Build an AI Agent
- JetBrains: How to Build an AI Agent
- Benmore: Build an internal tool with AI
- Polarising: AI Risk Management for Enterprise Leaders
