AI as the Engine
not the Product
AI History
Machine Learning used as early as the 1950s
(models trained for very specific use cases)
Imagined for much longer by Computer Science pioneers such as Alan Turing
Has been a common theme in science fiction for a long time (100+ years)
Transformer architecture changed everything
Beyond Prompting
Enhance personal AI usage with tools (💘MCP)
(web browser, computer access, any API!)
How do we move beyond personal use of AI?
We incorporate AI as a system component!
AI as a System Component
Most software isn't AI-based and shouldn't be
Q: When does AI make sense as a system component?
A: Whenever you need a skilled system user behind an API
Benefits of AI as an Engine
Domain expert access
Rapidly testable component
Easily upgraded (reference a new model)
AI as a system user
Treat AI as a non-human system user, capable but non-deterministic
Apply robust architecture and engineering patterns around your system users
Use validation, guardrails, clear communication
AI Architecture
Human user: user(wait) → frontend → backend
System user (AI): backend → system user(prompt)
Combined: human → frontend → backend → AI
Future state? human → AI → backend 🤔
Data Architecture Template (I-P-P-R-R)
Ingest: Gather the necessary data
Process: Transform data into usable format
Prompt: Construct effective prompts + tools
Record: Store results and metadata
Report: aggregate/summarise results, iterate!
3 Real World Examples
None of them are AI products
Startups through to enterprise
All used AI as a key system component
Contract Compliance
Problem: Large insurance provider needed validation for aging claims system
Solution: Take a large anonymised snapshot, follow I-P-P-R-R to gather meaningful insights
Tech: OpenAI model on Azure
Outcome: An automated review tool that could be powered by non-technical staff
Key Lessons
Operating on complex/messy data is much harder than AI integration
(database was optimised for storage limits of the 1970s)
There is no substitution for deeply understanding a problem domain to craft an effective solution
At one stage a single execution could have spent up to $50,000,000 (500x our budget)
Big O notation matters!
Medical Billing
Problem: billing provider needed data from 1000s of pages, with skilled human oversight
Solution: AI-powered OCR as an automated form-prefill
Tech: Claude via Bedrock & Poppler (PDF library)
Outcome: Higher quality OCR than dedicated OCR teams were able to offer
Key Lessons
Combination of printed and (mostly doctor 😅) handwriting is not easy to work with
Pre-filling can be more valuable than complete automation
AI can build its own guardrails! We scraped all the MBS (Medicare Benefits Schedule) rules, over 13k
Forward Planning Schedule
Problem: Multi-month schedule (1000s of transport events), 100s of soft/hard constraints
Solution: Create rule structures, get AI to fill them in with human approval
Tech: Bedrock, rules engine
Outcome: AI as a rule generator rather than a rule executor
Key Lessons
Context window limits and token usage needs to be considered carefully
Non-deterministic actors can build great deterministic solutions
Human users reviewing system users is a great synergy
Takeaways
AI as a system component is API + tools + strong communication
Improve your personal and system AI with tools
(MCP: model context protocol)
Ingest, Process, Prompt, Record, Report
Humans are at the heart of what we do, AI should empower rather than replace them
Use AI pragmatically, avoid the hype
Thank you for your time!
Questions?
Scan for slides