Practice standard

Five levels of AI proficiency.

A practical capability framework for AI advisors, consultants, specialists and engineers. It sets the minimum contemporary AI capability expected across roles in an AI advisory and consulting practice.

L1 Understand → L2 Apply → L3 Design & Challenge → L4 Build & Test → L5 Engineer & Operate

These are capability levels, not seniority levels. A highly experienced executive can be Level 1 or 2 technically; a younger engineer can be Level 5. The levels are cumulative.

Level 1 suits people in sales, marketing and relationship roles who need to discuss AI credibly without owning solution design. Levels 2 and 3 are the advisory and consulting levels: Level 2 to shape use cases and workflows, Level 3 to design and challenge an enterprise AI solution. Levels 4 and 5 are the build levels, where hands-on prototyping, production engineering, security, evaluation, data and platform depth are required.

Level 1

AI Literate

Understands what modern AI is, what it can and cannot do, and participates intelligently in business conversations without overstating capability.

Required competencies

Explain the practical difference among traditional analytics/ML, generative AI and agentic AI, in plain language.

Explain what an LLM is: trained on large amounts of data, predicts language, works within a context window, and produces probabilistic (not guaranteed) output.

Recognise the common application shapes: chat assistants, copilots, content generation, retrieval-grounded assistants, and agents that use tools to act.

Explain why outputs vary and why hallucination, stale knowledge and overconfidence matter.

Recognise baseline enterprise risks: privacy, data leakage, bias, prompt injection, inappropriate autonomy, regulatory exposure and cost.

Use a leading AI assistant productively for own work and review its output critically.

Level 2

Business AI Practitioner

Translates AI capability into credible business use cases and workflow opportunities, while recognising where AI should not be used.

Required competencies

Command the working vocabulary: prompt, context, grounding, embeddings, vector search, RAG, tool/function calling, agent, MCP, fine-tuning, inference and evaluation.

Recognise the main model categories: general-purpose, reasoning, multimodal, smaller/specialised and open-weight.

Decompose a business workflow into tasks, decisions, information inputs, systems, hand-offs, exceptions and controls.

Identify tasks suited to AI (interpretation, synthesis, extraction, drafting, classification, reasoning over variation) and distinguish them from tasks better handled by rules, SQL, APIs, workflow engines, RPA or conventional software.

Identify where human judgment or approval must remain mandatory due to consequence, ambiguity, authority or accountability.

Understand structured versus unstructured data; permissions, freshness, quality, metadata and source authority.

Sketch a conceptual flow: user/workflow → AI application → model → enterprise data/tools → output/action.

Discuss model choice at a business level: capability, latency, cost, privacy/data residency, modality and vendor constraints.

Identify common failure modes: weak use case, poor data, insufficient grounding, excessive autonomy, fragile integration, poor adoption, high operating cost or no measurable outcome.

Frame a basic value case around capacity, margin, capital, risk reduction or revenue, with before/after measures.

Discuss governance and regulation at a business level: the EU AI Act’s risk tiers and its Article 4 AI-literacy duty, ISO/IEC 42001, Singapore’s Model AI Governance Framework, and the direction of UK and UAE approaches — enough to recognise what a client is obliged to do.

Level 3

Architecture-Literate AI Advisor

The critical advisory level. Sits credibly between business and technical leadership, whiteboards a sound AI solution concept, and challenges whether a design is appropriate, safe, economical and production-ready.

Required competencies

Model and deployment selection: Compare frontier proprietary, smaller, reasoning, multimodal, specialised and open-weight models against task fit, quality, latency, cost, context, privacy, deployment, residency, sovereignty and tool-use requirements.

RAG, context engineering and enterprise knowledge: Explain ingestion, chunking, metadata, embeddings, vector/hybrid retrieval, reranking, context assembly, permissions, citations and freshness — and how what goes into the context window shapes output quality. Recognise that RAG improves grounding but does not guarantee correctness.

Agent architecture: Explain the loop from goal → model/reasoning → tool selection → action/observation → next step. Understand state, memory, orchestration, retries, hand-offs, approvals, multi-agent trade-offs and bounded action.

Tools and MCP: Explain how applications connect models to enterprise data and actions through APIs, tools and protocols such as MCP; distinguish interoperability from trust, authorisation and security.

AI versus deterministic software: Decide which workflow steps belong to AI, rules, APIs, workflow engines, databases or people.

Evaluation: Design evaluation with representative test sets, golden datasets, task-success measures, retrieval metrics, rubric/LLM grading, human review, regression testing, adversarial cases and release gates.

Human oversight: Design AI-proposes/human-decides, threshold-based autonomy, AI-acts/human-audits, AI-reads/deterministic-system-acts and tightly bounded autonomous patterns.

Identity and security: Understand authentication, authorisation, RBAC/ABAC, service identities, scoped credentials, least privilege, prompt injection, data leakage, tool permissions, output validation and auditability.

Governance advisory: Map a proposed solution against client obligations (EU AI Act risk tier and Article 4 literacy duty, ISO/IEC 42001, sector rules); advise on role-based AI literacy programmes; know when specialist legal input is required.

Enterprise integration: Reason about CRM, ERP, databases, repositories, SaaS, email, APIs and event/workflow systems without writing each connector.

Runtime economics: Evaluate token price plus retrieval, orchestration, infrastructure, monitoring, retries, human intervention, failed tasks, engineering and ongoing maintenance.

Production operation: Understand logging, tracing, observability, escalation, model/prompt/tool/index versioning, security monitoring, fallback, rollback and change management.

Architecture challenge: Ask: why this model, why an agent, why RAG, what happens when retrieval is wrong, who is authorised to act, how is success measured, what is the fallback, and what is the blast radius if it fails.

Level 4

Prototype-Capable AI Engineer

Converts a design into a working prototype against real data, tests representative cases, diagnoses failures, and explains what productionising it would require. The entry point of the engineering track.

Required competencies

Use model APIs/SDKs to build with structured outputs, multimodal inputs, context management, streaming and usage measurement.

Implement tool/function calling and connect to REST APIs, databases and SaaS with appropriate authentication.

Build a working RAG prototype: ingest, chunk, enrich with metadata, embed/index, retrieve/rerank, assemble context and generate attributed responses — applying deliberate context engineering rather than dumping data into the prompt.

Build agent/workflow prototypes using an appropriate orchestration approach: a provider SDK, LangGraph, LlamaIndex, n8n or equivalent.

Use agentic coding tools (for example Claude Code, Cursor or equivalent) to accelerate the build, while reviewing generated code and configurations with discipline.

Implement human approvals, retries, basic exception handling, bounded tools and simple state/memory where appropriate.

Create representative test sets, evaluation scripts, graders/rubrics and side-by-side model/prompt/retrieval comparisons.

Measure task success, groundedness, tool-selection accuracy, latency, token/runtime cost and common failure patterns.

Diagnose whether failure originates in instructions, model capability, retrieval, context, source data, tool/API behaviour, orchestration, permissions or evaluation design.

Use Python, JavaScript/TypeScript, notebooks or low-code orchestration to make the concept work end-to-end.

Document the prototype-to-production gap: security hardening, resilience, data pipelines, observability, scale, CI/CD, governance and operating ownership.

Level 5

Production AI Engineering

Designs, builds, secures, deploys and operates production AI systems with enterprise engineering discipline. Level 5 is held in a strand, not in general.

Every Level 5 engineer holds the core below. Each then goes deep in one strand and stays at Level 4 competence in the others. A team, not a person, holds all four.

Core — required in every strand

Application engineering: Production practice in Python, Java or TypeScript; API design; concurrency and asynchronous patterns; testing; dependency management; performance and error handling.

Production evaluation: Continuous and offline evaluation, regression suites, golden datasets, task-success measures, human feedback, red-team cases and release gates. The system does not ship without them.

Observability: Trace requests across model calls, retrieval, tool calls and agent steps; measure latency, errors, tokens and cost, retrieval quality, and user and business outcomes.

Resilience: Timeouts, retries, rate limits, circuit breakers, idempotency, fallbacks, graceful degradation, failover and recovery.

Security: Least-privilege identities, secret management, prompt-injection defence, tool and action permissions, output validation, auditability and supply-chain hygiene.

AI-assisted engineering: Direct coding agents across the software lifecycle — requirements, codebase exploration, implementation, testing, refactoring, documentation and review — with the judgment to verify, correct and override. An engineer who cannot supervise an AI coding tool should not be designing systems in which AI acts on its own.

Cost and performance: Model choice, batching and caching, retrieval, context size, orchestration, infrastructure, and service-level and cost trade-offs.

Governance controls: Implement the audit evidence, logging and human-oversight mechanisms that client obligations — EU AI Act, ISO/IEC 42001, sector rules — require of production systems.

Four strands

L5-A

Production AI Application Engineer

Builds and owns the AI parts of an application, where a person still sees the output before it matters. The default and largest engineering role in most organisations.

Production retrieval architecture and context engineering at scale.
Structured output, schema enforcement, validation and recovery from non-conforming responses.
Caching, state and session memory, and latency budgets under real load.
Integration with CRM, ERP, document repositories, SaaS platforms, email and event systems.
Safe action boundaries for anything the application writes back.
Batch processing over existing corpora, with sampling-based quality control.

L5-D

AI Data & Retrieval Engineer

Owns the context layer everything else reads from. Consistently the most under-planned role, and frequently the reason projects stall.

Ingestion pipelines from line-of-business systems, document stores, mail and operational systems.
Extraction of structure from real-world documents in whatever state they arrive.
Indexing strategy, embeddings, vector and hybrid search, reranking, and measured retrieval quality against a defined set.
Metadata, lineage, deduplication, freshness and access control.
Reference data, taxonomies and domain vocabularies that AI systems depend on.
Data residency, sovereignty and sensitivity constraints.

L5-G

Agentic AI Systems Engineer

Builds systems that take multiple steps and act without a person confirming each one. Assumes L5-A. The scarcest strand.

Orchestration, workflow state, and tool interfaces including MCP.
Memory where warranted, and the discipline to omit it where it is not.
Permission models, scoped credentials, approval gates, action limits and transaction boundaries.
Loop control, cost control and termination conditions as first-class design constraints.
Rollback, compensation and recovery so a failed run does not corrupt a business process.
Evaluation of multi-step behaviour, including cases where the correct action is to stop and escalate.
Separation of model intelligence from operational authority — the system may decide what to propose, but not what it is permitted to do.

L5-P

AI Platform Engineer

Builds the shared foundation many application teams build on. Needed when several teams are building, not when one team is building a lot. Smaller organisations reasonably buy these capabilities from vendors instead.

Model gateway providing consistent access to hosted and self-hosted models; routing, quotas, rate limiting and fallback.
Identity, secrets, scoped credentials and tenant or data isolation.
Token and cost tracking, attribution and controls.
Version management for prompts, configurations, tools and retrieval indexes, provided as a platform service.
Shared evaluation infrastructure so teams run evaluations without building harnesses.
Vector and search infrastructure.
Deployment, CI/CD, GenAIOps and operational monitoring for AI workloads; managed migration across providers and model versions.

L5-P stops at hosted model consumption. Distributed training, GPU cluster operations and inference optimisation are a separate discipline, relevant only when self-hosting models at scale.

Forward deployed — a modifier, not a level.

Your strand is what you engineer. Forward deployed is how closely you work with the problem and the client.

Any strand can be run forward deployed: working inside the client's environment, after the decision, owning the outcome there rather than shipping a feature and moving on. The engineering does not change. Four things are added:

Scoping directly with business owners who describe outcomes rather than requirements, and whose requirements stay ambiguous until interrogated.

Rapid prototyping in the client's environment, against their real data and systems.

Declining work that should not be automated, in the room, without damaging the relationship.

Owning adoption and handover — runbooks, training a team that is not yours, and surviving a security or risk review without treating it as an obstacle.

This is where the engineering track and the advisory track rejoin. A forward deployed engineer needs Level 4 or 5 in a strand and the Level 3 advisory competencies alongside it. That combination — engineering depth plus client-facing composure — is why the role is scarce. Most people are strong at one.

A caution on the label. The term is fashionable and applied loosely, including to support and pre-sales roles that involve little coding. Read the scope, not the title: does this person write and own production code inside the client environment, or design a plan that someone else builds? Those are different jobs, with different candidates.

Where do you sit?

If your level matches one of the roles and the rest of the profile fits, we would like to hear from you.

Express interest