For institutions about to pilot a tool, workflow or redesigned process and wanting the pilot to produce useful evidence.
Outcome: pilot design brief, success criteria, communication plan and evaluation framework the project team can run with.
Written for digital education teams running pilots, assessment, registry and quality teams needing evidence for decisions, academic departments testing redesigned approaches, and project teams coordinating pilots across departments.
Why assessment pilots are important
Assessment pilots can become fragile when they are treated as technical tests only. A tool may work well in a demo, a sandbox or one enthusiastic use case. That does not mean it is ready for wider use across departments, programmes or assessment types.
A useful pilot needs to show what changes in practice: whether workload is reduced, moved or increased; whether feedback quality improves; whether staff trust the process; whether students understand what is happening; whether Moodle or platform integration supports the workflow; whether evidence or analytics can be interpreted fairly; whether support needs are manageable; and whether the process can scale.
Who this is for
This support is useful for:
-
universities piloting AI-supported marking or feedback tools;
-
institutions testing assessment platforms or Moodle workflow changes;
-
digital education teams coordinating pilots across departments;
-
assessment, registry or quality teams needing evidence for decisions;
-
academic departments testing redesigned assessment approaches.
What you get
Depending on scope:
-
clarification of pilot purpose, scope and the assessment types being tested;
-
current-state and pilot-state workflow maps, with named roles, handoffs and decision points;
-
staff and student communication design, with governance, data and consent review;
-
evaluation questions, success criteria and feedback collection routes;
-
pilot findings synthesis into decision-support materials.
Live, sandbox or historical-submission pilots
Different pilot approaches answer different questions.
A sandbox pilot may be safer and useful for configuration, staff confidence, governance and evidence interpretation.
A live pilot can show how the process works under real assessment conditions, but it needs stronger communication, contingency planning and ethical care.
A pilot using historical submissions can support controlled testing, but may not reveal how staff and students experience the process in practice. The right approach depends on what the institution needs to learn.
Typical outputs
-
Pilot design brief and workflow maps.
-
Roles, responsibilities and risk/governance checklist.
-
Communication, support and escalation plan.
-
Evaluation framework and success criteria.
-
Findings summary and decision paper.
When this fits
This is a good fit when a university is considering or preparing a pilot, but needs clearer structure around workflow, risk, communication, adoption and evaluation.
It is also useful when a pilot has already begun and the team needs help making sense of what the pilot is actually showing.
What it is not
This is not a pilot delivery contract and it is not a software evaluation service.
It is structured support for designing, running and learning from a pilot that gives the institution solid evidence to act on.
Related work
See also:
Book a scoping conversation
If your institution is preparing an assessment, AI, Moodle or platform pilot, I’d be glad to hear more.
Frequently asked questions
What should an assessment pilot test?
A useful pilot should test the workflow, workload, staff confidence, student communication, accessibility, grade return, support burden and evidence for decision-making, not only whether a tool functions.
Should a pilot be live or sandboxed?
It depends on what the institution needs to learn. Sandbox pilots are safer for configuration and governance questions; live pilots show more about real staff and student experience.
What makes AI assessment pilots difficult?
AI pilots often involve questions about human oversight, evidence interpretation, student trust, academic standards, workload, data, policy and staff adoption.
What is the output of pilot workflow support?
Typical outputs include a pilot design brief, workflow maps, risk checklist, communication plan, evaluation framework, success criteria and decision-support summary.