Innovations in Assessment
With the rise in popularity of computer‑based testing and the resulting need for large volumes of high‑quality items, traditional assessment development, which can be time-consuming and costly, can no longer meet demand. AIR is dedicated to exploring, researching, and responsibly implementing artificial intelligence-based (AI) assessments.
The awarded methodology offers a data driven, cost-effective approach to industry practice analysis, providing real-time insight into the accounting job market and emerging skill trends such as technology proficiency, critical thinking, and cybersecurity. Its replicable design also enables identifying future trends across other professions.
Through exploration and implementation, we help our public- and non-profit sector partners, including professional associations, credential entities, and sectoral training program providers, in reducing the costs associated with item development and ensuring the ethical application of AI.
We are committed to understanding and using AI because it offers significant practical advantages, such as efficiently developing assessment items, improving test security through expanded item banks, and reducing subjectivity by designing and applying structured guidelines and algorithms.
We understand that AI-based assessments must meet the same rigorous legal and regulatory criteria that traditional assessments do, and we are particularly attuned to the risks of algorithmic bias and how to take steps to mitigate it. We help our assessment provider clients think through what they should consider before using AI to develop assessments.
We offer expertise in:
- Building automatic item generation (AIG) infrastructure that supports scalable, high-quality AI enabled test development within an organization’s assessment program. ‑quality AI‑enabled test development within an organization’s assessment program.
- Evaluating AI based assessment solutions and vendors, including key technical, psychometric, and regulatory factors programs must review before adopting AI‑based assessment solutions and vendors
- Identifying and addressing common operational challenges, from workflow integration to quality assurance, when implementing AI-driven assessments at scale.
- Determining when to adopt, pilot, or delay AI within a testing program, based on legal defensibility, readiness, data quality, model performance, and risk mitigation needs.
