Statistical and Psychometric Evaluation of AI Systems

---

Documentation for package ‘aiEvalR’ version 0.1.0

• DESCRIPTION file.
• User guides, package vignettes and other documentation.
• Package NEWS.

Help Pages

adversarial_stability	Adversarial stability
ai_bootstrap_reliability	Bootstrap confidence interval for AI reliability estimates
ai_calibration	Overall calibration summary for AI system outputs
ai_conditional_sem	Conditional standard error of measurement for AI outputs
ai_dashboard	Integrated AI Evaluation Dashboard
ai_decision_consistency	Decision consistency for AI classification/scoring decisions
ai_dif_mh	Mantel-Haenszel differential functioning test for AI-scored binary items
ai_dstudy	D-study: project generalizability/dependability for given facet sizes
ai_fairness	Overall fairness summary for AI system outputs
ai_generalizability	Multi-facet generalizability (G-)study for AI outputs
ai_group_disparity	Group-level disparity in AI outcomes
ai_internal_consistency	Internal consistency of AI responses across parallel items
ai_reliability	Overall reliability summary for AI system outputs
ai_robustness	Overall robustness summary for AI system outputs
ai_test_retest	Test-retest reliability of AI responses
brier_score	Brier score
calibration_curve	Calibration curve (reliability diagram data)
citation_accuracy	Citation accuracy
ece	Expected Calibration Error
empirical_output_interval	Empirical output interval for AI numeric outputs
fairness_dashboard	Fairness dashboard summary (text form)
hallucination_rate	Hallucination rate across AI outputs
lexical_overlap	Lexical overlap between an AI output and a reference source
prompt_sensitivity	Prompt sensitivity analysis
stress_test	Stress test AI responses under a set of perturbations