AI Data & Analytics Advanced
AI Benchmark Dataset Designer
An AI Benchmark Dataset Designer architects curated evaluation datasets that objectively measure AI model capabilities, safety, fa…
Demand 9.0/10
AI Risk 25%
Salary $110,000-$195,000/yr
Benchmark task design and taxonomy creationStatistical methodology for evaluation (confidence intervals, effect sizes, bootstrap)Prompt engineering and adversarial input craftingAnnotation pipeline design with inter-annotator reliability metrics (Cohen's kappa, Krippendorff's alpha) +8