Senior Data Labeling Specialist (Minneapolis)

Senior Data Labeling Jobs in Minneapolis (Remote, US)

Rex.zone is hiring a Mid-Senior

Senior Data Labeling Specialist to support large language model evaluation and LLM training pipelines. You will produce and validate high-quality labeled datasets through data labeling, RLHF evaluation, prompt evaluation, and QA evaluation. This role is listed under Minneapolis while remaining explicitly

Remote for US-based candidates.

What You Will Do

• Deliver high-accuracy labels and reviews for complex text and multimodal tasks
• Perform RLHF evaluation (preference ranking and critique) to improve model behavior
• Run prompt evaluation for instruction-following, factuality, reasoning quality, style, and safety
• Conduct QA evaluation (audits, spot checks, calibration, disagreement analysis, error categorization)
• Annotate NLP datasets including named entity recognition and related schema-driven tasks
• Support computer vision annotation (bounding boxes, polygons, attributes) when required
• Ensure annotation guidelines compliance, resolve edge cases, and document decisions
• Contribute to taxonomy development and continuous training data quality improvements

Required Qualifications

• Mid-Senior experience in data labeling/data annotation or AI evaluation workflows
• Experience with at least one of: RLHF evaluation, LLM evaluation, or prompt evaluation
• Strong writing and reasoning skills with consistent rationales for judgments
• Comfort with QA methods (calibration, audits, error categorization) and remote execution

Preferred Qualifications

• Named entity recognition and additional NLP annotation experience
• Exposure to computer vision annotation tools/formats and quality checks
• Familiarity with content safety labeling and escalation practices

Work Model

Remote, FULL_TIME, US-based. You will collaborate with distributed teams supporting AI labs, tech startups, and data annotation operations.

Compensation

$30–$50 per hour (HOURLY), depending on project scope and fit.