About Careertakes

👉 Important disclosure: Careertakes is a third-party recruiting platform supporting this hiring process. If selected, you will be employed directly by our client, IT Services and IT Consulting and Engineering Services.

Applicants for this role may also receive access to additional matched opportunities through the Careertakes platform.

---

Role Overview

Confidential Client is hiring a Software Engineer (Python / TypeScript) for a fully remote, short-term contractor engagement focused on AI training data, preference data evaluation, and rubric-driven quality processes. You will pair hands-on engineering (pipelines, tooling) with careful human judgment (rubrics, data review) to help scale reliable evaluation workflows for advanced language models.

---

What You’ll Do

• Design, refine, and iterate on rubrics and evaluation criteria for preference data.
• Review, label, and audit training/evaluation data with a focus on consistency and subtle quality issues.
• Build, maintain, and improve pipelines and infrastructure that generate, collect, and evaluate preference data.
• Collaborate with researchers and engineers to turn qualitative judgments into repeatable, scalable evaluation processes.
• Produce clear write-ups, summaries, and recommendations from evaluation findings to guide model development and data improvements.

---

Who We’re Looking For

• Proven proficiency in Python for building and maintaining data pipelines and evaluation tooling, or strong experience with TypeScript.
• Hands-on experience with preference data collection, evaluation rubric creation, or related data-quality workflows.
• Strong quality judgment: able to explain why a given example is good or problematic, not just mark it pass/fail.
• Excellent written and verbal communication: can summarize technical findings concisely for researchers and engineers.
• Comfortable working in fast-moving, ambiguous environments and translating qualitative criteria into engineering requirements.

---

Preferred (Nice-to-Have)

• Experience contributing to or navigating large TypeScript codebases (for example, VS Code or similar).
• Prior work on ML data pipelines, annotation tooling, or evaluation systems for large language models.

---

Engagement & Logistics

• Contract type: Independent contractor (freelance). This is not a W-2 employment role; benefits such as medical or paid leave are not included.
• Hours: Prefer 40 hours/week. Minimum overlap of six hours with Pacific Time required. If you cannot commit to 40 hours, state your weekly availability and overlap window.
• Duration: ~3 months, with an immediate start ideally.
• Compensation: Market rate. Candidates should provide a specific rate expectation (per the original posting).
• Location: Fully remote in the U.S.; candidates must be able to contract with a U.S.-based client.

---

Selection Process

• One interview (~30 minutes) focused on technical skills and fit.
• Short technical or sample rubric/data task may be requested for finalists.

---

Why Apply

• Contribute directly to evaluation processes that shape advanced AI models.
• Short, focused engagement with opportunities to extend or join future related projects.
• Work remotely with cross-functional research and engineering teams.

---

Notes on Eligibility & Compliance

• This role is offered by Confidential Client and is open to candidates who can legally contract in the United States.
• Careertakes supports a fair hiring process and will share matched opportunities on its platform where applicable.

---

Equal Opportunity & Hiring Transparency

Careertakes and our client are Equal Opportunity Employers committed to building a diverse and inclusive workforce. We prohibit discrimination or harassment of any kind. To support a fair and efficient hiring process, AI tools may be used to assist with application review or resume screening. These tools do not replace human decision-making. Final hiring decisions are made by people.

If you have questions about how your data is used, please contact us directly.