About Careertakes

👉 Important disclosure: Careertakes is a third-party recruiting platform supporting this hiring process. If selected, you will be employed directly by our client, IT.

Applicants for this role may also receive access to additional matched opportunities through the Careertakes platform.

---

About the Role

This is a remote, part-time freelance opportunity to evaluate LLM-based agents by designing clear, repeatable evaluation scenarios and scoring rubrics. You will create test cases that simulate human tasks, define gold-standard behavior, annotate expected outputs and edge cases, and help improve scenarios with engineering partners.

• Work type: Contractor, Part‑time, Remote (US residents only)
• Role level: Entry / early-career
• Pay: freelance rates up to $30/hour depending on skill and project

---

What You’ll Do

• Design structured test scenarios that reflect real-world tasks for LLM agents
• Define the “golden path” and acceptable agent behavior for each scenario
• Break tasks into annotated steps, expected outputs, and edge cases
• Create clear scoring rubrics and gold‑standard examples for comparison
• Review agent outputs, run test iterations, and adapt scenarios as needed
• Collaborate with developers to validate and improve scenario clarity and execution

---

Who We’re Looking For (Qualifications)

• Pursuing or holding a Bachelor’s or Master’s in Computer Science, Software Engineering, Data Science/Analytics, AI/ML, Computational Linguistics/NLP, Information Systems, or a related field
• Advanced English proficiency (C1+) — strong written communication and ability to document detailed guidelines
• Analytical mindset with high attention to detail and ability to spot edge cases
• Comfortable switching between tasks and learning new methods quickly
• Able to work independently on a laptop with a reliable internet connection
• Prior experience with annotation, test design, QA, or LLM/AI projects is a plus (not required)

---

Why this Freelance Opportunity Might Be a Great Fit

• Part‑time, remote, flexible scheduling to fit around studies or other work
• Earn for your expertise — rates can go up to $30/hour depending on skills and project needs
• Gain hands-on experience with LLM evaluation and AI product workflows
• Contribute to how future AI models understand and act in real-world tasks

---

Logistics & Compliance

• This opportunity is only open to candidates currently residing in the United States. Applicants must meet any local contractor requirements (tax and compliance) for the state where they live.
• Engagement type: independent contractor (part‑time); terms will be defined by the confidential client.
• Careertakes will not sponsor work visas for this freelance contractor role. Do not apply if you are not eligible to work as a contractor in the U.S.

---

Skills & Keywords (for candidates and screening)

• LLM evaluation
• Test scenario design
• Annotation & scoring rubrics
• AI/ML quality assurance
• Remote freelance / part‑time
• Up to $30/hour

---

Equal Opportunity & Hiring Transparency

Careertakes and our client are Equal Opportunity Employers committed to building a diverse and inclusive workforce. We prohibit discrimination or harassment of any kind. To support a fair and efficient hiring process, AI tools may be used to assist with application review or resume screening. These tools do not replace human decision-making. Final hiring decisions are made by people.

If you have questions about how your data is used, please contact us directly.