Senior Software Engineer - Python / TypeScript | AI Evaluation
The Senior Software Engineer will lead the design and maintenance of AI evaluation pipelines and workflows focused on preference data quality. This role involves collaborating with researchers to develop scalable evaluation rubrics, ensuring data consistency, and providing technical analysis and recommendations to improve AI evaluation processes.
About Careertakes
👉 Important disclosure: Careertakes is a third-party recruiting platform supporting this hiring process. If selected, you will be employed directly by our client, Information Technology & Services.
Applicants for this role may also receive access to additional matched opportunities through the Careertakes platform.
About the role
We’re hiring a Senior Software Engineer (Python / TypeScript) to help build and scale systems that support AI evaluation workflows. You’ll work with a cross-functional team to design production services, tools, and pipelines that evaluate, validate, and improve ML/AI models and data quality for a confidential client in the Information Technology & Services industry.
What You’ll Do
- Design, build, and maintain backend services and APIs used for AI evaluation and data workflows
- Implement tooling to automate model evaluation, annotation pipelines, and quality metrics
- Write clean, testable code in Python and TypeScript across backend and tooling stacks
- Collaborate with data scientists, QA, and product teams to convert evaluation needs into reliable systems
- Optimize performance and scalability for high-throughput evaluation workloads
- Contribute to CI/CD, observability, and reproducible deployment processes
- Mentor peers and participate in architectural discussions to improve engineering practices
What We’re Looking For
- Strong proficiency in Python and practical experience with TypeScript
- Proven track record building production-grade backend systems or developer tooling
- Experience working with APIs, data pipelines, and automated evaluation/test frameworks
- Solid engineering fundamentals: testing, debugging, version control, and CI/CD
- Excellent collaboration and communication skills; comfortable working with cross-functional teams
- Commitment to delivering reliable, maintainable systems that support ML/AI evaluation
Nice-to-haves
- Experience with machine learning evaluation, model validation, or data annotation systems
- Familiarity with cloud platforms (AWS/GCP/Azure), containers (Docker), and orchestration (Kubernetes)
- Background in observability (metrics, logging, tracing) and production incident response
- Knowledge of SQL and working with large datasets
Role details
- Location: Boardman, OR, United States
- Employment type: Contract (Associate)
- This role is posted by Careertakes on behalf of a confidential client in the Information Technology & Services industry.
- Applicants must be eligible to work in the United States.
Equal Opportunity & Hiring Transparency
Careertakes and our client are Equal Opportunity Employers committed to building a diverse and inclusive workforce. We prohibit discrimination or harassment of any kind. To support a fair and efficient hiring process, AI tools may be used to assist with application review or resume screening. These tools do not replace human decision-making. Final hiring decisions are made by people.
If you have questions about how your data is used, please contact us directly.
Negotiate a higher salary! Check the salary ranges for this job type in your area.
View My Salary Range