About Careertakes

👉 Important disclosure: Careertakes is a third-party recruiting platform supporting this hiring process. If selected, you will be employed directly by our client, Technology, Information and Internet.

Applicants for this role may also receive access to additional matched opportunities through the Careertakes platform.

---

About the Role

We are recruiting on behalf of a confidential client building a high-performance AI inference platform. As a Junior Software Engineer (IC1/IC2) on the Inference team, you will own a model-serving production service end-to-end: shipping features, improving observability and operational excellence, and collaborating with teams across runtime, scheduling, and networking to move key SLIs.

---

What You’ll Do

• Own one model-serving service end-to-end: design, implementation, tests, deployment, and operations.
• Add and tune observability: dashboards, alerts, traces; automate runbook steps to reduce toil.
• Improve latency and throughput via micro-batching, concurrency tuning, and caching strategies.
• Participate in on-call rotation and lead post-incident action items for your service.
• Review peers’ PRs and contribute design feedback during team reviews.

---

Qualifications

• 0–3 years of software engineering experience (or equivalent) building cloud/distributed services in production.
• Proficiency in Python or Go and familiarity with gRPC/HTTP APIs and containerized deployments.
• Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
• Experience with metrics and alerting systems (Prometheus, Grafana, OpenTelemetry) and structured logging.
• Basic understanding of GPU inference concepts (key-value caching, mixed precision, streaming tokens).

---

Nice to Have

• Experience with inference frameworks such as Triton, vLLM, TensorRT-LLM, Ray Serve, or TorchServe.
• Hands-on performance profiling and benchmarking experience (e.g., wrk, vegeta, perf, Nsight).
• Familiarity with reliability practices such as defining SLI/SLOs and conducting postmortems.

---

Compensation

• Base salary range: $120,000 - $160,000 (USD). The actual starting salary will be determined based on job-related knowledge, skills, experience, and market location. In addition to base salary, total rewards may include discretionary bonus, equity awards, and benefits (eligibility rules apply).

---

Why Join

• Work on an industry-defining inference platform that enables teams to deploy generative AI and real-time applications at scale.
• Learn from experienced engineers solving low-latency, high-throughput model-serving problems.
• Join a remote-friendly culture that values ownership, curiosity, and technical excellence.

---

Benefits (highlights)

• 100% employer-paid medical, dental, and vision
• 401(k) with employer match
• Flexible PTO and remote-friendly policies
• Access to professional development, tuition reimbursement, and mental wellness resources

---

Export Control & Eligibility

This position requires access to export-controlled information. To comply with applicable U.S. Government export regulations, applicants must be either:

• a U.S. person (U.S. citizen, U.S. national, lawful permanent resident, refugee, or asylee), or
• eligible to access the export-controlled information without a required export authorization, or
• eligible and reasonably likely to obtain the required export authorization from the applicable U.S. government agency.

The confidential client may, for legitimate business reasons, decline to pursue any export licensing process.

---

Equal Opportunity & Hiring Transparency

Careertakes and our client are Equal Opportunity Employers committed to building a diverse and inclusive workforce. We prohibit discrimination or harassment of any kind. To support a fair and efficient hiring process, AI tools may be used to assist with application review or resume screening. These tools do not replace human decision-making. Final hiring decisions are made by people.

If you have questions about how your data is used, please contact us directly.