Software Engineer, Inference (AI Data Engineering)
The Software Engineer, Inference role at Confidential Client involves designing and optimizing large-scale, high-performance AI inference systems to serve internal models critical to Confidential Client's engineering goals. The engineer will develop scalable distributed infrastructure, optimize GPU-based inference workloads, and ensure reliable, low-latency service delivery, contributing directly to mission-critical applications supporting launch vehicle production and Starlink operations. This position requires collaboration across AI teams and ownership of end-to-end system components within an innovative aerospace environment.
About Careertakes
👉 Important disclosure: Careertakes is a third-party recruiting platform supporting this hiring process. If selected, you will be employed directly by our client, Aviation and Aerospace Component Manufacturing.
Applicants for this role may also receive access to additional matched opportunities through the Careertakes platform.
Overview
Confidential Client is seeking a Software Engineer, Inference (AI Data Engineering) to join its Internal AI Infrastructure team in Palo Alto, CA. You will design and optimize large-scale model-serving systems end-to-end — from distributed infrastructure to low-level GPU and inference optimizations — to deliver reliable, high-throughput inference for mission-critical applications. Aerospace experience is not required; we value motivated engineers who collaborate respectfully and solve hard problems.
What You’ll Do
- Develop highly reliable, high-throughput inference systems that serve internal AI models
- Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache, continuous batching)
- Optimize latency and throughput under production workloads (GPU kernel work, quantization, speculative decoding, etc.)
- Build high-concurrency serving systems with low tail latency, high availability, and strong observability
- Own end-to-end components: request routing, SDKs, rate limiting, and efficient scaling
- Benchmark, fine-tune, and accelerate inference engines (e.g., SGLang, vLLM, TensorRT-LLM)
- Develop tools for tracing, replaying, and resolving issues across the stack — from orchestration to GPU kernels
- Create CI/CD for endpoint deployment, image publishing, and inference engine updates
- Collaborate across AI teams to integrate inference capabilities into broader systems and workflows
Basic Qualifications
- Bachelor’s degree in Computer Science, Engineering, Math, or a scientific discipline; OR 2+ years of professional software experience in lieu of a degree
- Experience designing, implementing, and maintaining reliable, horizontally scalable distributed systems
- 1+ years of production backend or full-stack development experience
- 1+ years of experience with Rust or C++
Preferred Skills & Experience
- Experience with LLM inference engines and serving frameworks (SGLang, vLLM, Triton, TensorRT-LLM)
- Deep low-level systems programming and optimizations: GPU kernels, code generation, batching, caching, parallelism, quantization, speculative decoding
- Experience with large-scale, high-concurrency production serving systems
- Knowledge of service observability and reliability best practices
- Experience operating databases such as PostgreSQL, ClickHouse, or MongoDB
- Experience with agent SDKs and agent orchestration frameworks
- Experience with Docker, Kubernetes, and containerized applications
- Expert knowledge of gRPC (unary, streaming, REST mapping)
- Programming experience in Python, Go, or similar languages
- Familiarity with version control, CI/CD, build systems, and monitoring
- Expertise in profiling and improving application performance
Additional Requirements
- You may be asked to work extended hours or weekends depending on launch cadence and platform demands
- This role requires you to be onsite in Palo Alto, CA. Remote or hybrid work will not be considered
- This role will report through Confidential Client Internal AI Infrastructure and will support both inference and training workloads
Compensation & Benefits
- Pay Range:
- Level 1: $135,000 - $175,000 (base)
- Level 2: $155,000 - $210,000 (base)
- Your actual level and base salary will be determined on a case-by-case basis and may vary based on job-related knowledge, skills, education, and experience
- Base salary is one part of the total rewards package; you may also be eligible for long-term incentives (equity or cash), discretionary bonuses, and participation in employee stock purchase plans
- Comprehensive medical, vision, and dental coverage; 401(k); short- and long-term disability; life insurance; paid parental leave; paid vacation (e.g., ~3 weeks) and 10+ paid holidays; paid sick leave per company policy
ITAR & Authorization Requirements
To comply with U.S. Government export regulations, applicants must be one of the following:
- U.S. citizen or national, or
- U.S. lawful permanent resident (green card holder), or
- Refugee under 8 U.S.C. 1157, or
- Asylee under 8 U.S.C. 1158, or
- Eligible to obtain required authorizations from the U.S. Department of State
Location
Onsite — Palo Alto, CA (no remote/hybrid options)
Equal Opportunity & Hiring Transparency
Careertakes and our client are Equal Opportunity Employers committed to building a diverse and inclusive workforce. We prohibit discrimination or harassment of any kind. To support a fair and efficient hiring process, AI tools may be used to assist with application review or resume screening. These tools do not replace human decision-making. Final hiring decisions are made by people.
If you have questions about how your data is used, please contact us directly.
Negotiate a higher salary! Check the salary ranges for this job type in your area.
View My Salary Range