Job Description

Inference Software Engineer

About Us

We are at the forefront of AI innovation, driving scalable and efficient solutions for enterprise AI workloads. The Inference team focuses on expanding the capabilities of deployable GPU architectures, optimizing performance, and building tools for efficient operations. Our work currently targets inference, with potential expansion into fine-tuning in the future.

Responsibilities

As an Inference Software Engineer, you will:

• Design, develop, and optimize GPU kernels from scratch and fine-tune existing kernels for both NVIDIA and non-NVIDIA platforms.
• Leverage CUDA and NCCL for distributed networking on NVIDIA GPUs and extend solutions to other architectures.
• Write and maintain code to distribute machine learning workloads across distributed systems.
• Contribute at lower levels (e.g., kernel or network programming).
• Contribute at higher levels (e.g., Kubernetes, operators, and ML frameworks built on Kubernetes).
• Collaborate with cross-functional teams to expand the footprint of deployable GPU architectures.
• Optimize inference pipelines for performance and scalability.
• Develop tools and workflows for efficient operation of GPU-based inference systems, with a future focus on supporting fine-tuning workloads.

Qualifications

We’re looking for someone with:

• Expertise in GPU kernel programming, including experience in CUDA and familiarity with NCCL for distributed networking.
• Proficiency in programming for distributed systems, with a strong foundation in building scalable ML solutions.
• Experience working with GPU architectures beyond NVIDIA.
• A solid understanding of systems engineering, with hands-on experience in one or more of the following areas:
• Kernel or network-level programming for distributed systems.
• Higher-level tools like Kubernetes, ML operators, or frameworks built on Kubernetes.
• Proficiency in programming languages such as C++, Python, or similar.
• Familiarity with ML frameworks like TensorFlow, PyTorch, or ONNX (a plus).
• A Bachelor’s, Master’s, or Ph.D. in Computer Science, Electrical Engineering, or a related field (or equivalent experience).

Preferred Skills

• Experience optimizing inference workloads across diverse GPU architectures.
• Hands-on knowledge of distributed networking tools and protocols, especially in ML contexts.
• Familiarity with quantization, pruning, or other model optimization techniques.
• Experience with profiling tools such as NVIDIA Nsight or AMD ROCm tools.

Why Join Us?

• Tackle cutting-edge challenges in GPU programming, distributed systems, and ML optimization.
• Collaborate with a dynamic, innovative team driving the future of enterprise AI.
• Enjoy competitive compensation and benefits, with significant opportunities for impact and growth.

Job Tags

Similar Jobs