Member of Technical Staff- Inference Job at Acceler8 Talent, Palo Alto, CA

YzFxT3VZamc0cnBxNnNIRG9EK0MxR0ZodGc9PQ==
  • Acceler8 Talent
  • Palo Alto, CA

Job Description

Inference Software Engineer

About Us

We are at the forefront of AI innovation, driving scalable and efficient solutions for enterprise AI workloads. The Inference team focuses on expanding the capabilities of deployable GPU architectures, optimizing performance, and building tools for efficient operations. Our work currently targets inference, with potential expansion into fine-tuning in the future.

Responsibilities

As an Inference Software Engineer, you will:

  • Design, develop, and optimize GPU kernels from scratch and fine-tune existing kernels for both NVIDIA and non-NVIDIA platforms.
  • Leverage CUDA and NCCL for distributed networking on NVIDIA GPUs and extend solutions to other architectures.
  • Write and maintain code to distribute machine learning workloads across distributed systems.
  • Contribute at lower levels (e.g., kernel or network programming).
  • Contribute at higher levels (e.g., Kubernetes, operators, and ML frameworks built on Kubernetes).
  • Collaborate with cross-functional teams to expand the footprint of deployable GPU architectures.
  • Optimize inference pipelines for performance and scalability.
  • Develop tools and workflows for efficient operation of GPU-based inference systems, with a future focus on supporting fine-tuning workloads.

Qualifications

We’re looking for someone with:

  • Expertise in GPU kernel programming, including experience in CUDA and familiarity with NCCL for distributed networking.
  • Proficiency in programming for distributed systems, with a strong foundation in building scalable ML solutions.
  • Experience working with GPU architectures beyond NVIDIA.
  • A solid understanding of systems engineering, with hands-on experience in one or more of the following areas:
  • Kernel or network-level programming for distributed systems.
  • Higher-level tools like Kubernetes, ML operators, or frameworks built on Kubernetes.
  • Proficiency in programming languages such as C++, Python, or similar.
  • Familiarity with ML frameworks like TensorFlow, PyTorch, or ONNX (a plus).
  • A Bachelor’s, Master’s, or Ph.D. in Computer Science, Electrical Engineering, or a related field (or equivalent experience).

Preferred Skills

  • Experience optimizing inference workloads across diverse GPU architectures.
  • Hands-on knowledge of distributed networking tools and protocols, especially in ML contexts.
  • Familiarity with quantization, pruning, or other model optimization techniques.
  • Experience with profiling tools such as NVIDIA Nsight or AMD ROCm tools.

Why Join Us?

  • Tackle cutting-edge challenges in GPU programming, distributed systems, and ML optimization.
  • Collaborate with a dynamic, innovative team driving the future of enterprise AI.
  • Enjoy competitive compensation and benefits, with significant opportunities for impact and growth.

Job Tags

Similar Jobs

Hawthorne Lane

Various Administrative & Executive Assistant Roles across Several Industries including Non-profit and Corporate! Job at Hawthorne Lane

 ...seeking their next step. Our clients offer hybrid, in-person, and remote work models, with roles ranging in responsibilities as well as...  ...positions. Positions that require anywhere from strong internship experience through school to multiple years of industry-specific... 

The Lawt

Administrative Assistant Job at The Lawt

 ...Administrative Assistant (Part-Time) Lakewood, CO A boutique law firm specializing in the representation of local governmentsincluding...  ...seen and empowered. The Lawt is certified by the Womens Business Enterprise National Council (WBENC) as a Womens Business... 

Diamond Constructors Inc.

Class A and B CDL Drivers Job at Diamond Constructors Inc.

 ...Role Description This is a full-time on-site role for Class A and B CDL Drivers at Diamond Constructors Inc. located in the Fayetteville, North...  ...Metropolitan Area. The CDL Drivers will be responsible for truck driving, unloading, and ensuring compliance with DOT... 

Promenade Group

Florist Job at Promenade Group

Come join our team at Sofi La Fleur, as we are currently seeking experienced floral designers to assist. If you have a minimum of 3 years of Retail Floral design experience, we would love to hear from you! As a floral designer, you will be responsible for processing flowers...

Tonic3

Field Sales Representative - Digital Solutions for SMBs Job at Tonic3

 ...small and medium-sized businesses modernize how they operatefrom automating workflows to improving customer engagement. As a Field Sales Rep, youll canvas businesses face-to-face, learning whats working (and whats not) and offering digital solutions that actually...