Principal Machine Learning Engineer
Help build the AI infrastructure behind HubSpot's next generation of agentic product experiences — remotely from the US.
What This Principal Machine Learning Engineer Role Is About
HubSpot is looking for a Principal Machine Learning Engineer to help build the foundational AI infrastructure behind its next generation of agentic product experiences.
This isn't described as a typical feature-team position. The role focuses on the infrastructure and platforms that other product and engineering teams can build on — including agent runtimes, evaluation systems, quality signals, model optimization, fine-tuning and model routing.
For principal-level engineering roles, don't rely only on a list of technologies on your CV. Show the scale of the systems you've built, the engineering decisions you made, and the measurable results.
What You'll Work On
According to the job description, this position will work across several foundational areas of HubSpot's AI platform.
1. Agent Runtime & Evaluation Infrastructure
You'll help build infrastructure for agent runtime, evaluation, quality measurement and model optimization. The goal is to create reusable systems rather than isolated solutions for individual products.
2. Better AI Evaluation
You'll create tooling that helps teams understand where their evaluations are strong, where coverage is missing and where users are asking questions that the system isn't prepared to handle.
3. Quality & User Signals
The role includes designing signal pipelines that can surface user frustration, agent failure states and quality problems before those issues become larger product-quality concerns.
4. AI Benchmarking
Another focus is helping define a HubSpot-specific AI benchmark for evaluating model performance against real HubSpot workloads.
5. Model Evaluation & Routing
You'll build systems that can evaluate new models across agents, helping teams improve product quality, manage costs and support future model-routing strategies.
6. Fine-Tuning & Optimization
The role also involves making fine-tuning and task-specific optimization more repeatable for HubSpot's AI use cases.
If you've built ML platforms, evaluation pipelines, model-serving systems, feedback loops or experimentation infrastructure, make those achievements easy to find near the top of your CV.
What HubSpot Is Looking For
This is a senior technical leadership position. The description emphasizes both deep technical experience and the ability to influence teams through technical credibility and clear judgment.
- Deep experience with production ML, LLM or AI infrastructure.
- Experience building systems at scale where reliability, latency, quality and cost matter.
- A platform-oriented mindset rather than a one-off solution approach.
- Ability to work across infrastructure, product, backend and ML teams.
- Experience with evaluation frameworks, model serving, fine-tuning, signal extraction, experimentation, feedback loops or model optimization.
- Comfort working on ambiguous foundational problems where the solution still needs to be developed.
- Ability to influence senior engineers and leaders through technical expertise and sound judgment.
Prepare 3–5 detailed stories about systems you've designed. Be ready to explain the architecture, trade-offs, reliability considerations, bottlenecks, cost implications and what you would change today.
Why This Role Matters
HubSpot describes this as an opportunity to help define how AI infrastructure works across the company.
Rather than focusing on one isolated AI feature, the position is centered on infrastructure that can support multiple AI agents and product experiences.
The job description specifically connects the role to Breeze Engine, which powers Breeze Assistant and other AI-powered product experiences, as well as Aviator, HubSpot's internal framework for building and running agents.
If you're targeting principal-level opportunities, look beyond the job title. Roles that let you solve platform-wide problems can also give you opportunities to demonstrate architecture, technical leadership and cross-team influence.
GabbyWall does not employ candidates or directly give out jobs. We share job opportunities from available sources to make them easier for job seekers to discover.
Salary, Compensation & Benefits
The job listing states that the cash compensation includes base salary, on-target commission for employees in eligible roles and annual bonus targets under HubSpot's bonus plan for eligible roles.
The listing also states that some roles may be eligible to participate in HubSpot's equity plan through restricted stock units (RSUs), while some positions may also be eligible for overtime pay.
HubSpot says individual compensation packages are tailored according to skills, experience, qualifications and other job-related factors.
Benefits are also described as an important part of the overall compensation package. The listing directs candidates to HubSpot's benefits and perks page for additional information.
Candidates can also review HubSpot's compensation philosophy before applying.
Compensation ranges are only one part of a senior-level opportunity. Before applying, review the complete role description and make sure your CV clearly communicates the scope and impact of your previous work.
Who May Be a Good Fit for This Opportunity?
Based on the supplied requirements, this opportunity may be relevant to experienced ML and AI engineers who have moved beyond individual models and features into larger production systems.
Your application may be particularly relevant if your background includes areas such as:
- Production machine learning infrastructure
- LLM or generative AI systems
- Model serving and inference
- Evaluation frameworks and AI benchmarks
- Fine-tuning pipelines
- Experimentation and feedback systems
- Model optimization
- AI platform engineering
- Distributed systems and high-scale infrastructure
- Cross-functional technical leadership
If you meet most of the core requirements but don't match every single phrase in the posting, review the complete description and assess your actual experience before deciding whether to apply.
Ready to Take the Next Step?
If this Principal Machine Learning Engineer opportunity matches your experience, review the official listing carefully and submit your application directly through HubSpot.
APPLY AT HUBSPOT →Application link opens the official HubSpot careers page.
📢 Know Someone Looking for a Remote Job?
Share this opportunity with someone who may meet the requirements. And if you're actively job hunting, join the GabbyWall WhatsApp community so you can discover new opportunities as they are published.
JOIN JOBS COMMUNITY👋 Need More Jobs?
Join the GabbyWall WhatsApp community for jobs, tech jobs, remote jobs and international opportunities.
Join Jobs Community Chat With GabbyWall
0 Comments