You will own the end-to-end cloud infrastructure for a high-performance compression API. This role involves building global, low-latency GPU ML inference systems that sit directly in the critical path of customer traffic. As a core member of a 5-person team, you will drive reliability, scalability, and cost-efficiency for a revolutionary AI product.
Member of Technical Staff, Infrastructure at The Token Company
You'll own the cloud systems for high-performance GPU ML inference, building the low-latency API that helps enterprises slash LLM costs. With a tiny team of 5 and massive traction from investors like the founders of Dropbox and Slack, this is a high-ownership role offering a $120K-$250K salary, significant equity, and incredible perks including housing and food in San Francisco. Help define the future of efficient AI infrastructure.
Want to apply for this role?
Jack finds you jobs at companies like The Token Company. Talk to Jack to get considered for roles that fit what you're great at.
Location
San Francisco, United States
Compensation
$120k-$250k + Equity
Company
The Token Company
Role overview
The Token Company builds LLM input compression middleware using a fast machine learning model that removes useless tokens from prompts via a drop-in API. The compression model preprocesses LLM inputs by removing least-significant tokens to cut token counts, latency, and inference costs. It compresses 100k tokens in under 100ms and typically reduces input tokens by approximately 66% while improving model accuracy and enabling larger context windows. The company’s bear-1 and bear-1.1 compression models preserve semantic intent while stripping noise, with benchmarks showing accuracy gains (e.g., +2.7 percentage points on financial QA with up to 20% fewer tokens) and up to 37% faster end-to-end latency. Integration takes minutes via a simple API.
What you will do
- Architect and manage global, high-throughput GPU inference infrastructure using AWS, Terraform, and Docker to ensure maximum performance and uptime.
- Own the entire infrastructure stack from initial deployment and CI/CD pipelines to long-term reliability and cost-efficiency research.
- Collaborate closely with research and product teams to integrate and scale compression models into a high-traffic production API.
Who this is a fit for
- Demonstrated experience building and operating production-grade cloud infrastructure at scale using tools like AWS and Terraform.
- Strong background in managing containerized environments with Docker and establishing robust CI/CD pipelines for high-reliability systems.
- Proven ability to take end-to-end ownership of an infrastructure stack, ideally within a fast-paced startup or high-growth engineering environment.
Why this role is remarkable
- Join an elite 5-person team backed by $12M from top-tier investors like First Round Capital and founders of Dropbox, Slack, and Supercell.
- Solve critical infrastructure challenges for LLM efficiency, building the low-latency systems that enable enterprises to scale AI cost-effectively.
- Benefit from an unparalleled San Francisco living experience with provided housing, food, laundry, and cleaning services, plus full visa sponsorship.
How Jack & Jill work together
I get to know what you’re great at, then find roles you’d never find yourself.Ok, I'll go first. I'm Jack, an AI that gets to know you on a quick call, learning what you're great at and what you want from your career. Then I help you land your dream job by finding unmissable opportunities as they come up, supporting you with applications, interview prep, and moral support.
I recruit from Jack’s network and make the intro when I spot a great match.And I'm Jill, an AI Recruiter who talks to companies to understand who they're looking to hire. Then I recruit from Jack's network, making an introduction when I spot an excellent candidate.
Jack gets to know what you're great at and what you want next, then searches 15 million jobs daily and helps you discover roles at companies like this.
Meet Jack
What happens next?
Jack’s an AI agent for job searching and career coaching. He works for you.
Jill is the AI recruiter working for the company. She recruits from Jack’s network.
If your profile’s a match and The Token Company wants to meet, Jill will make the intro. In the meantime, Jack will send you excellent alternatives.