Small Models, Big Agents: Fine-Tuning and Serving SLMs on Kubernetes
LLM, Kubernetes, KEDA, Karpenter, AI
Guillermo Ruiz
Small models are not a compromise. They are often the right tool. In this session we go end to end: fine tuning SLMs and a 70B model on Kubernetes, serving with llama.cpp, and autoscaling with KEDA and Karpenter. You will learn when to use small vs large models, how to optimize cost and performance, and how to run agentic workloads without overprovisioning.
canela Workshop
€50
Max. Attendees: 50
Free registration for Early Camarón & Bokeron ticket holders and a discount
for all others
Key Takeaways
Match the model to the task: which agent steps suit small CPU models, and which genuinely need a larger LLM.
Deploy efficient SLM inference on CPUs with llama.cpp.
Build an agent whose routing, tool selection, argument extraction, and self-critique run on small CPU models, handing off to a larger model for open-ended reasoning when that's the right call.
Train task-specific CPU specialists (classification, scoring, extraction, embedding) for each decision in the pipeline.
Give agents safe, read-only tools so answers stay grounded in live cluster state.
Right-size infrastructure with KEDA and Karpenter to balance cost and performance.
Apply "right tool for the right task" thinking to production agentic AI.
Target Audience
Platform Engineers
Kubernetes Engineers
MLOps Engineers
AI/ML Engineers
Cloud Architects
DevOps Engineers
Technical Leaders responsible for AI platforms
Requirements for Attendees
Technical Requirements
Basic understanding of Kubernetes concepts.
Familiarity with containers and cloud-native applications.
General knowledge of Generative AI, LLMs, and AI agents.
