Knative: The missing serving layer for Kubernetes Kubernetes is a great platform to deploy and run microservices –Everyone Kubernetes is a generic platform to run any workload, and "services" deserve better networking, rollout and monitoring capabilities from the infrastructure they run on -me 😇 Kubernetes: the good parts ✔ 1. A "declarative" and "goal-state driven" API. 2. Manage a large set of machines (i.e. a cluster) 3. APIs to run container workloads on those machines (Pod, Deployment, StatefulSet..) 4. Extensibility to define your own APIs (CRDs) and controllers around them to actuate resources. Pod smallest deployment unit (1..N containers) ReplicaSet a scalable set of identical stateless Pods Deployment ReplicaSet but with revisions and rolling updates StatefulSets Pods pinned to particular nodes Job Run a Pod to completion CronJob Run a Job periodically ● serves an API or web page ● stateless replicas ● load balancing ● autoscaling ● rollouts (blue/green) ● rollbacks Microservices noun. service, but smaller. usually a twelve-factor app. ● service discovery ● secure transport (TLS) ● request metrics ● graceful termination ● shield from spikes/DoS ● concurrency limits ● ... ● serves an API or web page ● stateless replicas ● load balancing ● autoscaling ● rollouts (blue/green) ● rollbacks Where Kubernetes falls short ● service discovery ● secure transport (TLS) ● request metrics ● graceful termination ● shield from spikes/DoS ● concurrency limits ● ... DIY microservice HTTP request HTTP response HTTP request HTTP response client TCP socket microservice client Kubernetes has no notion of application-layer (L7) requests (HTTP, gRPC, ...). ● Per-connection ● Causing uneven distribution ○ single client establishing too many connections ● Naturally "sticky sessions" ○ a client routed to the same Pod even if degraded or faulty Where Kubernetes falls short Load balancing Pod Pod Pod Pod Pod ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling 0.4 cpu Autoscaling target: 1.0 CPU Pod (1.5 CPU) ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu Pod (1.5 CPU) ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu Pod ● Based only on CPU and memory ● Delayed metrics collection ○ cannot easily handle spiky traffic patterns ○ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu Pod ● No support for highly spiky traffic patterns. ● Need a proxy or gateway to “front” the requests and “buffer” them. ● No "max N requests per container" Where Kubernetes falls short Meat shielding Concurrency Controls Rapid Autoscaling Meat shield Pod Pod Pod Pod Pod Pod ● Can't split traffic per-request, e.g. ○ 95% v1 ○ 5% v2 ● Need to implement blue/green rollouts yourself. ○ Deployment API gives some options for rolling updates, but not quite blue/green. Where Kubernetes falls short Rollouts Blue/green deployments Pod v1 Pod v2 95% 5% Pod ● Unused replicas keep consuming resources. ● Hard to have high utilization, because we almost always overprovision in Kubernetes. Where Kubernetes falls short Scale to zero Pod Pod