容器編排之戰-Container Orchestration Wars

05-07

本文轉載自我的blog： http://martinliu.cn

雖然容器編排器之戰已經被K8s終結，可是這篇文章對架構分層的整理和分析還是值得架構師們借鑒的

K8s在2017年底為這場速戰速決的站點話上了句號，結果是所有競爭對手都選擇了增加對K8s的支持。在各自的編排器框架里內置了K8s。本文是根據Mesosphere公司的大拿Karl KARL ISENBERG在各種大會上分享過多次的一份演講稿改版的。

KARL ISENBERG 是誰？

所在公司?

Mesosphere（當前）
Pivotal

做過的產品?

DC/OS
Kubernetes
CloudFoundry
BOSH

聯絡信息：

http://github.com/karlkfi
http://twitter.com/karlkfi
http://linkedin.com/in/karlkfi
http://karl.isenberg.us

基礎架構的進化

傳統的應用架構在逐漸向下面兩種架構演變。

可擴展的單體應用架構

關鍵詞：

Online 基於互聯網
Latency Routed 用戶訪問基於延遲路由
Multi-Region 多區部署
Load Balanced 負載均衡接入
Multi-Zone 多個Zone
Replicated 應用實例多副本
Auto-Scaled 容量自動化收縮
Data Replication 區內數據多副本
Data Synchronization 跨區數據同步

可擴展的微服務架構

上圖出處：Wheel of Doom ，來自 A Journey into Microservices by Hailo

應用+裸金屬伺服器

APPLICATION PROVISIONING ON BARE METAL

應用+IaaS

APPLICATION PROVISIONING ON VIRTUAL INFRASTRUCTURE PLATFORM (IaaS)

「Ultimately, utility cloud providers have exposed how difficult it is to properly operate data centers — and reminded all of us that the ability to expertly operate infrastructure is what really fuels the consumption of open source infrastructure.」 –Brian Stein (Rackspace VP - 2017)

應用+PaaS/aPaaS+IaaS

APPLICATION PLATFORM (PaaS / aPaaS) ON INFRASTRUCTURE PLATFORM (IaaS)

「The goal of Cloud Foundry is to put more of the controls back in the hands of developers so they can self-provision, so there aren』t a lot of roadblocks in their way. But it gives a lot of guardrails.」 – Chip Childers (Cloud Foundry Foundation CTO - 2017)

容器編排器+IaaS

CONTAINER ORCHESTRATION ON INFRASTRUCTURE PLATFORM (IaaS)

「…traditional 「PaaS」 roles have now been taken over by containers… The piece that is left for PaaS is the part that was always the most important part of PaaS in the first place, and that』s the opinionated developer experience.」 –Brendan Burns (Kubernetes Cofounder - 2017)

CaaS+IaaS

CONTAINER PLATFORM (CaaS) ON INFRASTRUCTURE PLATFORM (IaaS)

CaaS+裸金屬伺服器

CONTAINER PLATFORM (CaaS) ON BARE METAL

FaaS+IaaS

FUNCTION PLATFORM (FaaS) ON INFRASTRUCTURE PLATFORM (IaaS)

「If your PaaS can efficiently start instances in 20ms that run for half a second, then call it serverless.」 –Adrian Cockcroft-(AWS VP - 2016)

FaaS+CaaS

FUNCTION PLATFORM (FaaS) ON CONTAINER PLATFORM (CaaS)

FaaS+CaaS+IaaS

FUNCTION PLATFORM (FaaS) ON CONTAINER PLATFORM (CaaS) ON INFRASTRUCTURE PLATFORM (IaaS)

平台頻譜 - PLATFORM SPECTRUM

從左到右，資源的抽象程度不斷提高；最左側的彈性最高，最右側的速率最高。

下圖是不同類型里的廠商和軟體。

容器平台層次

容器編排器的層次如下：

User workloads 用戶工作負載
Distributed container management 分散式容器管理
Local container management 本地容器管理
Container agnostic infrastructure 容器無關性基礎架構

容器平台的層次如下：

CONTAINER PLATFORM

User workloads 用戶工作負載
System management & service enablement 系統管理和服務管理
Distributed container management 分散式容器管理
Local container management 本地容器管理
Container aware infrastructure 容器感知的基礎架構
Container agnostic infrastructure 容器無關的基礎架構

分散式操作系統的層次如下：

容器平台功能點

CONTAINER PLATFORM CAPABILITIES

運行態的能力

1 容器

Resource Isolation
Resource Constraints
Process Tree
Environment Isolation
Shell / Exec

2 鏡像

Build
Layers
Download
Cache
Publish
Prune

3 網路

Container
Bridge
Host
Virtual
Overlay
Remote
User-defined
Port Mapping

4 數據卷 - Ephemeral - Host - Backup / Restore - Copy In / Out - Shared

編排器的能力

調度

Placement
Replication/Scaling
Readiness Checking
Resurrection
Rescheduling
Rolling Updates
Collocation
Daemons
Cron Jobs

資源管理

Memory
CPU
GPU
Ephemeral Volumes
Remote Persistent Volumes
Local Persistent Volumes
Ports
IPs (per container)

服務管理

Labels
Groups/Namespaces
Dependencies
Load Balancing (L7)
VIPs (L3/L4 LB)
DNS
DNS Proxy
Secrets
Config Mgmt

運維方面的能力

管理

GUI
CLI
Metrics API
Logs API
Events API
Rolling Upgrades
Backups & Restores

MULTI-INFRASTRUCTURE

Multi-cloud
Multi-zone
Multi-region
Hybrid-cloud
Federation

系統服務

Auto-Scaling
Package Management
Service Catalog
Service Brokers
Admin Proxy
API Gateway

平台的能力

容器網路

Overlay
Routing
Network Address Translation (NAT)
Firewalls
Access Control Lists
Quality of Service

容器存儲

Local Volumes
Remote Volumes
Block Storage
File System Storage
Object Storage

平台資料庫

Lock Service
Key-Value Database
Relational Database
Time Series Database

安全

User Accounts
Service Accounts
System/User Space
E2E Encryption
Non-root User Workloads
Audit Logging
Public Key Infrastructure
Certifications

多租戶

User Groups
Permissions
RBAC
ABAC
Resource Sharing

FIFO
Fair
Quotas

Branding
Quality of Service

非功能需求

穩定性

Performance
Responsiveness
Efficiency

可用性

Fault Tolerance
Robustness,
Reliability,
Resilience,
Disaster Recovery

靈活性

Format Support,
Interoperability,
Extensibility,
Container Runtimes

可用度

Familiarity,
Maintainability,
Compatibility,
Debuggability

可移植性

Host OS,
Cloud,
Bare-Metal,
Hybrid

安全性

Encryption Quality,
Vulnerability Process,
Fast Patching,
Backporting

容器平台對比

市場里的主要技術廠商如下。

其它值得考慮的廠商如下。

下面的能力對比的時間點是 06/2017，這個時候K8s是否能勝出還是個懸念。

調度

圖示說明：

綠勾：包含此能力
橫杠：New/External/Partial/Experimental

資源管理

服務管理

如何選擇

第一陣營：重量級

KUBERNETES

Huge community
Solid API
Some assembly required
Multitude of vendors/installers

OPENSHIFT

Application platform based on Kubernetes
Always trailing Kubernetes releases
No assembly required
Open core, enterprise platform

DC/OS

Runs native applications (non-Docker)
Specialized in data services
Ambitious scope (on-prem AWS)
No assembly required
Open core, enterprise platform

DOCKER

Huge community
Fast moving API
Integrated orchestration and runtime
Recent pivot from runtime to orchestration
Open core, enterprise platform

第二陣營：輕量級

EC2 CONTAINER SERVICE (ECS)

Hosted-only solution
Tight integration with AWS services
Closed platform

RANCHER CATTLE

Gateway to Kubernetes, Mesos, and Docker
Open platform, enterprise support

NOMAD

Provisioner with orchestration features
Runs native applications (non-Docker)
Tight integration with Vault and Consul
Some assembly required
Open platform, enterprise support

KONTENA

Simple to set up
No assembly required
Open core, enterprise platform

Karl個人的考察點？

Which is more important to you: velocity or flexibility?
Do you want an opinionated application platform?
Do you need to support Big Data initiatives and pipelines?
Do you want a hosted solution?
Are you willing to build out your own integrations?
Do you need on-prem & hybrid capabilities?
Do you want to avoid infrastructure lock-in?
Are you already invested in a specific infrastructure?
Are you already invested in a specific operating system?
Do you need **federation and multi-region **support?
Do you want multi-tenancy or is multi-instance good enough?
How important are seamless automated rolling upgrades?
How many nines do your customers need?
How important is reverse compatibility & API stability?
Do you need to support non-Docker workloads?