Kubebuilder 介绍

Kubebuilder是由kubernetes-sigs维护的，基于 k8s 控制器运行时封装的 k8s Operator 主流开发工具

介绍

Kubebuilder 的核心组件具有3个职责：

负责运行所有的 Controllers
初始化共享 caches，包含 listAndWatch 功能
初始化 clients 用于与 kubernetes ApiServer 通信

一些关键字：

GV: GroupVersion
GVK: GroupVersionKind
GVR: GroupVersionResource
API Group：是相关API功能的集合，每个 Group 拥有一或多个 Versions，用于接口的演进
Kinds: 每个GV都包含的多个API类型
Resource 是 Kind 的对象标识（resource type）
Finalizer：在一般情况下，如果资源被删除之后，我们虽然能够被触发删除事件，但是这个时候从Cache里面无法读取任何被删除对象的信息，这样一来导致很多垃圾清理工作因为信息不足无法进行，K8s的Finalizer字段用于处理这种情况

开发环境

docker-ce 20.10.+
go
- v1.15+ (kubebuilder v3.0 < v3.1)
- v1.16+ (kubebuilder v3.1 < v3.3)
- v1.17+ (kubebuilder v3.3+)
kubectl 1.22.0+
kubernetes v1.22.0+ cluster，参考：Kubernetes 安装部署

安装

Mac/Linux

curl -L -o kubebuilder "https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)"
chmod +x kubebuilder && mv kubebuilder /usr/local/bin/

Windows 需要使用源码编译

安装后，使用 kubebuilder -h 查看命令帮助

生成代码

示例的代码见：https://github.com/kbcx/xca-operator

工程初始化

# mkdir -p xca-operator
# cd xca-operator
# $ kubebuilder init --domain kb.cx --project-name xca-operator --fetch-deps false --repo github.com/kbcx/xca-operator --owner xiexianbin
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.12.2
Update dependencies:
$ go mod tidy
Next: define a resource with:
$ kubebuilder create api

init 时默认参数 --plugins=""go.kubebuilder.io/v3 --project-version=3，使用 kubebuilder help init 查看 init 命令详情：

$ kubebuilder help init
Initialize a new project including the following files:
  - a "go.mod" with project dependencies
  - a "PROJECT" file that stores project configuration
  - a "Makefile" with several useful make targets for the project
  - several YAML files for project deployment under the "config" directory
  - a "main.go" file that creates the manager that will run the project controllers
  ...

创建 API，生成 CRD 和 Controller

# kubebuilder create api -h
# kubebuilder create api --group xca --version v1alpha1 --kind Xtls
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1alpha1/xtls_types.go
controllers/xtls_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin
test -s /Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin/controller-gen || GOBIN=/Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.9.2
/Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifests

说明：

GVK
- group 组
- version 版本
- kind 自定义资源类型
以上参数组成自定义 yaml 的 apiVersion 和 kind
如果需要在 webapp CRUD 时进行合法性检查，需生成 webhook:

# kubebuilder create webhook --group xca --version v1alpha1 --kind Xtls --defaulting --programmatic-validation
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1alpha1/xtls_webhook.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
/Users/xiexianbin/workspace/code/github.com/kbcx/xca-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new Webhook and generate the manifests with:
$ make manifests

初始化基础的依赖包信息

go mod tidy

project-tree ↕

$ tree ../xca-operator
../xca-operator
├── Dockerfile  # 制作镜像的 Dockerfile
├── LICENSE
├── Makefile
├── PROJECT
├── README.md
├── api
│   └── v1alpha1  # API 资源，Resource 对象的相关信息
│       ├── groupversion_info.go
│       ├── webhook_suite_test.go
│       ├── xtls_types.go  # 自定义CRD类型
│       ├── xtls_webhook.go
│       └── zz_generated.deepcopy.go
├── bin
│   ├── controller-gen
│   ├── kustomize
│   ├── manager
│   └── setup-envtest
├── config
│   ├── certmanager
│   │   ├── certificate.yaml
│   │   ├── kustomization.yaml
│   │   └── kustomizeconfig.yaml
│   ├── crd  # 注册到 K8S 的自定义资源（CRD）的描述文件（YAML）
│   │   ├── bases
│   │   │   └── xca.kb.cx_xtls.yaml
│   │   ├── kustomization.yaml
│   │   ├── kustomizeconfig.yaml
│   │   └── patches
│   │       ├── cainjection_in_xtls.yaml
│   │       └── webhook_in_xtls.yaml
│   ├── default
│   │   ├── kustomization.yaml
│   │   ├── manager_auth_proxy_patch.yaml
│   │   ├── manager_config_patch.yaml
│   │   ├── manager_webhook_patch.yaml
│   │   └── webhookcainjection_patch.yaml
│   ├── manager  # 部署到K8S的 manager YAML
│   │   ├── controller_manager_config.yaml
│   │   ├── kustomization.yaml
│   │   └── manager.yaml
│   ├── prometheus
│   │   ├── kustomization.yaml
│   │   └── monitor.yaml
│   ├── rbac   # 部署到K8S的 rbac YAML
│   │   ├── auth_proxy_client_clusterrole.yaml
│   │   ├── auth_proxy_role.yaml
│   │   ├── auth_proxy_role_binding.yaml
│   │   ├── auth_proxy_service.yaml
│   │   ├── kustomization.yaml
│   │   ├── leader_election_role.yaml
│   │   ├── leader_election_role_binding.yaml
│   │   ├── role.yaml
│   │   ├── role_binding.yaml
│   │   ├── service_account.yaml
│   │   ├── xtls_editor_role.yaml
│   │   └── xtls_viewer_role.yaml
│   ├── samples  # CRD 的样例文件，可以通过它直接 apply 到 k8s 集群中
│   │   └── xca_v1alpha1_xtls.yaml
│   └── webhook
│       ├── kustomization.yaml
│       ├── kustomizeconfig.yaml
│       ├── manifests.yaml
│       └── service.yaml
├── controllers  # Controller 代码
│   ├── suite_test.go
│   └── xtls_controller.go  # 协调自定义资源终态的控制逻辑
├── go.mod
├── go.sum
├── hack
│   └── boilerplate.go.txt
├── main.go  # 函数入口
├── utils
│   ├── utils.go
│   └── utils_test.go
└── vendor
    ...

792 directories, 3800 files

安装

make manifests 重新生成修改后的 CRD 描述
make generate 重新生成代码
开发阶段，可以 本地运行 controller

make run

构建镜像

make docker-build
make docker-build docker-push IMG=xiexianbin/xca-operator:latest

make install 安装 CRDs 到 k8s 环境
- 默认选择 ~/.kube/config 指定的 k8s 环境
- 采用 kubectl get crd 查看自定义资源是否安装到指定的 k8s 环境
make uninstall 卸载 CRDs

验证 CRD

config/samples/xca_v1alpha1_xtls.yaml

apiVersion: xca.kb.cx/v1alpha1
kind: Xtls
metadata:
  name: xtls-sample
spec:
  # TODO(user): Add fields here

创建资源

kubectl apply -f config/samples/xca_v1alpha1_xtls.yaml

查看资源

kubectl get Xca
kubectl get Xca -o yaml

删除资源

kubectl delete -f config/samples/xca_v1alpha1_xtls.yaml

实操日志

xca-demo-log ↕

$ kubectl apply -f config/samples/xca_v1alpha1_xtls.yaml
xtls.xca.kb.cx/xtls-sample created
$ kubectl get xtls
NAME          AGE
xtls-sample   16s
$ kubectl get xtls -o yaml
apiVersion: v1
items:
- apiVersion: xca.kb.cx/v1alpha1
  kind: Xtls
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"xca.kb.cx/v1alpha1","kind":"Xtls","metadata":{"annotations":{},"name":"xtls-sample","namespace":"default"},"spec":{"cn":"xiexianbin.cn","days":365,"domains":["xiexianbin.cn","www.xiexianbin.cn"],"ips":["1.1.1.1","127.0.0.1"],"keyBits":2048}}
    creationTimestamp: "2022-09-03T04:07:58Z"
    generation: 1
    name: xtls-sample
    namespace: default
    resourceVersion: "708034"
    uid: 992ab4e9-ace8-447d-93f7-1ccdf6f648d0
  spec:
    cn: xiexianbin.cn
    days: 365
    domains:
    - xiexianbin.cn
    - www.xiexianbin.cn
    ips:
    - 1.1.1.1
    - 127.0.0.1
    keyBits: 2048
kind: List
metadata:
  resourceVersion: ""
$ kubectl get xtls xtls-sample -o yaml
apiVersion: xca.kb.cx/v1alpha1
kind: Xtls
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"xca.kb.cx/v1alpha1","kind":"Xtls","metadata":{"annotations":{},"name":"xtls-sample","namespace":"default"},"spec":{"cn":"xiexianbin.cn","days":365,"domains":["xiexianbin.cn","www.xiexianbin.cn"],"ips":["1.1.1.1","127.0.0.1"],"keyBits":2048}}
  creationTimestamp: "2022-09-03T04:07:58Z"
  generation: 1
  name: xtls-sample
  namespace: default
  resourceVersion: "708034"
  uid: 992ab4e9-ace8-447d-93f7-1ccdf6f648d0
spec:
  cn: xiexianbin.cn
  days: 365
  domains:
  - xiexianbin.cn
  - www.xiexianbin.cn
  ips:
  - 1.1.1.1
  - 127.0.0.1
  keyBits: 2048
root@k8s-master:~# kubectl get xtls.xca.kb.cx
NAME          AGE
xtls-sample   75m
root@k8s-master:~# kubectl get xtls.xca.kb.cx  -o yaml
apiVersion: v1
items:
- apiVersion: xca.kb.cx/v1alpha1
  kind: Xtls
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"xca.kb.cx/v1alpha1","kind":"Xtls","metadata":{"annotations":{},"name":"xtls-sample","namespace":"default"},"spec":{"cn":"xiexianbin.cn","days":365,"domains":["xiexianbin.cn","www.xiexianbin.cn"],"ips":["1.1.1.1","127.0.0.1"],"keyBits":2048}}
    creationTimestamp: "2022-09-13T13:45:51Z"
    generation: 1
    name: xtls-sample
    namespace: default
    resourceVersion: "748371"
    uid: 8744e36a-5e95-4b44-9926-724c2f53703b
  spec:
    cn: xiexianbin.cn
    days: 365
    domains:
    - xiexianbin.cn
    - www.xiexianbin.cn
    ips:
    - 1.1.1.1
    - 127.0.0.1
    keyBits: 2048
  status:
    active: true
    lastUpdateTime: "2022-09-13T13:45:51Z"
kind: List
metadata:
  resourceVersion: ""
root@k8s-master:~# kubectl get secrets xtls-sample -o yaml
apiVersion: v1
data:
  tls.crt: xxx
  tls.key: xxx
kind: Secret
metadata:
  creationTimestamp: "2022-09-13T13:45:23Z"
  labels:
    createAtms: "1663076722870"
    creatorBy: xca-operator
  name: xtls-sample
  namespace: default
  resourceVersion: "748326"
  uid: 0b277c5a-4e69-4375-b39c-4e69a9230c59
type: kubernetes.io/tls

介绍