Ingress Nginx 介绍

发布时间： 更新时间： 总字数：1445 阅读时间：3m 作者：IP：上海 网址

ingress-nginx是使用 nginx 实现的kubernetes ingress

介绍

本质是基于 OpenResty 开发，构建脚本如下：

部署

下载部署文件

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
  • 若安装失败，可以修改镜像源
k8s.gcr.io/ingress-nginx/controller -> k8sgcrioingressnginx/controller
k8s.gcr.io/ingress-nginx/kube-webhook-certgen -> k8sgcrioingressnginx/kube-webhook-certgen
k8s.gcr.io/ingress-nginx/kube-webhook-certgen -> k8sgcrioingressnginx/kube-webhook-certgen

安装

$ kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

plugin

部署 ingress-nginx kubectl-plugin（可选）

https://kubernetes.github.io/ingress-nginx/kubectl-plugin/

原理

  • 渲染时的模板：https://github.com/kubernetes/ingress-nginx/blob/26fe69cb47/rootfs/etc/nginx/template/nginx.tmpl
  • 注入的 lua 脚本：https://github.com/kubernetes/ingress-nginx/tree/26fe69cb47295f6677576fa8585afa98d72536e2/rootfs/etc/nginx/lua

暴露服务方式

  • 使用 LoadBalancer，如 MetalLB 负载均衡器使用介绍，本示例使用的方式
  • Deployment Pod 共享宿主机 net namesapce
  • 也可以修改 service type 为NodePort，如下：
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort    # 修改为 NodePort 类型 *
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
      nodePort: 30080    # http请求对外映射 30080 端口 *
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443    # https请求对外映射30443端口 *
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

使用

查看和访问 ingress-nginx

$ kubectl -n ingress-nginx get svc
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.105.71.57     172.20.0.200   80:30668/TCP,443:31198/TCP   99s
ingress-nginx-controller-admission   ClusterIP      10.105.141.141   <none>         443/TCP                      29m
$ kubectl -n ingress-nginx get ingressclasses.networking.k8s.io
NAME    CONTROLLER             PARAMETERS   AGE
nginx   k8s.io/ingress-nginx   <none>       45m

访问：https://172.20.0.200 进入 ingress-nginx，默认没有配置任何 backend。

日志

nginx-ingress-controller 执行 configmap，确定

 apiVersion: v1
 data:
    log-format: '{remote_address: $remote_addr, remote_user: "$remote_user", time_date: [$time_local], request: "$request", status: $status, http_referer: "$http_referer", http_user_agent: "$http_user_agent", request_id: $request_id}'
    log-format-escape-json: "true"
    enable-syslog: "true"
    syslog-host: <syslog-ip>
    syslog-port: "<syslog-port>"
 kind: ConfigMap
 metadata:
   name: nginx-ingress-controller-cm
   namespace: kube-system

nginx-ingress 日志说明：

  • 推荐把日志发送到 syslog 中
    • 针对 access_log、error_log 通过上面的方式配置
    • 针对 server 的日志，推荐配置如下：
access_log syslog:server=[2001:db8::1]:1234,facility=local7,tag=nginx,severity=info;
  • /var/log/nginx/access.log 重定向到 /dev/stdout
  • /var/log/nginx/error.log 重定向到 /dev/stderr
  • 上述的 Dockerfile 命令
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
    && ln -sf /dev/stderr /var/log/nginx/error.log
  • 或直接配置 nginx 输出
access_log /dev/stdout main;
error_log /dev/stdout warn;

Upstream 配置

默认情况下 NGINX ingress controller NGINX upstream 配置的 endpoints 为 Pod IP/port，可以通过 nginx.ingress.kubernetes.io/service-upstream 配置。

Stream snippet

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/stream-snippet: |
      server {
        listen 8000;
        proxy_pass 127.0.0.1:80;
      }

websocket

apiVersion: networking.k8s.io/v1​
kind: Ingress​
metadata:​
  annotations:
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"​
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"

rewrite

apiVersion: networking.k8s.io/v1​
kind: Ingress​
metadata:​
  name: abc​
  annotations:​
    nginx.ingress.kubernetes.io/rewrite-target: /$1​
spec:​
  ingressClassName: nginx​
  rules:​
  - host: abc.xiexianbin.cn
    http:​
      paths:​
      - path: /weixin/(.*)​
        pathType: Prefix​
        backend:​
          service:​
            name: weixin​
            port:​
              number: 80​
      - path: /(.*)​
        pathType: Prefix​
        backend:​
          service:​
            name: frontend​
            port:​
              number: 80​

x-request-id

  • 客户端请求到达 Ingress-Nginx Controllerr 时会自动添加一个 X-Request-ID 的请求 Header，参考

header 中下划线丢失

  • nginx 对 header name 中如果包含下划线(_)，则自动忽略，默认参数 underscores_in_headers: off
  • ingress-nginx 配置 ingress-nginx/templates/controller-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    ...
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  // 配置这里
  enable-underscores-in-headers: "true"
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    ...
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  // 配置这里
  large-client-header-buffers: "4 32k"

others

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: my-app-name
  namespace: my-namespace
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/proxy-body-size: 16m
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"

调优

启动 keep-alive

  • 用来避免建立/释放连接的开销，是代理更高效，性能更高，大大提高并发数
$ kubectl -n kube-system edit configmap nginx-configuration
...
apiVersion: v1
data:
  keep-alive: "60"
  keep-alive-requests: "100"
  upstream-keepalive-connections: "10000"
  upstream-keepalive-requests: "100"
  upstream-keepalive-timeout: "60"
kind: ConfigMap
...

长链接断开问题

ingress-nginx 在有新配置时，会触发 nginx -s reload 动作，虽然长链接会保持（由 worker_shutdown_timeout 控制），但超时时会被杀死，建议扩大该超时时间，或业务高峰期减少配置调整次数

最新评论
本文总阅读量 次 本站总访问量 次 本站总访客数 人