侧边栏壁纸
博主头像
cloudnative-blog博主等级

I can break through the limitations !

  • 累计撰写 23 篇文章
  • 累计创建 12 个标签
  • 累计收到 3 条评论

目 录CONTENT

文章目录
k8s-进阶

二.ingress-nginx

周锐豪
周锐豪
2023-11-23 / 0 评论 / 0 点赞 / 159 阅读 / 18181 字 / 正在检测是否收录...

一.ingress简介

1.简介

    Ingress 是 k8s 官方提供的用于对外暴露服务的方式，也是在生产环境用的比较多的方式，一般情况是 LB + Ingress Ctroller 方式对外提供服务，这样就可以在一个 LB 的情况下根据域名路由到对应后端的 Service。

image-20240117155746446

    实际上，Ingress相当于一个七层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx，可以理解为Ingress里面建立了诸多映射规则，Ingress Controller通过监听这些配置规则并转化为Nginx的反向代理配置，然后对外提供服务。
    Ingress：kubernetes中的一个对象，作用是定义请求如何转发到Service的规则。
    Ingress Controller：具体实现反向代理及负载均衡的程序，对Ingress定义的规则进行解析，根据配置的规则来实现请求转发，实现的方式有很多，比如Nginx，Contour，Haproxy等。

2.ingress解决的问题

2.1Pod 漂移问题

    Kubernetes 具有强大的副本控制能力，能保证在任意副本（Pod）挂掉时自动从其他机器启动一个新的，还可以动态扩容等，通俗地说，这个 Pod 可能在任何时刻出现在任何节点上，也可能在任何时刻死在任何节点上；那么自然随着 Pod 的创建和销毁，Pod IP 肯定会动态变化；那么如何把这个动态的 Pod IP 暴露出去？这里借助于 Kubernetes 的 Service 机制，Service 可以以标签的形式选定一组带有指定标签的 Pod，并监控和自动负载他们的 Pod IP，那么我们向外暴露只暴露 Service IP 就行了；这就是 NodePort 模式：即在每个节点上开起一个端口，然后转发到内部 Pod IP 上，如下图所示：此时的访问方式：http://nodeip:nodeport/

image-20240117160652076

2.2端口的管理问题

    采用 NodePort 方式暴露服务面临问题是，服务一旦多起来，NodePort 在每个节点上开启的端口会及其庞大，而且难以维护；这时，我们可以能否使用一个Nginx直接对内进行转发呢？众所周知的是，Pod与Pod之间是可以互相通信的，而Pod是可以共享宿主机的网络名称空间的，也就是说当在共享网络名称空间时，Pod上所监听的就是Node上的端口。那么这又该如何实现呢？简单的实现就是使用 DaemonSet 在每个 Node 上监听 80，然后写好规则，因为 Nginx 外面绑定了宿主机 80 端口（就像 NodePort），本身又在集群内，那么向后直接转发到相应Service IP就行了 。如下图所示：

image-20240117160906187

2.3域名分配及动态更新问题

用户编写Ingress规则，说明那个域名对应kubernetes集群中的那个Service。
Ingress控制器动态感知Ingress服务规则的变化，然后生成一段对应的Nginx的反向代理配置。
Ingress控制器会将生成的Nginx配置写入到一个运行着的Nginx服务中，并动态更新。
到此为止，其实真正在工作的就是一个Nginx了，内部配置了用户定义的请求规则。

image-20240117161642282

二.ingress的配置

准备环境，一个deployment，service，ingress，secret域名证书

1.域名重定向

#重定向，访问nginx.zhouruihao.com则会重定向到https://cloudnative-blog.com/地址
annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: 'https://cloudnative-blog.com/'

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: 'https://cloudnative-blog.com/'
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

2.单主机ingress和多主机ingress

2.1单主机

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

2.2多主机

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
spec:
  rules:
  - host: nginx1.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx1-svc
            port:
              number: 80
  - host: nginx2.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx2-svc
            port:
              number: 80

3.基于url的ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
spec:
  rules:
  - host: nginx1.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/nginx1"
        backend:
          service:
            name: nginx1-svc
            port:
              number: 80
  - host: nginx2.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/nginx2"
        backend:
          service:
            name: nginx2-svc
            port:
              number: 80

4.配置https证书

4.1创建证书

kubectl create secret tls zhouruihao --key zhouruihao.com.key --cert zhouruihao.com.crt -n zhouruihao

4.2创建ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: view
  annotations:
    #如果设置为true，那么http请求会强制跳转https，默认也是强制跳转https，设置为false是希望也能够访问到http请求
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  tls:
  - hosts:
    - nginx.zhouruihao.com
    secretName: zhouruihao
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

5.白名单和黑名单

5.1白名单

#白名单建议配置单域名，可以是网段也可以是单ip
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.1.1/32，2.2.2.0/24

#想要全局配置白名单，则直接在ingress-nginx里面修改配置文件即可

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.1.1/32,2.2.2.0/24
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

5.2黑名单

#在ingress使用nginx配置文件，只在当前ingress生效
nginx.ingress.kubernetes.io/server-snippet

#配置黑名单，不允许1.1.1.1的ip地址访问当前域名
    nginx.ingress.kubernetes.io/server-snippet: |-
      deny 1.1.1.1;
      
#想要全局配置黑名单，则直接在ingress-nginx里面修改配置文件即可

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
      deny 1.1.1.1;
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

6.限流

参数注释
nginx.ingress.kubernetes.io/limit-connections#单个IP地址允许的并发连接数。超出此限制时，默认将返回503错误。
nginx.ingress.kubernetes.io/limit-rps每秒从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数，默认乘数为5。当客户端超过此限制时，将返回limit-req-status-code默认值：503。
nginx.ingress.kubernetes.io/limit-rpm每分钟从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数，默认乘数为5。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503。
nginx.ingress.kubernetes.io/limit-burst-multiplier突发大小限制速率的倍数。默认的脉冲串乘数为5，此注释将覆盖默认的乘数。当客户端超过此限制时，将 返回limit-req-status-code默认值： 503。
nginx.ingress.kubernetes.io/limit-rate-after最初的千字节数，在此之后，对给定连接的响应的进一步传输将受到速率的限制。必须在启用代理缓冲的情况下使用此功能。
nginx.ingress.kubernetes.io/limit-rate每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。
nginx.ingress.kubernetes.io/limit-whitelist每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。
nginx.ingress.kubernetes.io/limit-whitelist客户端IP源范围要从速率限制中排除。该值是逗号分隔的CIDR列表。
 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/limit-connections: "20"
    nginx.ingress.kubernetes.io/limit-rps: ”10“
    nginx.ingress.kubernetes.io/limit-rpm: “100”
    nginx.ingress.kubernetes.io/limit-burst-multiplier: ”5“
    nginx.ingress.kubernetes.io/global-rate-limit: ”100“
    nginx.ingress.kubernetes.io/global-rate-limit-window: 1s
    nginx.ingress.kubernetes.io/limit-rate: ”10240“
    nginx.ingress.kubernetes.io/limit-rate-after: ”10240“
    nginx.ingress.kubernetes.io/limit-whitelist: “1.1.1.1”
    #配置限速超出后的响应码为429，默认是503
    nginx.ingress.kubernetes.io/configuration-snippet: |
        limit_req_status 429;
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

7.爬虫

#下面示例是单个ingress域名配置反爬虫，也可以全局配置，全局配置在ingress-nginx

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
#禁止Scrapy等工具的抓取
      if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) {
         return 403;
      }
#禁止指定UA及UA为空的访问
      if ($http_user_agent ~ "WinHttp|WebZIP|FetchURL|node-superagent|java/|FeedDemon|Jullo|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|Java|Feedly|Apache-HttpAsyncClient|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|EasouSpider|Ezooms|BOT/0.1|YandexBot|FlightDeckReports|Linguee Bot|^$" ) {
         return 403;
      }
#禁止非GET|HEAD|POST|OPTIONS方式的抓取
      if ($request_method !~ ^(GET|HEAD|POST|OPTIONS)$) {
         return 403;
      }
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

8.auth认证

8.1创建用户

htpasswd -c auth zrh

8.2配置secret

kubectl create secret generic zrh --from-file=auth -n view

8.3配置ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    nginx.ingress.kubernetes.io/auth-realm: Need to longin
    nginx.ingress.kubernetes.io/auth-secret: zrh
    nginx.ingress.kubernetes.io/auth-type: basic
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

9.配置连接超时和数据大小

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "300" ##连接超时时间,默认为5s
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600" ##后端服务器回转数据超时时间,默认为60s
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300" ##后端服务器响应超时时间,默认为60s
    nginx.ingress.kubernetes.io/proxy-body-size: "500m" ##客户端上传文件，最大大小，默认为20m
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

10.配置单域名日志位置

ingress-nginx默认打印所有域名日志到控制台，可以通过配置文件将每个域名日志打印到本地(需提前创建好目录，否则会报错)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
      access_log /var/log/ingress/nginx-view/access.log upstreaminfo if=$loggable;
      error_log  /var/log/ingress/nginx-view/error.log;
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

11.限制访问目录

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
       location = /nginx {
            return 403;
       }
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

12.灰度发布

环境

环境旧pod: nginx-v1
新pod: nginx-v2

nginx-v1

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: view
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx1
        image: nginx
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
  namespace: view
spec:
  type: ClusterIP
  selector:
    app: nginx
    version: v1
  ports:
  - port:  80
    targetPort: 80

nginx-v2

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: view
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx2
        image: nginx
        imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v2
  namespace: view
spec:
  type: ClusterIP
  selector:
    app: nginx
    version: v2
  ports:
  - port:  80
    targetPort: 80

1.基于header请求头的流量切分

默认流量会进入nginx-v1
如果请求头中携带version: v2的信息，那么流量会进入nginx-v2

1.1nginx-v1的ingress(默认流量)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v1
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v1
            port:
              number: 80

1.2nginx-v2的ingress(匹配请求头)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v2
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "version"    #基于Request Header的流量切分
    nginx.ingress.kubernetes.io/canary-by-header-pattern: "v2" #Header信息中带有 version=v2 的转发到ingress
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v2
            port:
              number: 80

1.3流量验证

[root@master01]# curl  https://zhouruihao.com
nginx-v1
[root@master01]# curl  -H "version: v2" https://zhouruihao.com
nginx-v2

2.基于Cookie的流量切分

默认流量会进入nginx-v1
如果cookie中的信息带有version，那么流量会进入nginx-v2

2.1nginx-v1的ingress(默认流量)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v1
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v1
            port:
              number: 80

2.2nginx-v2的ingress(匹配cookie)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v2
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "version" #匹配cookie携带version,使用该ingress规则
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v2
            port:
              number: 80

2.3流量验证

[root@master01]# curl  https://zhouruihao.com
nginx-v1
[root@master01]# curl -s --cookie "version=always" https://zhouruihao.com
nginx-v2

3.基于权重的流量切分

90%流量会进入nginx-v1
10%流量会进入nginx-v2

3.1nginx-v1的ingress(权重90%)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v1
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v1
            port:
              number: 80

3.2nginx-v2的ingress(权重10%)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v2
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"   #10%的流量会转发到这个ingress
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v2
            port:
              number: 80

3.3流量验证

[root@master01]# for i in {1..10}; do curl https://nginx.zhouruihao.com; done;
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v2
0
k8s ingress
版权归属： 周锐豪
本文链接： https://cloudnative-blog.com/archives/er.ingress-nginx
许可协议： 本文使用《署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0)》协议授权

评论区