侧边栏壁纸
博主头像
cloudnative-blog博主等级

I can break through the limitations !

  • 累计撰写 23 篇文章
  • 累计创建 12 个标签
  • 累计收到 1 条评论

目 录CONTENT

文章目录

二.ingress-nginx

周锐豪
2023-11-23 / 0 评论 / 0 点赞 / 92 阅读 / 18181 字 / 正在检测是否收录...

一.ingress简介

1.简介

    Ingress 是 k8s 官方提供的用于对外暴露服务的方式,也是在生产环境用的比较多的方式,一般情况是 LB + Ingress Ctroller 方式对外提供服务,这样就可以在一个 LB 的情况下根据域名路由到对应后端的 Service。

image-20240117155746446

    实际上,Ingress相当于一个七层的负载均衡器,是kubernetes对反向代理的一个抽象,它的工作原理类似于Nginx,可以理解为Ingress里面建立了诸多映射规则,Ingress Controller通过监听这些配置规则并转化为Nginx的反向代理配置,然后对外提供服务。
    Ingress:kubernetes中的一个对象,作用是定义请求如何转发到Service的规则。
    Ingress Controller:具体实现反向代理及负载均衡的程序,对Ingress定义的规则进行解析,根据配置的规则来实现请求转发,实现的方式有很多,比如Nginx,Contour,Haproxy等。

2.ingress解决的问题

2.1Pod 漂移问题

    Kubernetes 具有强大的副本控制能力,能保证在任意副本(Pod)挂掉时自动从其他机器启动一个新的,还可以动态扩容等,通俗地说,这个 Pod 可能在任何时刻出现在任何节点上,也可能在任何时刻死在任何节点上;那么自然随着 Pod 的创建和销毁,Pod IP 肯定会动态变化;那么如何把这个动态的 Pod IP 暴露出去?这里借助于 Kubernetes 的 Service 机制,Service 可以以标签的形式选定一组带有指定标签的 Pod,并监控和自动负载他们的 Pod IP,那么我们向外暴露只暴露 Service IP 就行了;这就是 NodePort 模式:即在每个节点上开起一个端口,然后转发到内部 Pod IP 上,如下图所示:此时的访问方式:http://nodeip:nodeport/

image-20240117160652076

2.2端口的管理问题

    采用 NodePort 方式暴露服务面临问题是,服务一旦多起来,NodePort 在每个节点上开启的端口会及其庞大,而且难以维护;这时,我们可以能否使用一个Nginx直接对内进行转发呢?众所周知的是,Pod与Pod之间是可以互相通信的,而Pod是可以共享宿主机的网络名称空间的,也就是说当在共享网络名称空间时,Pod上所监听的就是Node上的端口。那么这又该如何实现呢?简单的实现就是使用 DaemonSet 在每个 Node 上监听 80,然后写好规则,因为 Nginx 外面绑定了宿主机 80 端口(就像 NodePort),本身又在集群内,那么向后直接转发到相应Service IP就行了 。如下图所示:

image-20240117160906187

2.3域名分配及动态更新问题

用户编写Ingress规则,说明那个域名对应kubernetes集群中的那个Service。
Ingress控制器动态感知Ingress服务规则的变化,然后生成一段对应的Nginx的反向代理配置。
Ingress控制器会将生成的Nginx配置写入到一个运行着的Nginx服务中,并动态更新。
到此为止,其实真正在工作的就是一个Nginx了,内部配置了用户定义的请求规则。

image-20240117161642282

二.ingress的配置

准备环境,一个deployment,service,ingress,secret域名证书

1.域名重定向

#重定向,访问nginx.zhouruihao.com则会重定向到https://cloudnative-blog.com/地址
annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: 'https://cloudnative-blog.com/'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: 'https://cloudnative-blog.com/'
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

2.单主机ingress和多主机ingress

2.1单主机

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

2.2多主机

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
spec:
  rules:
  - host: nginx1.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx1-svc
            port:
              number: 80
  - host: nginx2.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx2-svc
            port:
              number: 80

3.基于url的ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
spec:
  rules:
  - host: nginx1.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/nginx1"
        backend:
          service:
            name: nginx1-svc
            port:
              number: 80
  - host: nginx2.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/nginx2"
        backend:
          service:
            name: nginx2-svc
            port:
              number: 80

4.配置https证书

4.1创建证书

kubectl create secret tls zhouruihao --key zhouruihao.com.key --cert zhouruihao.com.crt -n zhouruihao

4.2创建ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: view
  annotations:
    #如果设置为true,那么http请求会强制跳转https,默认也是强制跳转https,设置为false是希望也能够访问到http请求
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  tls:
  - hosts:
    - nginx.zhouruihao.com
    secretName: zhouruihao
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

5.白名单和黑名单

5.1白名单

#白名单建议配置单域名,可以是网段也可以是单ip
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.1.1/32,2.2.2.0/24

#想要全局配置白名单,则直接在ingress-nginx里面修改配置文件即可
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.1.1/32,2.2.2.0/24
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

5.2黑名单

#在ingress使用nginx配置文件,只在当前ingress生效
nginx.ingress.kubernetes.io/server-snippet

#配置黑名单,不允许1.1.1.1的ip地址访问当前域名
    nginx.ingress.kubernetes.io/server-snippet: |-
      deny 1.1.1.1;
      
#想要全局配置黑名单,则直接在ingress-nginx里面修改配置文件即可
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
      deny 1.1.1.1;
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

6.限流

参数注释
nginx.ingress.kubernetes.io/limit-connections#单个IP地址允许的并发连接数。超出此限制时,默认将返回503错误。
nginx.ingress.kubernetes.io/limit-rps每秒从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数,默认乘数为5。当客户端超过此限制时,将返回limit-req-status-code默认值:503。
nginx.ingress.kubernetes.io/limit-rpm每分钟从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数,默认乘数为5。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。
nginx.ingress.kubernetes.io/limit-burst-multiplier突发大小限制速率的倍数。默认的脉冲串乘数为5,此注释将覆盖默认的乘数。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。
nginx.ingress.kubernetes.io/limit-rate-after最初的千字节数,在此之后,对给定连接的响应的进一步传输将受到速率的限制。必须在启用代理缓冲的情况下使用此功能。
nginx.ingress.kubernetes.io/limit-rate每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。
nginx.ingress.kubernetes.io/limit-whitelist每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。
nginx.ingress.kubernetes.io/limit-whitelist客户端IP源范围要从速率限制中排除。该值是逗号分隔的CIDR列表。
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/limit-connections: "20"
    nginx.ingress.kubernetes.io/limit-rps: ”10“
    nginx.ingress.kubernetes.io/limit-rpm: “100”
    nginx.ingress.kubernetes.io/limit-burst-multiplier: ”5“
    nginx.ingress.kubernetes.io/global-rate-limit: ”100“
    nginx.ingress.kubernetes.io/global-rate-limit-window: 1s
    nginx.ingress.kubernetes.io/limit-rate: ”10240“
    nginx.ingress.kubernetes.io/limit-rate-after: ”10240“
    nginx.ingress.kubernetes.io/limit-whitelist: “1.1.1.1”
    #配置限速超出后的响应码为429,默认是503
    nginx.ingress.kubernetes.io/configuration-snippet: |
        limit_req_status 429;
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

7.爬虫

#下面示例是单个ingress域名配置反爬虫,也可以全局配置,全局配置在ingress-nginx
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
#禁止Scrapy等工具的抓取
      if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) {
         return 403;
      }
#禁止指定UA及UA为空的访问
      if ($http_user_agent ~ "WinHttp|WebZIP|FetchURL|node-superagent|java/|FeedDemon|Jullo|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|Java|Feedly|Apache-HttpAsyncClient|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|EasouSpider|Ezooms|BOT/0.1|YandexBot|FlightDeckReports|Linguee Bot|^$" ) {
         return 403;
      }
#禁止非GET|HEAD|POST|OPTIONS方式的抓取
      if ($request_method !~ ^(GET|HEAD|POST|OPTIONS)$) {
         return 403;
      }
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

8.auth认证

8.1创建用户

htpasswd -c auth zrh

8.2配置secret

kubectl create secret generic zrh --from-file=auth -n view

8.3配置ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    nginx.ingress.kubernetes.io/auth-realm: Need to longin
    nginx.ingress.kubernetes.io/auth-secret: zrh
    nginx.ingress.kubernetes.io/auth-type: basic
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

9.配置连接超时和数据大小

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "300" ##连接超时时间,默认为5s
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600" ##后端服务器回转数据超时时间,默认为60s
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300" ##后端服务器响应超时时间,默认为60s
    nginx.ingress.kubernetes.io/proxy-body-size: "500m" ##客户端上传文件,最大大小,默认为20m
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

10.配置单域名日志位置

ingress-nginx默认打印所有域名日志到控制台,可以通过配置文件将每个域名日志打印到本地(需提前创建好目录,否则会报错)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
      access_log /var/log/ingress/nginx-view/access.log upstreaminfo if=$loggable;
      error_log  /var/log/ingress/nginx-view/error.log;
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

11.限制访问目录

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-view
  namespace: view
    annotations:
    nginx.ingress.kubernetes.io/server-snippet: |-
       location = /nginx {
            return 403;
       }
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

12.灰度发布

环境

环境旧pod: nginx-v1
新pod: nginx-v2

nginx-v1

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: view
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx1
        image: nginx
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
  namespace: view
spec:
  type: ClusterIP
  selector:
    app: nginx
    version: v1
  ports:
  - port:  80
    targetPort: 80

nginx-v2

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: view
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx2
        image: nginx
        imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-v2
  namespace: view
spec:
  type: ClusterIP
  selector:
    app: nginx
    version: v2
  ports:
  - port:  80
    targetPort: 80

1.基于header请求头的流量切分

默认流量会进入nginx-v1
如果请求头中携带version: v2的信息,那么流量会进入nginx-v2

1.1nginx-v1的ingress(默认流量)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v1
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v1
            port:
              number: 80

1.2nginx-v2的ingress(匹配请求头)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v2
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "version"    #基于Request Header的流量切分
    nginx.ingress.kubernetes.io/canary-by-header-pattern: "v2" #Header信息中带有 version=v2 的转发到ingress
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v2
            port:
              number: 80

1.3流量验证

[root@master01]# curl  https://zhouruihao.com
nginx-v1
[root@master01]# curl  -H "version: v2" https://zhouruihao.com
nginx-v2

2.基于Cookie的流量切分

默认流量会进入nginx-v1
如果cookie中的信息带有version,那么流量会进入nginx-v2

2.1nginx-v1的ingress(默认流量)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v1
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v1
            port:
              number: 80

2.2nginx-v2的ingress(匹配cookie)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v2
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "version" #匹配cookie携带version,使用该ingress规则
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v2
            port:
              number: 80

2.3流量验证

[root@master01]# curl  https://zhouruihao.com
nginx-v1
[root@master01]# curl -s --cookie "version=always" https://zhouruihao.com
nginx-v2

3.基于权重的流量切分

90%流量会进入nginx-v1
10%流量会进入nginx-v2

3.1nginx-v1的ingress(权重90%)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v1
  namespace: view
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v1
            port:
              number: 80

3.2nginx-v2的ingress(权重10%)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-v2
  namespace: view
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"   #10%的流量会转发到这个ingress
spec:
  rules:
  - host: nginx.zhouruihao.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: nginx-v2
            port:
              number: 80

3.3流量验证

[root@master01]# for i in {1..10}; do curl https://nginx.zhouruihao.com; done;
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v2
0

评论区