nginx+rsyslog+kafka+clickhouse+grafana 实现nginx 网关监控

需求

  • 我想做一个类似腾讯云网关日志最终以仪表方式呈现，比如说qps、p99、p95的请求响应时间等等

流程图

数据流转就像标题
nginx ----> rsyslog ----> kafka —> clickhouse —> grafana

部署

kafka

kafka 相关部署这里不做赘述，只要创建一个topic 就可以
这里kafka地址是 192.168.1.180,topic是``

rsyslog 设置

rsyslog 具体是啥东西这个我这里也不做介绍，本人也是一个后端开发不是做运维的，只知道这个东西性能不错，算是logstash 平替把

# 安装rsyslog-kafka 插件
yum install -y rsyslog-kafka
# 创建一个配置

vim /etc/rsyslog.d/rsyslog_nginx_kafka_cluster.conf

conf 内容

module(load="imudp")
input(type="imudp" port="514")

# nginx access log ==> rsyslog server(local) ==> kafka
module(load="omkafka")

template(name="nginxLog" type="string" string="%msg%")

if $inputname == "imudp" then {
    # 这里的名字和下面nginx 配置tag 相同
    if ($programname == "nginx_access_log") then
        action(type="omkafka"
            template="nginxLog"
            # kafka 地址
            broker=["192.168.1.180:9092"]
            # topic 名字
            topic="rsyslog_nginx"
            partitions.auto="on"
            confParam=[
                "socket.keepalive.enable=true"
            ]
        )
}

:rawmsg, contains, "nginx_access_log" ~

最后重启rsyslog 即可

nginx

http 节点中设置如下

log_format jsonlog '{'
        '"host": "$host",'
        '"server_addr": "$server_addr",'
        '"remote_addr":"$remote_addr",'
        '"time_format":"$time_ms",'
        '"time_sec":$times,'
        '"timestamp":$timestamp,'
        '"method":"$request_method",'
        '"request_url":"$request_uri",'
        '"status":$status,'
        '"upstream_name":"$upstream_name",'
        '"http_user_agent":"$http_user_agent",'
        '"upstream_addr":"$upstream_addr",'
        '"trace_id":"$http_traceid",'
        '"upstream_status":"$upstream_status",'
        '"upstream_response_time":"$upstream_response_time",'
        '"request_time":$request_time,'
        '"nginx_host":"$nginx_host"'
        '}';
	# 配置rsyslog，tag 和上文rsyslog 中相同！！！
    access_log syslog:server=192.168.1.179,facility=local7,tag=nginx_access_log,severity=info jsonlog;
    #access_log  /var/log/nginx/access.log  jsonlog;

配置实际转发upstream

upstream testcontrol{
	server 192.168.10.123:8081;
}

server {
    listen       18081;
    server_name  _;
    # 设置up name
    set $upstream_name 'testcontrol';
    include /etc/nginx/conf/time.conf;

    location / {
	proxy_set_header traceid $http_traceid;
        proxy_set_header n-t $msec;
        #proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://testcontrol;
    }

    location ~ /status {
        stub_status   on;
    }
}

/etc/nginx/conf/time.conf; 配置

set $nginx_host '192.168.8.64';
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})") {
        set $year $1;
        set $month $2;
        set $day $3;
        set $hour $4;
        set $minutes $5;
        set $seconds $6;
        set $time_sec "$1-$2-$3 $4:$5:$6";
    }

    # 获取毫秒时间戳，并拼接到$time_zh之后
    if ($msec ~ "(\d+)\.(\d+)") {
       # 时间戳-ms
        set $timestamp $1$2;
        # 时间戳-s
	     set $times $1;
	     # 格式化之后的时间ms
        set $time_ms $time_sec.$2;
        # traceId，方便查日志
	    set $http_traceid $nginx_host#$timestamp#$request_id;
}

clickhouse

先创建一个 nginx_gw_log 表


create table app_logs.nginx_gw_log
(
    timestamp              DateTime64(3, 'Asia/Shanghai'),
    server_addr            String,
    remote_addr            String,
    time_sec               DateTime('Asia/Shanghai'),
    method                 String,
    request_url            String,
    status                 Int16,
    upstream_name          String,
    http_user_agent        String,
    upstream_addr          String,
    trace_id               String,
    upstream_status        String,
    upstream_response_time String,
    request_time           Int32,
    nginx_host              String,
    host                   String
) engine = MergeTree PARTITION BY toYYYYMMDD(timestamp)
      ORDER BY timestamp
      TTL toDateTime(timestamp) + toIntervalDay(1);

上面设置了过期1天，可以自由修改
添加kafka 引擎

create table app_logs.nginx_gw_log_kafka
(
    timestamp              DateTime64(3, 'Asia/Shanghai'),
    server_addr            String,
    remote_addr            String,
    time_sec               DateTime('Asia/Shanghai'),
    method                 String,
    request_url            String,
    status                 Int32,
    upstream_name          String,
    http_user_agent        String,
    upstream_addr          String,
    trace_id               String,
    upstream_status        String,
    upstream_response_time String,
    request_time           Decimal32(3),
    nginx_host              String,
    host                   String
) engine = Kafka()
      SETTINGS kafka_broker_list = '192.168.1.180:9092',
          kafka_topic_list = 'rsyslog_nginx',
          kafka_group_name = 'nginx_ck',
          kafka_format = 'JSONEachRow',
          kafka_skip_broken_messages = 100000,
          kafka_num_consumers = 1;

最后设置mv

CREATE MATERIALIZED VIEW app_logs.nginx_gw_log_mv TO app_logs.nginx_gw_log AS
SELECT timestamp,
       server_addr,
       remote_addr,
       time_sec,
       method,
       request_url,
       status,
       upstream_name,
       http_user_agent,
       upstream_addr,
       trace_id,
       upstream_status,
       upstream_response_time,
       toInt32(multiply(request_time, 1000)) as request_time,
       host
FROM app_logs.nginx_gw_log_kafka;

这里做了一个类型转换，因为nginx，request-time 单位是s，我想最终呈现在grafana 中是ms，所以这里做了转换，当然grafana中也可以做

上面都配置完了之后可以先验证下，保证数据最终到ck，如果有问题，需要再每个节点调试，比如先调试nginx->rsyslog ，可以先不配置kafka 输出，配置为console或者文件输出都可以，具体这里就不写了

grafana

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 17,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT time_sec as a, count() as num FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime )\r\ngroup by a order by a desc ;\r\n",
          "refId": "A",
          "selectedFormat": 4
        }
      ],
      "title": "qps",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": false,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 12,
        "y": 0
      },
      "id": 3,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT time_sec as a, toInt32(quantile(0.95)(request_time))  as num FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime )\r\ngroup by a order by a desc ;",
          "refId": "p95",
          "selectedFormat": 4
        },
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "hide": false,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT time_sec as a, toInt32(quantile(0.99)(request_time))  as num FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime )\r\ngroup by a order by a desc ;",
          "refId": "p99",
          "selectedFormat": 4
        }
      ],
      "title": "响应时间统计",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "align": "auto",
            "displayMode": "auto",
            "inspect": false
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 9
      },
      "id": 4,
      "options": {
        "footer": {
          "fields": "",
          "reducer": [
            "sum"
          ],
          "show": false
        },
        "showHeader": true
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT request_time as k,request_url,trace_id  FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime )\r\n order by k desc limit 10;\r\n",
          "refId": "A",
          "selectedFormat": 4
        }
      ],
      "title": "最耗时",
      "type": "table"
    },
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "align": "auto",
            "displayMode": "auto",
            "inspect": false
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 12,
        "y": 9
      },
      "id": 5,
      "options": {
        "footer": {
          "fields": "",
          "reducer": [
            "sum"
          ],
          "show": false
        },
        "showHeader": true
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT request_url, count() as num FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime )\r\ngroup by request_url order by request_url desc ;",
          "refId": "A",
          "selectedFormat": 4
        }
      ],
      "title": "接口请求数量",
      "type": "table"
    },
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "custom": {
            "align": "auto",
            "displayMode": "auto",
            "inspect": false
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 18
      },
      "id": 6,
      "options": {
        "footer": {
          "fields": "",
          "reducer": [
            "sum"
          ],
          "show": false
        },
        "showHeader": true
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT DISTINCT host  FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime );\r\n",
          "refId": "A",
          "selectedFormat": 4
        }
      ],
      "title": "upstream",
      "type": "table"
    },
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "custom": {
            "align": "auto",
            "displayMode": "auto",
            "inspect": false
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 12,
        "y": 18
      },
      "id": 8,
      "options": {
        "footer": {
          "fields": "",
          "reducer": [
            "sum"
          ],
          "show": false
        },
        "showHeader": true
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT  upstream_status,count() as num  FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime ) group by upstream_status;\r\n",
          "refId": "A",
          "selectedFormat": 4
        }
      ],
      "title": "upstream",
      "type": "table"
    },
    {
      "datasource": {
        "type": "grafana-clickhouse-datasource",
        "uid": "${datasource}"
      },
      "fieldConfig": {
        "defaults": {
          "mappings": [],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "orange",
                "value": 70
              },
              {
                "color": "red",
                "value": 85
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 12,
        "y": 27
      },
      "id": 7,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "9.1.1",
      "targets": [
        {
          "builderOptions": {
            "database": "app_logs",
            "fields": [],
            "filters": [
              {
                "condition": "AND",
                "filterType": "custom",
                "key": "timestamp",
                "operator": "WITH IN DASHBOARD TIME RANGE",
                "restrictToFields": [
                  {
                    "label": "timestamp",
                    "name": "timestamp",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  },
                  {
                    "label": "startTime",
                    "name": "startTime",
                    "picklistValues": [],
                    "type": "DateTime64(3, 'Asia/Shanghai')"
                  }
                ],
                "type": "datetime"
              }
            ],
            "groupBy": [],
            "limit": 100,
            "metrics": [
              {
                "aggregation": "count",
                "field": ""
              }
            ],
            "mode": "list",
            "orderBy": [],
            "table": "tk_wx_control_req_kafka",
            "timeField": "timestamp",
            "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
          },
          "datasource": {
            "type": "grafana-clickhouse-datasource",
            "uid": "${datasource}"
          },
          "format": 1,
          "meta": {
            "builderOptions": {
              "database": "app_logs",
              "fields": [],
              "filters": [
                {
                  "condition": "AND",
                  "filterType": "custom",
                  "key": "timestamp",
                  "operator": "WITH IN DASHBOARD TIME RANGE",
                  "restrictToFields": [
                    {
                      "label": "timestamp",
                      "name": "timestamp",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    },
                    {
                      "label": "startTime",
                      "name": "startTime",
                      "picklistValues": [],
                      "type": "DateTime64(3, 'Asia/Shanghai')"
                    }
                  ],
                  "type": "datetime"
                }
              ],
              "groupBy": [],
              "limit": 100,
              "metrics": [
                {
                  "aggregation": "count",
                  "field": ""
                }
              ],
              "mode": "list",
              "orderBy": [],
              "table": "tk_wx_control_req_kafka",
              "timeField": "timestamp",
              "timeFieldType": "DateTime64(3, 'Asia/Shanghai')"
            }
          },
          "queryType": "sql",
          "rawSql": "SELECT count()  FROM \"app_logs\".${table_name} where ( timestamp  >= $__fromTime AND timestamp <= $__toTime );\r\n",
          "refId": "A",
          "selectedFormat": 4
        }
      ],
      "title": "请求总数",
      "type": "gauge"
    }
  ],
  "schemaVersion": 37,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "current": {
          "selected": false,
          "text": "nginx_gw_log",
          "value": "nginx_gw_log"
        },
        "datasource": {
          "type": "grafana-clickhouse-datasource",
          "uid": "${datasource}"
        },
        "definition": "show  tables from app_logs where name = 'nginx_gw_log' ",
        "hide": 0,
        "includeAll": false,
        "label": "table_name",
        "multi": false,
        "name": "table_name",
        "options": [],
        "query": "show  tables from app_logs where name = 'nginx_gw_log' ",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "current": {
          "selected": false,
          "text": "ClickHouse_1.178",
          "value": "ClickHouse_1.178"
        },
        "hide": 0,
        "includeAll": false,
        "label": "datasource",
        "multi": false,
        "name": "datasource",
        "options": [],
        "query": "grafana-clickhouse-datasource",
        "queryValue": "",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      },
      {
        "current": {
          "isNone": true,
          "selected": false,
          "text": "None",
          "value": ""
        },
        "datasource": {
          "type": "grafana-clickhouse-datasource",
          "uid": "${datasource}"
        },
        "definition": "select upstream_name from app_logs.nginx_gw_log",
        "hide": 0,
        "includeAll": false,
        "label": "upstream_name",
        "multi": false,
        "name": "upstream_name",
        "options": [],
        "query": "select upstream_name from app_logs.nginx_gw_log",
        "refresh": 2,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "current": {
          "isNone": true,
          "selected": false,
          "text": "None",
          "value": ""
        },
        "datasource": {
          "type": "grafana-clickhouse-datasource",
          "uid": "${datasource}"
        },
        "definition": "select nginx_host from app_logs.nginx_gw_log where upstream_name = \"$upstream_name\"",
        "hide": 0,
        "includeAll": false,
        "label": "nginx_host",
        "multi": false,
        "name": "nginx_host",
        "options": [],
        "query": "select nginx_host from app_logs.nginx_gw_log where upstream_name = \"$upstream_name\"",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-5m",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "nginx_gw",
  "uid": "lZrbSYOIkA",
  "version": 6,
  "weekStart": ""
}

最终呈现效果

在这里插入图片描述

隐患点

  • clickhouse 本身查询qps 就不高，如果数据量很大可以考虑集群或者其他的存储，doris、es等等
  • 不知道nginx+rsyslog 对性能有多少影响，目前测试单机nginx大几千的qps 都没啥问题

有其他更好方案的小伙伴留言哦

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：/a/277709.html

如若内容造成侵权/违法违规/事实不符，请联系我们进行投诉反馈qq邮箱809451989@qq.com，一经查实，立即删除！

相关文章

爬虫工作量由小到大的思维转变---＜第三十四章 Scrapy 的部署scrapyd+Gerapy＞

爬虫工作量由小到大的思维转变---＜第三十四章 Scrapy 的部署scrapyd+Gerapy＞

前言: scrapy-redis没被部署,感觉讲起来很无力;因为实在编不出一个能让scrapy-redis发挥用武之地的案子;所以,索性直接先把分布式爬虫的部署问题给讲清楚!! 然后,曲线救国式地再在部署的服务器上,讲scrapy redis我感觉这样才好! 正文: 现在还有不少人在用scrapy web进行爬虫管…
阅读更多...
JProfiler for Mac/win中文版：Java性能分析工具的首选

JProfiler for Mac/win中文版：Java性能分析工具的首选

JProfiler是一款功能强大的Java性能分析工具&#xff0c;它可以帮助开发人员快速定位和解决应用程序中的性能问题。无论是在开发阶段还是在生产环境中&#xff0c;JProfiler都能提供全面的性能分析和优化功能。 首先&#xff0c;JProfiler提供了一系列强大的分析工具&#xff…
阅读更多...
[鹏城杯 2022]简单包含

[鹏城杯 2022]简单包含

[鹏城杯 2022]简单包含 wp 题目代码如下&#xff1a; <?php highlight_file(__FILE__); include($_POST["flag"]); //flag in /var/www/html/flag.php; 直接 POST 传参&#xff1a; flag/var/www/html/flag.php 会触发 waf 。 尝试用伪协议读取&#xff1a; …
阅读更多...
IP地址SSL证书

IP地址SSL证书

IP地址SSL证书是一种专门针对公网IP地址颁发的数字证书。与常规的域名SSL证书类似&#xff0c;其主要目标是提供数据加密和身份验证。以下几点概述了IP地址SSL证书的重要特性及其申请过程&#xff1a; 1. 保护直接IP访问&#xff1a; 当用户直接通过IP地址访问服务时&#xff…
阅读更多...
家庭记账本，记账项目图表分析

家庭记账本，记账项目图表分析

随着生活的节奏加快&#xff0c;财务的数字化、透明化成为了越来越多人的需求。而在这背后&#xff0c;记账成为了实现这一需求的关键所在。一个好的记账软件可以在深度上为我们提供了更多的数据参考&#xff0c;帮我们理清财务管理的思路&#xff0c;进而做到开源节流。 所需…
阅读更多...
RabbitMQ 核心概念(交换机、队列、路由键)，队列类型等介绍

RabbitMQ 核心概念(交换机、队列、路由键)，队列类型等介绍

RabbitMQ 核心概念(交换机、队列、路由键)&#xff0c;队列类型等介绍 RabbitMQ 是一个消息队列系统&#xff0c;它的核心概念包括交换机&#xff08;Exchange&#xff09;、队列&#xff08;Queue&#xff09;和路由键&#xff08;Routing Key&#xff09;&#xff0c;它们一起…
阅读更多...
C# ASP.NET 实验室 检验中心 医疗LIS源码

C# ASP.NET 实验室 检验中心 医疗LIS源码

LIS系统能够自动处理大量的医学数据&#xff0c;包括样本采集、样本处理、检测分析、报告生成等。它能够快速、准确地进行化验检测&#xff0c;提高医院的运营效率。LIS系统还提供了丰富的数据分析功能&#xff0c;能够对医院化验室的业务流程进行全面、细致的监控。 LIS系统优…
阅读更多...
【每日一题】收集巧克力

【每日一题】收集巧克力

文章目录 Tag题目来源题目解读解题思路方法一&#xff1a;枚举操作数 写在最后 Tag 【枚举】【数组】【2023-12-28】 题目来源 2735. 收集巧克力 题目解读 有长度为 n, 下标从 0 开始的整数数组 nums, 表示收集不同类型的巧克力的成本. nums[i] 表示收集类型 i 巧克力的成本…
阅读更多...
取证工具volatility插件版学习记录

取证工具volatility插件版学习记录

更新时间&#xff1a;2023年12月18日11:48:29 1. 背景描述 在以前学习过volatility的基础功能&#xff0c;主要是使用volatility独立版进行学习的&#xff0c;前几天遇到一个ctf赛事&#xff0c;需要用到的是volatility的mimikatz模块&#xff0c;因为以前没使用过那个模块&…
阅读更多...
腾讯云轻量应用服务器性能差吗？

腾讯云轻量应用服务器性能差吗？

腾讯云轻量应用服务器性能如何&#xff1f;轻量服务器CPU采用什么型号&#xff1f;处理器计算性能如何&#xff1f;轻量应用服务器会不会比云服务器CVM性能差&#xff1f;腾讯云服务器网txyfwq.com详解轻量CPU型号主频、处理器性能、内存、公网带宽、月流量、不同地域速度测试、…
阅读更多...
腾讯云价格计算器，一键计算精准报价，好用！

腾讯云价格计算器，一键计算精准报价，好用！

腾讯云价格计算器&#xff1a;可以计算腾讯云服务器不同CVM实例规格、CPU内存、公网带宽和系统盘费用明细表&#xff0c;可以一键计算出精准报价明细表&#xff0c;腾讯云服务器网txyfwq.com分享大家腾讯云服务器价格计算器入口链接、使用方法及限制说明&#xff1a; 腾讯云服…
阅读更多...
前端图片适配不同屏幕方案

前端图片适配不同屏幕方案

预备知识&#xff1a; 设备独立像素,以下图的iphone12 Pro为例&#xff0c;390*844表示的就是设备独立像素&#xff08;DIP&#xff09;,也可以理解为CSS像素 物理像素&#xff08;设备像素&#xff09;&#xff0c;就是屏幕的分辨率&#xff0c;显示屏就是由一个个物理像素…
阅读更多...
绝地求生：大逃杀，鼠标灵敏度设置教程及枪法练习技巧 鼠标灵敏度怎么设置

绝地求生：大逃杀，鼠标灵敏度设置教程及枪法练习技巧 鼠标灵敏度怎么设置

《绝地求生大逃杀》鼠标灵敏度怎么设置&#xff1f;作为一款FPS游戏&#xff0c;如何调整鼠标参数是大家急需掌握的&#xff0c;今天闲游盒带来“院长尼克”分享的《绝地求生大逃杀》鼠标灵敏度设置教程及枪法练习技巧&#xff0c;废话不多说&#xff0c;下面我们一起来看吧。 …
阅读更多...
AIGC时代下，结合ChatGPT谈谈儿童教育

AIGC时代下，结合ChatGPT谈谈儿童教育

引言 都2024年了&#xff0c;谈到儿童教育&#xff0c;各位有什么新奇的想法嘛 我觉得第一要务&#xff0c;要注重习惯养成&#xff0c;我觉得聊习惯养成这件事情范围有点太大了&#xff0c;我想把习惯归纳于底层逻辑&#xff0c;我们大家都知道&#xff0c;在中国式教育下&a…
阅读更多...
jdk17安装

jdk17安装

前言 也许是太久没有新建java项目了&#xff0c;官网新建spring项目最低到17了&#xff0c;吃惊… 最近正好项目需要&#xff0c;就安装下&#xff0c;顺便记录下&#xff0c;与诸君共勉&#xff01;抱拳~ 参考文章 JDK17的下载安装与配置(详细教程) 文件下载地址 jdk17-win…
阅读更多...
众和策略：人工智能风起云涌 算力基建支撑加速前进

众和策略：人工智能风起云涌 算力基建支撑加速前进

2023年&#xff0c;人工智能技术完结质的飞跃。通过生成式AI&#xff08;AIGC&#xff09;技术&#xff0c;人们可用自然语言与机器进行便捷交互&#xff0c;并将海量的数据通过训练、推理&#xff0c;快速转化为出产力&#xff0c;发生实践商业价值。 AI技术加快向各行各业渗…
阅读更多...
vue2导入

vue2导入

父页面 <template><div><div><el-button size"small" click"excelFn">导入</el-button></div><div v-if"ExcelInSure"><excelln :names"names" close"closeFn"></exce…
阅读更多...
【解决问题】pyinstaller打包python应用进行快速分发

【解决问题】pyinstaller打包python应用进行快速分发

pyinstaller打包python应用进行快速分发 问题起因先利其器再善其事试用运行 问题起因 有同学问我要接口的应用&#xff0c;于是试了一下python打包成exe的过程。 先利其器 主要使用pyinstaller&#xff0c;可以通过pip安装 pip install pyinstaller安装过程如图 再善其事…
阅读更多...
文本的剪切和复制有区别吗？有什么区别

文本的剪切和复制有区别吗？有什么区别

在电脑操作中&#xff0c;文本的剪切与复制是我们经常进行的操作。尽管它们看起来都是对文本的“复制”行为&#xff0c;但两者在使用和功能上存在明显的差异。本文将详细介绍剪切与复制的区别&#xff0c;以帮助您更好地理解它们的适用场景和作用&#xff0c;并介绍剪切后如何…
阅读更多...
亚信安慧AntDB数据并行加载工具的实现（二）

亚信安慧AntDB数据并行加载工具的实现（二）

3.功能性说明 本节对并行加载工具的部分支持的功能进行简要说明。 1) 支持表类型 并行加载工具支持普通表、分区表。 2) 支持指定导入字段 文件中并不是必须包含表中所有的字段&#xff0c;用户可以指定导入某些字段&#xff0c;但是指定的字段数要和文件中的字段数保持一…
阅读更多...
最新文章

Copyright @ 2022~2023