利用ELK里的gateway日志快速构建网关仪表盘

原创于 2025-08-07 14:08:52 发布 · 246 阅读

6 ·

CC 4.0 BY-SA版权

文章标签：

#elk #gateway

百度了一圈又是没找到相关教程，那我继续首发吧

目标：

基于spring cloud gateway日志，快速构建网关仪表盘。可以实现对高频、慢请求、超慢请求、报错请求的聚合分析，同时可以看指定接口的耗时分布图。

下面开搞！

一、日志结构化

作此声明：以下3行日志经过了安全脱敏，所有认证信息均为二次生成，无任何安全隐患

supos-gateway-577c6484-ntmkp 2025-08-07 01:47:29.922  INFO 8 [biz_worker-138] accessLogger : [upstream] [4f59766f-1150769312194957312-global] Resolved [200 OK] for GET /inter-api/broker/ws?token=AOWD3FNj341QM0cn2w, host [global.ims.whchem.com], authorization [Bearer AOWD3FNje2t1QM0cnvbhw], suposTicket cookie [AOWD3FNje2t1QM0cnvb2w], sourceIp [10.13.1.103],serviceName [10.13.1.103], target [ws://100.99.127.122:5069/inter-api/broker/ws?token=AOWD3FNje2t1QM045nvbhw], clientId [], elapsed [4]ms
supos-gateway-577c6484-ntmkp 2025-08-07 01:47:30.035  INFO 8 [biz_worker-144] accessLogger : [upstream] [8f1ae1ef-1150769312626970624-global] Resolved [200 OK] for GET /open-api/supos/oodm/v2/template/system/Template_8754/instance/Instance_18163/attribute/self?pageIndex=1&pageSize=500, host [supos-gateway], authorization [Sign 3ba8a60b48185a9e23d38e962b4123-57af6d62a0a66b2415f6549780f8bc11f286e013a33690fd943cd00069029e82], suposTicket cookie [], sourceIp [100.93.226.155],serviceName [100.93.226.155], target [http://oodm-gateway.default:18090/open-api/supos/oodm/v2/template/system/Template_8754/instance/Instance_18163/attribute/self?pageIndex=1&pageSize=500], clientId [], elapsed [12]ms
supos-gateway-577c6484-ntmkp 2025-08-07 01:47:19.038 ERROR 8 [biz_worker-43] accessLogger : [error] [7272764c-1150769266539958272-global] Resolved [GatewayResponseStatusException: 403 "600105999"] for HTTP GET /msService/HierarchicalMod/baseService/workbench/download/getAuthenticationMessage, host [global.ims.whchem.com], authorization [Bearer 1F4fX5O4HTwo3XkBYsoI1], suposTicket cookie [], source [10.10.229.123], target [], clientId []

需要将日志结构化保存到es中

我这里利用grok脚本解析日志(利用grok Debugger反复调试)

要注意的是，我认为path里的？参数传参应该被拿掉，这样更有利于后期的聚合分析，所以用了(?<path>[^? ]+)(?:\?[^ ]*)? 把问号后的字符省掉了

%{DATA:hostname} %{TIMESTAMP_ISO8601:timestamp}%{SPACE}+%{WORD:loglevel}%{SPACE}%{INT:thread} \[%{DATA:component}\] %{WORD:logger} : (?:\[%{DATA:tag}\] )?\[%{DATA:id}\] Resolved \[(?<status>[^\]]+)\] for (?:HTTP )?%{WORD:method} (?<path>[^? ]+)(?:\?[^ ]*)?, host \[%{DATA:domain}\], authorization \[%{DATA:authorization}\], suposTicket cookie \[%{DATA:suposTicket}\], (?:sourceIp|source) \[%{IP:sourceIp}\](?:,serviceName \[%{DATA:serviceName}\])?,\s*target \[%{DATA:target}\], clientId \[%{DATA:clientId}\](?:, elapsed \[%{NUMBER:elapsed:int}%{DATA}\]ms)?

利用ELK里的gateway日志快速构建网关仪表盘_elk