arthas线上问题排查

最新推荐文章于 2024-12-02 16:26:33 发布

原创最新推荐文章于 2024-12-02 16:26:33 发布 · 701 阅读

10 ·

CC 4.0 BY-SA版权

文章标签：

#java

文章目录

场景一：接口很卡，不知道卡在哪里
场景二：接口异常报错，猜测是方法入参的问题，或者是某个方法返回值的问题
场景三：线上问题无法定位，需要添加日志打印进行查看，同时又不能随便的更新程序
场景四：不确定某段逻辑线上代码是不是最新的

场景一：接口很卡，不知道卡在哪里

一般是数据库操作的时候容易耗时久; 也可能是哪个方法有问题，导致执行时间过长；也可能是数据量大，一直循环处理。如果没有线上分析工具，只能一行行看代码，然后猜

启动arthas
java -jar arthas-boot.jar
[1]: 11300 wgcloud-server-release.jar
[2]: 11189 starter-1.0-SNAPSHOT.jar
输入数字，选择目标进程
2
找到需要排查的方法信息，主要是类名、方法名
trace net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl saveWorkOrderBasicInfo
观察输出结果，排查耗时长的方法

`---ts=2024-03-19 11:51:06;thread_name=wbwf-task-33;id=282;is_daemon=false;priority=5;TCCL=org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader@305f7627
   `---[2601.331906ms] net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl:saveWorkOrderBasicInfo()
       +---[0.00% 0.117245ms ] org.slf4j.Logger:info() #286
       +---[0.00% 0.016244ms ] org.activiti.engine.HistoryService:createNativeHistoricProcessInstanceQuery() #290
       +---[0.00% 0.011803ms ] org.activiti.engine.history.NativeHistoricProcessInstanceQuery:sql() #290
       +---[26.41% 687.003973ms ] org.activiti.engine.history.NativeHistoricProcessInstanceQuery:list() #290
       +---[1.20% 31.11858ms ] net.rjgf.bg.service.IUserService:getAllUsers() #295
       +---[0.00% 0.014238ms ] net.rjgf.wbwf.nanning.util.UserUtil:getInstance() #296
       +---[0.01% 0.333835ms ] net.rjgf.wbwf.nanning.util.UserUtil:getUserNameAndLoginMap() #296
       +---[0.12% 3.084186ms ] net.rjgf.bg.service.IDeptService:getAllDepts() #297
       +---[0.00% 0.011104ms ] net.rjgf.wbwf.nanning.util.DeptUtil:getInstance() #298
       +---[0.01% 0.166378ms ] net.rjgf.wbwf.nanning.util.DeptUtil:getDeptFullNameMap() #298
       +---[21.58% 561.275484ms ] net.rjgf.wbwf.nanning.service.IActivitiTaskVariableService:selectAllByType() #300
       +---[0.00% 0.017066ms ] net.rjgf.wbwf.nanning.util.VariableUtil:getInstance() #301
       +---[0.27% 6.944702ms ] net.rjgf.wbwf.nanning.util.VariableUtil:getOrderNoMap() #301
       +---[0.00% 0.034451ms ] org.activiti.engine.TaskService:createNativeTaskQuery() #303
       +---[0.00% 0.011501ms ] org.activiti.engine.task.NativeTaskQuery:sql() #303
       +---[4.56% 118.572527ms ] org.activiti.engine.task.NativeTaskQuery:list() #303
       +---[0.00% 0.017192ms ] net.rjgf.wbwf.nanning.util.TaskUtil:getInstance() #304
       +---[0.00% 0.086447ms ] net.rjgf.wbwf.nanning.util.TaskUtil:getTaskMap() #304
       +---[0.22% 5.62728ms ] net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl:getInstanceData() #307
       +---[0.00% 0.085351ms ] org.slf4j.Logger:info() #308
       +---[0.00% 0.029081ms ] org.slf4j.Logger:info() #309
       +---[0.00% 0.009288ms ] net.rjgf.activiti.util.DictionariesUtil:getInstance() #316
       +---[0.00% 0.028683ms ] net.rjgf.activiti.util.DictionariesUtil:getMap() #317
       +---[0.00% 0.002926ms ] net.rjgf.activiti.util.DictionariesUtil:getMap() #318
       +---[0.00% 0.002337ms ] net.rjgf.activiti.util.DictionariesUtil:getMap() #319
       +---[0.00% 0.002699ms ] net.rjgf.activiti.util.DictionariesUtil:getMap() #320
       +---[0.00% 0.002635ms ] net.rjgf.activiti.util.DictionariesUtil:getMap() #321
       +---[0.00% 0.003257ms ] net.rjgf.wbwf.nanning.util.DeptUtil:getInstance() #322
       +---[0.01% 0.169717ms ] net.rjgf.wbwf.nanning.util.DeptUtil:getDeptNameMap() #322
       +---[13.46% 350.26264ms ] net.rjgf.bg.service.IDeptService:getAllUserFullName() #323
       +---[0.00% 0.020828ms ] net.rjgf.wbwf.nanning.util.AreaLinkFullPathUtil:getInstance() #324
       +---[0.02% 0.554155ms ] net.rjgf.wbwf.nanning.repository.IAreaLinkFullPathRepository:getArea() #324
       +---[0.00% 0.112096ms ] net.rjgf.wbwf.nanning.util.AreaLinkFullPathUtil:getAreaFullPathMap() #324
       +---[28.81% 749.382182ms ] net.rjgf.activiti.repository.HiPhotoRepository:selectPhotoForTerminalConstruction() #327
       +---[0.05% 1.427354ms ] net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl:getRecPicListMap() #328
       +---[0.04% 1.151782ms ] org.slf4j.Logger:info() #331

场景二：接口异常报错，猜测是方法入参的问题，或者是某个方法返回值的问题

启动arthas
java -jar arthas-boot.jar
[1]: 11300 wgcloud-server-release.jar
[2]: 11189 starter-1.0-SNAPSHOT.jar
输入数字，选择目标进程
2
找到需要排查的方法信息，主要是类名、方法名
watch net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl saveWorkOrderBasicInfo “{params,returnObj}” -x 2 -b
5 观察输出结果

method=net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl.saveWorkOrderBasicInfo location=AtEnter
ts=2024-03-19 11:56:57; [cost=0.033533ms] result=@ArrayList[
   @Object[][
       @String[SELECT  ACT_HI_PROCINST.* FROM ACT_HI_PROCINST WHERE ACT_HI_PROCINST.PROC_INST_ID_ in(129401859,129538405)],
       @String[SELECT  ACT_HI_TASKINST.* FROM ACT_HI_TASKINST WHERE ACT_HI_TASKINST.PROC_INST_ID_ in(129401859,129538405) ORDER BY START_TIME_ ASC],
       @String[SELECT ACT_RU_TASK.* FROM ACT_RU_TASK WHERE ACT_RU_TASK.PROC_INST_ID_ in(129401859,129538405)],
   ],
   null,
]

-b 表示监控函数调用前的信息，所以只有入参信息
根据结果可以看到方法有三个 String 类型的参数

场景三：线上问题无法定位，需要添加日志打印进行查看，同时又不能随便的更新程序

在本地修改代码添加日志，并生成.class文件
上传.class文件到线上服务器
启动arthas
java -jar arthas-boot.jar
[1]: 11300 wgcloud-server-release.jar
[2]: 11189 starter-1.0-SNAPSHOT.jar
输入数字，选择目标进程
2
重新加载类文件
retransform ./WorkOrderManageServiceImpl.class

retransform success, size: 1, classes:
net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl

7 反编译代码，验证代码是否更新
jad --source-only net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl saveWorkOrderBasicInfo

[arthas@11189]$ jad --source-only net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl saveWorkOrderBasicInfo
       public void saveWorkOrderBasicInfo() {
/*235*/     logger.info("清洗已进入工单列表的建设工单：测试日志, LOCK 状态:{}", (Object)WashLock.WBWF_LOCK);

8 操作完后，恢复到之前的类文件
查看 retransform entry

Id              ClassName       TransformCount  LoaderHash      LoaderClassName 
1               net.rjgf.wbwf.n 2               null            null            
               anning.service.                                                 
               impl.WorkOrderM                                                 
               anageServiceImp                                                 
               l

删除指定 retransform entry

retransform -d 1

删除所有 retransform entry

retransform --deleteAll

场景四：不确定某段逻辑线上代码是不是最新的

启动arthas
java -jar arthas-boot.jar
[1]: 11300 wgcloud-server-release.jar
[2]: 11189 starter-1.0-SNAPSHOT.jar
输入数字，选择目标进程
2
找到需要排查的方法信息，主要是类名、方法名
反编译全类：
jad --source-only net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl
反编译指定方法：
jad --source-only net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl saveWorkOrderBasicInfo
反编译完输出到文件
jad --source-only net.rjgf.wbwf.nanning.service.impl.WorkOrderManageServiceImpl saveWorkOrderBasicInfo > /tmp/WorkOrderManageServiceImpl.java