shell脚本遍历redis集群所有节点日志，并入库到hadoop中，进行数据分析

最新推荐文章于 2025-07-07 16:50:52 发布

原创

最新推荐文章于 2025-07-07 16:50:52 发布 · 482 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#shell脚本遍历redis集群所有节点日志，并入库到had

本文介绍了如何使用shell脚本遍历Redis集群的所有节点日志，将这些日志数据导入到Hadoop数据库中，以便进行进一步的数据分析。关键步骤包括使用kinit进行身份验证，确保在K8S环境中能跨域操作。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

shell脚本遍历redis集群所有节点，并入库到hadoop中的数据库中，进行数据分析，在此留存，以备他用

[AAAAAAnilRedis@FXYY-FS-redis2 ~]$ cat /home/AAAAAAnilRedis/script/fetchRecord.sh
#/bin/bash

export HADOOP_ROOT_LOGGER=INFO,console

date
UNLOCK_FILE="/home/AAAAAAnilRedis/lock/unlock.txt"
LOG_FILE="/home/AAAAAAnilRedis/script/runlog.txt"
rm /home/AAAAAAnilRedis/script/runlog.txt

rm /home/AAAAAAnilRedis/data/otc*
echo "start fetch....." > ${LOG_FILE}
echo `date` >> ${LOG_FILE}
echo `date` >> /home/AAAAAAnilRedis/script/delay.txt

fetch(){

        echo "begin fetching....."
        echo $1

}

fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30001  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc10.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30002  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc11.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30003  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc12.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30004  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc13.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30005  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc14.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30006  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc15.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30007  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc16.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30008  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc17.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31001  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc18.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31002  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc19.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31003  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc20.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31004  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc21.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31005  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc22.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31006  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc23.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31007  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc24.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 31008  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc25.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30009  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc26.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30010  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc27.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 30011  -h 10.252.149.180 > /home/AAAAAAnilRedis/data/otc28.txt &
fetch "keys 'SM_*'" | /home/AAAAAAnilRedis/redis/redis-3.2.1/src/redis-cli -c -p 3