cmd安装linux服务器,linux 服务器安装s3cmd

yum install python-devel python-setuptools -y

easy_install pip

报错：

[root@localhost ~]# easy_install pip

Searching for pip

Reading http://pypi.python.org/simple/pip/

Couldn't find index page for 'pip' (maybe misspelled?)

Scanning index of all packages (this may take a while)

Reading http://pypi.python.org/simple/

No local packages or download links found for pip

error: Could not find suitable distribution for Requirement.parse('pip')

原因是服务器没有安装pip命令

安装pip命令解决此报错(参考博主Linux下安装pip命令)

提示：

一开始cd /usr/lib/python2.6/site-packages/

[root@localhost site-packages]# cd S3

-bash: cd: S3: 没有那个文件或目录

tar xf s3cmd.tar.gz

cd /root/s3cmd-1.6.0

pip install requests s3cmd==1.6.0

[root@localhost s3cmd-1.6.0]# pip install requests s3cmd==1.6.0

Downloading/unpacking requests

Downloading requests-2.18.4-py2.py3-none-any.whl (88kB): 88kB downloaded

Downloading/unpacking s3cmd==1.6.0

Downloading s3cmd-1.6.0.tar.gz (100kB): 100kB downloaded

Running setup.py (path:/tmp/pip_build_root/s3cmd/setup.py) egg_info for package s3cmd

Using xml.etree.ElementTree for XML processing

Downloading/unpacking certifi>=2017.4.17 (from requests)

Downloading certifi-2017.11.5-py2.py3-none-any.whl (330kB): 330kB downloaded

Downloading/unpacking idna>=2.5,<2.7 (from requests)

Downloading idna-2.6-py2.py3-none-any.whl (56kB): 56kB downloaded

Downloading/unpacking chardet>=3.0.2,<3.1.0 (from requests)

Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB): 133kB downloaded

Downloading/unpacking urllib3>=1.21.1,<1.23 (from requests)

Downloading urllib3-1.22-py2.py3-none-any.whl (132kB): 132kB downloaded

Downloading/unpacking python-dateutil (from s3cmd==1.6.0)

Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB): 194kB downloaded

Downloading/unpacking python-magic (from s3cmd==1.6.0)

Downloading python_magic-0.4.15-py2.py3-none-any.whl

Downloading/unpacking six>=1.5 (from python-dateutil->s3cmd==1.6.0)

Downloading six-1.11.0-py2.py3-none-any.whl

Installing collected packages: requests, s3cmd, certifi, idna, chardet, urllib3, python-dateutil, python-magic, six

Running setup.py install for s3cmd

Using xml.etree.ElementTree for XML processing

changing mode of build/scripts-2.6/s3cmd from 644 to 755

changing mode of /usr/bin/s3cmd to 755

Successfully installed requests s3cmd certifi idna chardet urllib3 python-dateutil python-magic six

Cleaning up...

cd /usr/lib/python2.6/site-packages/S3

[root@localhost site-packages]# cd S3/

[root@localhost S3]# ls

AccessLog.py BidirMap.py Config.py Crypto.py ExitCodes.py FileLists.py __init__.py PkgInfo.py S3.py SortedDict.py

AccessLog.pyc BidirMap.pyc Config.pyc Crypto.pyc ExitCodes.pyc FileLists.pyc __init__.pyc PkgInfo.pyc S3.pyc SortedDict.pyc

ACL.py CloudFront.py ConnMan.py Exceptions.py FileDict.py HashCache.py MultiPart.py Progress.py S3Uri.py Utils.py

ACL.pyc CloudFront.pyc ConnMan.pyc Exceptions.pyc FileDict.pyc HashCache.pyc MultiPart.pyc Progress.pyc S3Uri.pyc Utils.pyc

rz -y .s3cfg 上传密钥文件到服务器root目录下。

[root@localhost ~]# ls /root/.s3cfg

/root/.s3cfg

ll /usr/lib/python2.6/site-packages/S3/Config.py

cd /usr/lib/python2.6/site-packages/S3/

cp Config.py Config.py.bak

rz -y Config.py 上传Config.py 文件到服务器/usr/lib/python2.6/site-packages/S3/ 下

cp /root/s3cmd-1.6.0/s3cmd_mult /usr/local/sbin/

cd /usr/local/sbin/

chmod +x s3cmd_mult

同步到云存储的定时脚本

[root@localhost iplog]# cat /data/web/sync.sh

#!/bin/bash

newdate=`date '+%Y-%m-%d'`

IP=`/sbin/ifconfig |grep 'inet addr' |awk -F":" '{print $2}' |awk -F' ' '{print $1}' |awk 'NR==1{print $1}' |awk -F'.' '{a=$3"."$4;print 'a'}'`

for i in `ls /data/log`

do

if [ "$i" != "$newdate" ];then

/usr/local/sbin/s3cmd_mult sync /data/log/"$i"/ s3://gaode/"$i"/cpc/"$IP"/ --no-check-md5 -P --parallel && rm -rf /data/log/$i >/dev/null 2>&1

fi

done

把云端存储的日志文件目录以及目录下的文件拉取到服务器本地：

[root@localhost test001]# s3cmd sync s3://yuncongchu/2018-02-07/cpv/1.248/pv /opt/test001/ -v --no-check-md5

INFO: Retrieving list of remote files for s3://yuncongchu/2018-02-07/cpv/1.248/pv ...

INFO: Compiling list of local files...

INFO: Running stat() and reading/calculating MD5 values on 0 files, this may take some time...

INFO: Found 24 remote files, 0 local files

INFO: Verifying attributes...

INFO: Summary: 24 remote files to download, 0 local files to delete, 0 local files to hardlink

's3://yuncongchu/2018-02-07/cpv/1.248/pv/0.txt' -> '/opt/test001/pv/0.txt' [1 of 24]

76405655 of 76405655 100% in 43s 1713.03 kB/s done

's3://yuncongchu/2018-02-07/cpv/1.248/pv/1.txt' -> '/opt/test001/pv/1.txt' [2 of 24]

56033850 of 56033850 100% in 31s 1713.76 kB/s done

's3://yuncongchu/2018-02-07/cpv/1.248/pv/10.txt' -> '/opt/test001/pv/10.txt' [3 of 24]

19660800 of 80397458 24% in 11s 1703.68 kB/s^CSee ya!

[root@localhost test001]# ll /opt/test001/pv/*

-rw-r--r-- 1 apache apache 76405655 2月 7 23:59 /opt/test001/pv/0.txt

-rw-r--r-- 1 apache apache 56033850 2月 7 01:59 /opt/test001/pv/1.txt

拉取云存储的上的单个日志文件到服务器本地：

[root@localhost test001]# s3cmd get s3://yuncongchu/2018-02-07/cpv/1.248/pv/9.txt ./9.txt

's3://yuncongchu/2018-02-07/cpv/1.248/pv/9.txt' -> './9.txt' [1 of 1]

75481869 of 75481869 100% in 43s 1707.12 kB/s done

[root@localhost test001]# s3cmd get s3://yuncongchu/2018-02-07/cpv/1.248/pv/8.txt ./

's3://yuncongchu/2018-02-07/cpv/1.248/pv/8.txt' -> './8.txt' [1 of 1]

76622153 of 76622153 100% in 42s 1779.85 kB/s done

[root@localhost test001]# ls

8.txt 9.txt pv

查看云存储的上日志文件：

[root@localhost test001]# s3cmd ls s3://yuncongchu/2018-02-07/cpv/1.248/pv/*

2018-02-07 20:19 76405655 s3://yuncongchu/2018-02-07/cpv/1.248/pv/0.txt

2018-02-07 20:18 56033850 s3://yuncongchu/2018-02-07/cpv/1.248/pv/1.txt

[root@localhost test001]# s3cmd ls s3://yuncongchu/2018-02-07/cpv/1.248/*

DIR s3://yuncongchu/2018-02-07/cpv/1.248/count/

DIR s3://yuncongchu/2018-02-07/cpv/1.248/pv/