入侵者可能會删除機器的日志資訊
可以檢視日志資訊是否還存在或者是否被清空,相關指令示例:
[[email protected] ~]# ls -h /var/log/*
/var/log/boot.log /var/log/dnf.librepo.log-20210502 /var/log/hawkey.log-20210502
/var/log/boot.log-20200828 /var/log/dnf.librepo.log-20210509 /var/log/hawkey.log-20210509
/var/log/boot.log-20210318 /var/log/dnf.log /var/log/lastlog
/var/log/btmp /var/log/dnf.log.1 /var/log/maillog
/var/log/btmp-20210501 /var/log/dnf.log.2 /var/log/messages
/var/log/cloud-init.log /var/log/dnf.log-20200828 /var/log/qcloud_action.log
/var/log/cloud-init-output.log /var/log/dnf.rpm.log /var/log/secure
/var/log/cron /var/log/dnf.rpm.log-20200828 /var/log/secure-202103281616874601.gz
/var/log/dnf.librepo.log /var/log/hawkey.log /var/log/spooler
/var/log/dnf.librepo.log-20210418 /var/log/hawkey.log-20210418 /var/log/wtmp
/var/log/dnf.librepo.log-20210425 /var/log/hawkey.log-20210425
/var/log/anaconda:
anaconda.log dnf.librepo.log ifcfg.log ks-script-64obidnb.log ks-script-xc9zm2f2.log program.log syslog
dbus.log hawkey.log journal.log ks-script-drwrp_wh.log packaging.log storage.log
/var/log/audit:
audit.log audit.log.1 audit.log.2 audit.log.3
/var/log/chrony:
/var/log/insights-client:
/var/log/journal:
33790f3e0323419f9a055840e9d10b13
/var/log/nginx:
access.log access.log-20210319.gz error.log error.log-20210319.gz
/var/log/private:
/var/log/qemu-ga:
/var/log/samba:
old
/var/log/sssd:
sssd_implicit_files.log sssd_kcm.log sssd.log-20210502.gz sssd_nss.log-20200828.gz
sssd_implicit_files.log-20200828.gz sssd_kcm.log-20210318 sssd.log-20210509 sssd_nss.log-20210318
sssd_implicit_files.log-20210318 sssd.log sssd_nss.log
/var/log/tuned:
tuned.log
[[email protected] ~]# du -sh /var/log/*
56K /var/log/anaconda
29M /var/log/audit
0 /var/log/boot.log
4.0K /var/log/boot.log-20200828
4.0K /var/log/boot.log-20210318
34M /var/log/btmp
184M /var/log/btmp-20210501
4.0K /var/log/chrony
212K /var/log/cloud-init.log
16K /var/log/cloud-init-output.log
14M /var/log/cron
52K /var/log/dnf.librepo.log
136K /var/log/dnf.librepo.log-20210418
136K /var/log/dnf.librepo.log-20210425
136K /var/log/dnf.librepo.log-20210502
132K /var/log/dnf.librepo.log-20210509
112K /var/log/dnf.log
1.1M /var/log/dnf.log.1
1.1M /var/log/dnf.log.2
4.0K /var/log/dnf.log-20200828
192K /var/log/dnf.rpm.log
4.0K /var/log/dnf.rpm.log-20200828
4.0K /var/log/hawkey.log
4.0K /var/log/hawkey.log-20210418
4.0K /var/log/hawkey.log-20210425
4.0K /var/log/hawkey.log-20210502
4.0K /var/log/hawkey.log-20210509
4.0K /var/log/insights-client
1001M /var/log/journal
8.0K /var/log/lastlog
4.0K /var/log/maillog
2.8M /var/log/messages
12K /var/log/nginx
4.0K /var/log/private
4.0K /var/log/qcloud_action.log
4.0K /var/log/qemu-ga
8.0K /var/log/samba
0 /var/log/secure
3.8M /var/log/secure-202103281616874601.gz
4.0K /var/log/spooler
36K /var/log/sssd
8.0K /var/log/tuned
4.0K /var/log/wtmp
入侵者可能建立一個新的存放使用者名及密碼檔案
[[email protected] ~]# ll /etc/passwd*
-rw-r--r-- 1 root root 1578 Mar 18 16:41 /etc/passwd
-rw-r--r--. 1 root root 1516 Mar 18 16:18 /etc/passwd-
[[email protected] ~]# ll /etc/shadow*
---------- 1 root root 799 Mar 18 16:41 /etc/shadow
----------. 1 root root 778 Mar 18 16:18 /etc/shadow-
入侵者可能修改使用者名及密碼檔案
[[email protected] ~]# more /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
systemd-coredump:x:999:997:systemd Core Dumper:/:/sbin/nologin
systemd-resolve:x:193:193:systemd Resolver:/:/sbin/nologin
tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
polkitd:x:998:996:User for polkitd:/:/sbin/nologin
unbound:x:997:994:Unbound DNS resolver:/etc/unbound:/sbin/nologin
libstoragemgmt:x:996:993:daemon account for libstoragemgmt:/var/run/lsm:/sbin/nologin
cockpit-ws:x:995:991:User for cockpit-ws:/:/sbin/nologin
setroubleshoot:x:994:990::/var/lib/setroubleshoot:/sbin/nologin
sssd:x:993:989:User for sssd:/:/sbin/nologin
insights:x:992:988:Red Hat Insights:/var/lib/insights:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
chrony:x:991:987::/var/lib/chrony:/sbin/nologin
tcpdump:x:72:72::/:/sbin/nologin
syslog:x:990:986::/home/syslog:/bin/false
cockpit-wsinstance:x:989:985:User for cockpit-ws instances:/nonexisting:/sbin/nologin
nginx:x:988:984:Nginx web server:/var/lib/nginx:/sbin/nologin
[[email protected] ~]# more /etc/shadow
root:$1$MiAxHcSj$H9Peb.P53VkD4YxSkr6g9.:18704:0:99999:7:::
bin:*:18027:0:99999:7:::
daemon:*:18027:0:99999:7:::
adm:*:18027:0:99999:7:::
lp:*:18027:0:99999:7:::
sync:*:18027:0:99999:7:::
shutdown:*:18027:0:99999:7:::
halt:*:18027:0:99999:7:::
mail:*:18027:0:99999:7:::
operator:*:18027:0:99999:7:::
games:*:18027:0:99999:7:::
ftp:*:18027:0:99999:7:::
nobody:*:18027:0:99999:7:::
dbus:!!:18226::::::
systemd-coredump:!!:18226::::::
systemd-resolve:!!:18226::::::
tss:!!:18226::::::
polkitd:!!:18226::::::
unbound:!!:18226::::::
libstoragemgmt:!!:18226::::::
cockpit-ws:!!:18226::::::
setroubleshoot:!!:18226::::::
sssd:!!:18226::::::
insights:!!:18226::::::
sshd:!!:18226::::::
chrony:!!:18226::::::
tcpdump:!!:18226::::::
syslog:!!:18240::::::
cockpit-wsinstance:!!:18704::::::
nginx:!!:18704::::::
檢視機器最近成功登陸的事件和最後一次不成功的登陸事件
對應日志“/var/log/lastlog”,相關指令示例:
[[email protected] ~]# lastlog
Username Port From Latest
root pts/0 221.217.94.106 Tue May 11 13:31:28 +0800 2021
bin **Never logged in**
daemon **Never logged in**
adm **Never logged in**
lp **Never logged in**
sync **Never logged in**
shutdown **Never logged in**
halt **Never logged in**
mail **Never logged in**
operator **Never logged in**
games **Never logged in**
ftp **Never logged in**
nobody **Never logged in**
dbus **Never logged in**
systemd-coredump **Never logged in**
systemd-resolve **Never logged in**
tss **Never logged in**
polkitd **Never logged in**
unbound **Never logged in**
libstoragemgmt **Never logged in**
cockpit-ws **Never logged in**
setroubleshoot **Never logged in**
sssd **Never logged in**
insights **Never logged in**
sshd **Never logged in**
chrony **Never logged in**
tcpdump **Never logged in**
syslog **Never logged in**
cockpit-wsinstance **Never logged in**
nginx
檢視全部使用者的登入事件
Linux檢視/var/log/wtmp檔案檢視可疑IP登陸
[[email protected] ~]$ last -f /var/log/wtmp
weiyan pts/2 218.94.128.194 Thu Feb 2 13:51 - 13:51 (00:00)
weiyan pts/1 218.94.128.194 Thu Feb 2 13:36 still logged in
weiyan pts/0 218.94.128.194 Thu Feb 2 13:36 still logged in
weiyan pts/4 218.94.128.194 Thu Feb 2 11:30 - 11:33 (00:02)
weiyan pts/3 218.94.128.194 Thu Feb 2 11:30 - 11:33 (00:02)
weiyan pts/6 218.94.128.194 Thu Feb 2 11:12 - 11:12 (00:00)
weiyan pts/3 218.94.128.194 Thu Feb 2 11:12 - 11:12 (00:00)
weiyan pts/2 218.94.128.194 Thu Feb 2 10:59 - 12:22 (01:23)
該日志檔案永久記錄每個使用者登入、登出及系統的啟動、停機的事件。是以随着系統正常運作時間的增加,該檔案的大小也會越來越大,
增加的速度取決于系統使用者登入的次數。該日志檔案可以用來檢視使用者的登入記錄,
last指令就通過通路這個檔案獲得這些資訊,并以反序從後向前顯示使用者的登入記錄,last也能根據使用者、終端tty或時間顯示相應的記錄。
檢視/var/log/secure檔案尋找可疑IP(218.94.128.194)登陸次數
[[email protected] ~]$ sudo cat /var/log/secure|grep 218.94.128.194
Feb 2 10:06:52 VM-6-168-centos sshd[4131]: Accepted password for weiyan from 218.94.128.194 port 6380 ssh2
Feb 2 10:38:57 VM-6-168-centos sshd[8976]: Accepted password for weiyan from 218.94.128.194 port 3308 ssh2
Feb 2 10:38:58 VM-6-168-centos sshd[9026]: Accepted password for weiyan from 218.94.128.194 port 3474 ssh2
Feb 2 10:39:37 VM-6-168-centos sshd[9839]: Accepted password for weiyan from 218.94.128.194 port 4872 ssh2
Feb 2 10:39:42 VM-6-168-centos sshd[10259]: Received disconnect from 218.94.128.194 port 4872:11: disconnected by user
Feb 2 10:39:42 VM-6-168-centos sshd[10259]: Disconnected from 218.94.128.194 port 4872
檢視機器目前登入的全部使用者
對應日志檔案“/var/run/utmp”,相關指令示例:
[[email protected] ~]# who
root pts/0 2021-05-11 13:31 (221.217.94.106)
root pts/1 2021-05-11 13:45 (221.217.94.106)
檢視機器所有使用者的連接配接時間(小時)
對應日志檔案“/var/log/wtmp”,相關指令示例:
[[email protected] ~]# yum install psacct -y
[[email protected] ~]# ac -dp
root 1.47
Mar 18 total 1.47
root 1.07
Today total 1.07
如果發現機器産生了異常流量
可以使用指令“tcpdump”抓取網絡包檢視流量情況或者使用工具”iperf”檢視流量情況
可以檢視/var/log/secure日志檔案
嘗試發現入侵者的資訊,相關指令示例:
[[email protected] ~]# cat /var/log/secure
[[email protected] ~]# cat /var/log/secure |grep -i "accepted password"
查詢異常程序所對應的執行腳本檔案
top指令檢視異常程序對應的PID
在虛拟檔案系統目錄查找該程序的可執行檔案
[[email protected] ~]# ll /proc/68405/ |grep -i exe
lrwxrwxrwx 1 root root 0 Mar 18 16:52 exe -> /usr/lib/jvm/java-11-openjdk-11.0.9.11-
3.el8_3.x86_64/bin/java
[[email protected] ~]#
[[email protected] ~]# ll /usr/lib/jvm/java-11-openjdk-11.0.9.11-3.el8_3.x86_64/bin/java
-rwxr-xr-x 1 root root 16048 Jan 5 01:07 /usr/lib/jvm/java-11-openjdk-11.0.9.11-3.el8_3.x86_64/bin/java
[[email protected] ~]#
如果确認機器已被入侵,重要檔案已被删除,可以嘗試找回被删除的檔案Note:
1、當程序打開了某個檔案時,隻要該程序保持打開該檔案,即使将其删除,它依然存在于磁盤中。這意味着,程序并不知道檔案已經被删除,它仍然可以向打開該檔案時提供給它的檔案描述符進行讀取和寫入。除了該程序之外,這個檔案是不可見的,因為已經删除了其相應的目錄索引節點。
2、在/proc 目錄下,其中包含了反映核心和程序樹的各種檔案。/proc目錄挂載的是在記憶體中所映射的一塊區域,是以這些檔案和目錄并不存在于磁盤中,是以當我們對這些檔案進行讀取和寫入時,實際上是在從記憶體中擷取相關資訊。大多數與 lsof 相關的資訊都存儲于以程序的 PID 命名的目錄中,即 /proc/1234 中包含的是 PID 為 1234 的程序的資訊。每個程序目錄中存在着各種檔案,它們可以使得應用程式簡單地了解程序的記憶體空間、檔案描述符清單、指向磁盤上的檔案的符号連結和其他系統資訊。lsof 程式使用該資訊和其他關于核心内部狀态的資訊來産生其輸出。是以lsof 可以顯示程序的檔案描述符和相關的檔案名等資訊。也就是我們通過通路程序的檔案描述符可以找到該檔案的相關資訊。
3、當系統中的某個檔案被意外地删除了,隻要這個時候系統中還有程序正在通路該檔案,那麼我們就可以通過lsof從/proc目錄下恢複該檔案的内容。
假設入侵者将/var/log/secure檔案删除掉了,嘗試将/var/log/secure檔案恢複的方法可以參考如下:
a.檢視/var/log/secure檔案,發現已經沒有該檔案
[[email protected] ~]# ll /var/log/secure
-rw------- 1 root root 0 Mar 28 03:50 /var/log/secure
### 假設沒有該檔案
b.使用lsof指令檢視目前是否有程序打開/var/log/secure,
[[email protected] ~]# lsof |grep /var/log/secure
rsyslogd 29014 root 7w REG 253,1 268745466 399444 /var/log/secure-202103281616874601 (deleted)
rsyslogd 29014 29016 in:imjour root 7w REG 253,1 268745466 399444 /var/log/secure-202103281616874601 (deleted)
rsyslogd 29014 29017 rs:main root 7w REG 253,1 268745466 399444 /var/log/secure-202103281616874601 (deleted)
c.從上面的資訊可以看到 PID 29014(rsyslogd)打開檔案的檔案描述符為7。同時還可以看到/var/log/ secure已經标記為被删除了。是以我們可以在/proc/29014/fd/7(fd下的每個以數字命名的檔案表示程序對應的檔案描述符)中檢視相應的資訊,如下:
[[email protected] ~]# tail /proc/29014/fd/7
May 11 14:19:22 centos8 sshd[3022959]: Received disconnect from 68.183.84.221 port 54968:11: Bye Bye [preauth]
May 11 14:19:22 centos8 sshd[3022959]: Disconnected from invalid user alldigitalGE_ 68.183.84.221 port 54968 [preauth]
May 11 14:20:12 centos8 sshd[3023088]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=60.191.119.124 user=root
May 11 14:20:14 centos8 sshd[3023088]: Failed password for root from 60.191.119.124 port 64295 ssh2
May 11 14:20:15 centos8 sshd[3023088]: Received disconnect from 60.191.119.124 port 64295:11: Bye Bye [preauth]
May 11 14:20:15 centos8 sshd[3023088]: Disconnected from authenticating user root 60.191.119.124 port 64295 [preauth]
May 11 14:20:24 centos8 sshd[3023106]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=159.203.85.196 user=root
May 11 14:20:26 centos8 sshd[3023106]: Failed password for root from 159.203.85.196 port 58781 ssh2
May 11 14:20:27 centos8 sshd[3023106]: Received disconnect from 159.203.85.196 port 58781:11: Bye Bye [preauth]
May 11 14:20:27 centos8 sshd[3023106]: Disconnected from authenticating user root 159.203.85.196 port 58781 [preauth]
d.從上面的資訊可以看出,檢視/proc/29014/fd/7就可以得到所要恢複的資料。如果可以通過檔案描述符檢視相應的資料,那麼就可以使用I/O重定向将其重定向到檔案中,如:
e.再次檢視/var/log/secure,發現該檔案已經存在。對于許多應用程式,尤其是日志檔案和資料庫,這種恢複删除檔案的方法非常有用。
[[email protected] ~]# ll /var/log/secure
-rw------- 1 root root 0 Mar 28 03:50 /var/log/secure