記一次MySQL崩潰修複案例，再也不用删庫跑路了

一、問題描述

研究MySQL源代碼，調試并壓測MySQL源代碼時，MySQL崩潰了！問題是它竟然崩潰了！而且還損壞了InnoDB檔案！！還好是在調試環境下發生的，趕緊看看如何解決這個問題，經過一系列的查閱資料、驗證、對比、MySQL源碼調試跟蹤、修複損壞的InnoDB檔案、總結等流程，整理成此文，如果以後真的發生線上上的生産壞境，也不用擔心是不是要跑路的問題了，可以分分鐘搞定MySQL的崩潰問題了！！

檢視錯誤日志，如下：

-----------------------------------------

161108 23:36:45 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/var

2019-12-01 23:36:46 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).

2019-12-01 23:36:46 5497 [Note] Plugin 'FEDERATED' is disabled.

2019-12-01 23:36:46 7f11c48e1720 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.

2019-12-01 23:36:46 5497 [Note] InnoDB: Using atomics to ref count buffer pool pages

2019-12-01 23:36:46 5497 [Note] InnoDB: The InnoDB memory heap is disabled

2019-12-01 23:36:46 5497 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

2019-12-01 23:36:46 5497 [Note] InnoDB: Memory barrier is not used

2019-12-01 23:36:46 5497 [Note] InnoDB: Compressed tables use zlib 1.2.3

2019-12-01 23:36:46 5497 [Note] InnoDB: Using CPU crc32 instructions

2019-12-01 23:36:46 5497 [Note] InnoDB: Initializing buffer pool, size = 16.0M

2019-12-01 23:36:46 5497 [Note] InnoDB: Completed initialization of buffer pool

InnoDB: Database page corruption on disk or a failed

InnoDB: file read of page 5.

InnoDB: You may have to recover from a backup.

2019-12-01 23:36:46 7f11c48e1720 InnoDB: Page dump in ascii and hex (16384 bytes):

 len 16384; hex 7478d078000000050000000000000000000000000f271f4d000700000000000000000000000000000000001b4000000000000000000200f20000000000000006000000000000002d000000000000002e000000000000002f0000000000000030000000000（省略很多類似代碼）

InnoDB: End of page dump

2019-12-01 23:36:46 7f11c48e1720 InnoDB: uncompressed page, stored checksum in field1 1954074744, calculated checksums for field1: crc32 993334256, innodb 2046145943, none 3735928559, stored checksum in field2 1139795846, calculated checksums for field2: crc32 993334256, innodb 1606613742, none 3735928559, page LSN 0 254222157, low 4 bytes of LSN at page end 254221236, page number (if stored to page already) 5, space id (if created with >= MySQL-4.1.1 and stored already) 0

InnoDB: Page may be a transaction system page

InnoDB: Database page corruption on disk or a failed

InnoDB: file read of page 5.

InnoDB: You may have to recover from a backup.

InnoDB: It is also possible that your operating

InnoDB: system has corrupted its own file cache

InnoDB: and rebooting your computer removes the

InnoDB: error.

InnoDB: If the corrupt page is an index page

InnoDB: you can also try to fix the corruption

InnoDB: by dumping, dropping, and reimporting

InnoDB: the corrupt table. You can use CHECK

InnoDB: TABLE to scan your table for corruption.

InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

InnoDB: about forcing recovery.

InnoDB: Ending processing because of a corrupt database page.

2019-12-01 23:36:46 7f11c48e1720  InnoDB: Assertion failure in thread 139714288817952 in file buf0buf.cc line 4201

InnoDB: We intentionally generate a memory trap.

InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

InnoDB: If you get repeated assertion failures or crashes, even

InnoDB: immediately after the mysqld startup, there may be

InnoDB: corruption in the InnoDB tablespace. Please refer to

InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

InnoDB: about forcing recovery.

03:36:46 UTC - mysqld got signal 6 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed,

something is definitely wrong and this may fail.

key_buffer_size=16777216

read_buffer_size=262144

max_used_connections=0

max_threads=1000

thread_count=0

connection_count=0

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 798063 K  bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0 thread_stack 0x40000

/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0x8e64b5]

/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x41b)[0x652fbb]

/lib64/libpthread.so.0(+0xf7e0)[0x7f11c44c77e0]

/lib64/libc.so.6(gsignal+0x35)[0x7f11c315d625]

/lib64/libc.so.6(abort+0x175)[0x7f11c315ee05]

/usr/local/mysql/bin/mysqld[0xa585c5]

/usr/local/mysql/bin/mysqld[0xa6c7b4]

/usr/local/mysql/bin/mysqld[0xa6cbc7]

/usr/local/mysql/bin/mysqld[0xa5bce2]

/usr/local/mysql/bin/mysqld[0xa1e2ba]

/usr/local/mysql/bin/mysqld[0xa0bf60]

/usr/local/mysql/bin/mysqld[0x95a427]

/usr/local/mysql/bin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x48)[0x58f788]

/usr/local/mysql/bin/mysqld[0x6e4a36]

/usr/local/mysql/bin/mysqld(_Z11plugin_initPiPPci+0xb3e)[0x6e826e]

/usr/local/mysql/bin/mysqld[0x582d85]

/usr/local/mysql/bin/mysqld(_Z11mysqld_mainiPPc+0x4d8)[0x587d18]

/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f11c3149d5d]

/usr/local/mysql/bin/mysqld[0x57a019]

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains

information that should help you find out what is causing the crash.

161108 23:36:46 mysqld_safe mysqld from pid file /usr/local/mysql/var/VM_241_49_centos.pid ended

------------------------------------------------------------------------------

二、問題分析

從日志中可以看出是innodb引擎出了問題。日志裡提示到

http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

檢視強制恢複的方法。在mysql的配置檔案my.cnf裡找到 [mysqld]字段下，添加 innodb_force_recovery=1：

[mysqld]
innodb_force_recovery = 1

如果innodb_force_recovery = 1不生效，則可嘗試2——6幾個數字

然後重新開機mysql，重新開機成功。然後使用mysqldump或 pma 導出資料，執行修複操作等。修複完成後，把該參數注釋掉，還原預設值0。

配置檔案的參數：innodb_force_recovery

innodb_force_recovery影響整個InnoDB存儲引擎的恢複狀況。預設為0，表示當需要恢複時執行所有的恢複操作（即校驗資料頁/purge undo/insert buffer merge/rolling back&forward），當不能進行有效的恢複操作時，mysql有可能無法啟動，并記錄錯誤日志；

innodb_force_recovery可以設定為1-6,大的數字包含前面所有數字的影響。當設定參數值大于0後，可以對表進行select,create,drop操作,但insert,update或者delete這類操作是不允許的。

(SRV_FORCE_IGNORE_CORRUPT):忽略檢查到的corrupt頁。
(SRV_FORCE_NO_BACKGROUND):阻止主線程的運作，如主線程需要執行full purge操作，會導緻crash。
(SRV_FORCE_NO_TRX_UNDO):不執行事務復原操作。
(SRV_FORCE_NO_IBUF_MERGE):不執行插入緩沖的合并操作。
(SRV_FORCE_NO_UNDO_LOG_SCAN):不檢視重做日志，InnoDB存儲引擎會将未送出的事務視為已送出。
(SRV_FORCE_NO_LOG_REDO):不執行前滾的操作。

三、解決方案

一般修複方法參考：

第一種方法

建立一張新表：

create table demo_bak  #和原表結構一樣，隻是把INNODB改成了MYISAM。

把資料導進去

insert into demo_bak select * from demo;

删除掉原表：

drop table demo;

注釋掉 innodb_force_recovery 之後，重新開機。

重命名：

rename table demo_bak to demo;

最後改回存儲引擎：

alter table demo engine = innodb

第二種方法

另一個方法是使用mysqldump将表格導出，然後再導回到InnoDB表中。這兩種方法的結果是相同的。

備份導出(包括結構和資料)：

mysqldump -uroot -p123 test > test.sql

還原方法1：

use test;source test.sql

還原方法2（系統指令行）：

mysql -uroot -p123 test < test.sql;

注意，CHECK TABLE指令在InnoDB資料庫中基本上是沒有用的。

第三種方法

1、配置my.cnf

配置innodb_force_recovery = 1或2——6幾個數字，重新開機MySQL

2、導出資料腳本

mysqldump -uroot -p123 test > test.sql

導出SQL腳本。或者用Navicat将所有資料庫/表導入到其他伺服器的資料庫中。

注意：這裡的資料一定要備份成功。然後删除原資料庫中的資料。

3、删除ib_logfile0、ib_logfile1、ibdata1

備份MySQL資料目錄下的ib_logfile0、ib_logfile1、ibdata1三個檔案，然後将這三個檔案删除

4、配置my.cnf

将my.cnf中innodb_force_recovery = 1或2——6幾個數字這行配置删除或者配置為innodb_force_recovery = 0，重新開機MySQL服務

5、将資料導入MySQL資料庫

mysql -uroot -p123 test < test.sql; 或者用Navicat将備份的資料導入到資料庫中。

此種方法下要注意的問題：

ib_logfile0、ib_logfile1、ibdata1這三個檔案一定要先備份後删除；
一定要确認原資料導出成功了
當資料導出成功後，删除原資料庫中的資料時，如果提示不能删除，可在指令行進入MySQL的資料目錄，手動删除相關資料庫的檔案夾或者資料庫檔案夾下的資料表檔案，前提是資料一定導出或備份成功。

記一次MySQL崩潰修複案例，再也不用删庫跑路了

一、問題描述

三、解決方案

繼續閱讀

Ubuntu16.04安裝Apache+MySQL+PHP1. 安裝Apache2. 安裝MySQL3. 安裝PHP4. 安裝phpMyAdmin

Linxu常用指令技巧彙總

ERROR 1 (HY000): Can't create/write to file '/tmp/#sql_4188_1.MYI' (Errcode: 28)

艱難安裝LDAP,SSL認證

《Linux指令行與Shell腳本程式設計大全第2版.布盧姆》pdf

MySQL的4種隔離級别？出現問題

XX系統實施過程問題總結

無元件上傳圖檔到資料庫中，最完整解決方案

【MySQL資料庫】資料庫索引事務1.索引2.事務

neo4j之cypher使用文檔

NOSQL安全攻擊

mybatis_入門程式Mybatis入門

mysql使用source指令導入.sql檔案

登入plsql 報錯 the account is locked --使用者被鎖

sqlServer根據經緯查距離

SequoiaDB巨杉資料庫C++驅動概述