基本参考twitter的实现,代码很简单,直接调用的breakpad的接口,关键是修改cmake文件比较蛋疼。
需要作部分修改,这里记录下我的操作步骤
1.下载google-breakpad,如果是在r1001之前的版本直接编译安装即可(未尝试),如果是checkout的最新版本,需要把源代码内的my_strchr全部替换掉,俺直接替换成bp_my_strchr,这个会和mysql的定义冲突。
2.配置环境变量breakpad_root,不然breakpad是不会编译到mysql中的:
$echo $breakpad_root
/u01/project/breakpad
3.configure && make && make install
4. backport twitter的patch
有点小不一样,在mysql5.5.20之前的版本,调用函数my_write_minidump是在mysqld.cc的handle_segfault中,而在之后的版本中,为了解决bug#54082,这部分代码独立在单独文件sql/signal_handler.cc文件中,因此需要在函数handle_fatal_signal中调用my_write_minidump
cmake加上 -denabled_embedded_server:bool=on
5.用前几天report的一个bug尝试了一把,可以看到所有线程的堆栈/寄存器信息都被保存了下来,文件大小才629k。
通过minidump_stackwalk解析出来的部分内容如下:
thread 34 (crashed)
0 mysqld!google_breakpad::exceptionhandler::writeminidump [exception_handler.cc : 534 + 0xd]
rbx = 0x0000000000000000 r12 = 0x0000000000000000
r13 = 0x0000000000000000 r14 = 0x0000000000000080
r15 = 0x00007f0834008f10 rip = 0x0000000000991d3c
rsp = 0x00007f0891da4240 rbp = 0x00007f0891da4910
found by: given as instruction pointer in context
1 mysqld!google_breakpad::exceptionhandler::writeminidump [exception_handler.cc : 513 + 0x7]
rbx = 0x00007f0891da48b0 r12 = 0x0000000000000000
r15 = 0x00007f0834008f10 rip = 0x00000000009928ad
rsp = 0x00007f0891da48a0 rbp = 0x00007f0891da4910
found by: call frame info
2 mysqld!my_write_minidump [minidump.cc : 71 + 0x21]
rbx = 0x0000000002af23d0 r12 = 0x00007f0891da497f
r13 = 0x00007f0891da4970 r14 = 0x0000000000000080
r15 = 0x00007f0834008f10 rip = 0x00000000006737dc
rsp = 0x00007f0891da4970 rbp = 0x00007f0891da49a0
3 mysqld!handle_segfault [mysqld.cc : 2566 + 0xb]
rbx = 0x000000000000000b r12 = 0x000000000c57b6b0
r13 = 0x00000000009a167e r14 = 0x0000000000000080
r15 = 0x00007f0834008f10 rip = 0x000000000054e38a
rsp = 0x00007f0891da49b0 rbp = 0x00007f0891da4a30
4 libpthread-2.12.so + 0xf51f
rbx = 0x0000000000008810 r12 = 0x0000000000000000
r13 = 0x0000000000000000 r14 = 0x00000000000000b0
r15 = 0x00007f0834008f10 rip = 0x0000003888c0f520
rsp = 0x00007f0891da4a40 rbp = 0x00007f0891da4f20
5 mysqld!mi_state_info_read [mi_open.c : 958 + 0x2b]
rip = 0x00000000008d1954 rsp = 0x00007f0891da4af0
rbp = 0x00007f0891da4f20
found by: stack scanning
6 mysqld!mi_open [mi_open.c : 210 + 0xf]
rbx = 0x0000000000000000 r12 = 0x00007f0891dae840
r13 = 0x00007f0891dadeb0 r14 = 0x00000000000000b0
rip = 0x00000000008d3832 rsp = 0x00007f0891da4f30
rbp = 0x00007f0891daeb50
7 libc-2.12.so + 0x79adc
rip = 0x0000003888879add rsp = 0x00007f0891da4f70
8 mysqld!my_malloc [my_malloc.c : 38 + 0x7]
rip = 0x000000000077c6e2 rsp = 0x00007f0891da4f90
9 mysqld!my_strdup [my_malloc.c : 147 + 0xd]
rip = 0x000000000077c822 rsp = 0x00007f0891da4fc0
10 mysqld!my_register_filename [mysql_thread.h : 671 + 0x7]
rip = 0x000000000077cfb0 rsp = 0x00007f0891da4ff0
thread 0
0 libc-2.12.so + 0xdd1e3
rbx = 0x0000000000000002 r12 = 0x000000000000003b
r13 = 0x0000000000000012 r14 = 0x00007fffb1eee56c
r15 = 0x000000000c5530c0 rip = 0x00000038888dd1e3
rsp = 0x00007fffb1eee400 rbp = 0x00007fffb1eee5a0
1 mysqld!handle_connections_sockets [mysqld.cc : 5376 + 0x14]
rip = 0x000000000054eaa4 rsp = 0x00007fffb1eee430
rbp = 0x00007fffb1eee5a0
2 mysqld!mysqld_main [mysqld.cc : 4815 + 0x4]
rbx = 0x00007fffb1eee5c0 r12 = 0x0000000000000000
r13 = 0x00007fffb1eee6e0 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x0000000000551ee6
rsp = 0x00007fffb1eee5b0 rbp = 0x00007fffb1eee600
3 libc-2.12.so + 0x1ec9c
rbx = 0x0000000000000000 r12 = 0x0000000000549340
r15 = 0x0000000000000000 rip = 0x000000388881ec9d
rsp = 0x00007fffb1eee610 rbp = 0x0000000000000000
4 mysqld + 0x14942f
rip = 0x0000000000549430 rsp = 0x00007fffb1eee630
……
thread 37
0 libpthread-2.12.so + 0xb44c
rbx = 0x0000000001049b00 r12 = 0x0000000001048de0
r13 = 0x00000000009a4588 r14 = 0x0000000000000879
r15 = 0x0000000000000003 rip = 0x0000003888c0b44c
rsp = 0x00007f08902b5d48 rbp = 0x00007f08902b5df0
1 mysqld!inline_mysql_cond_wait [mysql_thread.h : 980 + 0xa]
rip = 0x000000000054d093 rsp = 0x00007f08902b5d70
rbp = 0x00007f08902b5df0
2 mysqld!cache_thread [mysqld.cc : 2169 + 0x18]
rbx = 0x0000000000000001 r12 = 0x00007f08902b5e58
r13 = 0x00007f08902b69c0 r14 = 0x0000000000000000
r15 = 0x0000000000000003 rip = 0x000000000054e5d9
rsp = 0x00007f08902b5e00 rbp = 0x00007f08902b5e20
3 mysqld!one_thread_per_connection_end [mysqld.cc : 2234 + 0x4]
r15 = 0x0000000000000003 rip = 0x000000000054e79d
rsp = 0x00007f08902b5e30 rbp = 0x00007f08902b5e40
4 mysqld!do_handle_one_connection [sql_connect.cc : 1418 + 0x1f]
r15 = 0x0000000000000003 rip = 0x0000000000638b8c
rsp = 0x00007f08902b5e50 rbp = 0x00007f08902b5e70
5 mysqld!handle_one_connection [sql_connect.cc : 1315 + 0x7]
rbx = 0x000000000c5b1ce0 r12 = 0x0000000001049420
r15 = 0x0000000000000003 rip = 0x0000000000638c30
rsp = 0x00007f08902b5e80 rbp = 0x00007f08902b5e90
6 libpthread-2.12.so + 0x77e0
rbx = 0x0000000000000000 r12 = 0x0000000001049420
r15 = 0x0000000000000003 rip = 0x0000003888c077e1
rsp = 0x00007f08902b5ea0 rbp = 0x0000000000000000
loaded modules:
0x00400000 – 0x00d99fff mysqld ??? (main)
0x3888000000 – 0x388801ffff ld-2.12.so ???
0x3888400000 – 0x3888603fff libdl-2.12.so ???
0x3888800000 – 0x3888b8bfff libc-2.12.so ???
0x3888c00000 – 0x3888e18fff libpthread-2.12.so ???
0x3889400000 – 0x3889683fff libm-2.12.so ???
0x3889800000 – 0x3889a07fff librt-2.12.so ???
0x388b400000 – 0x388b615fff libgcc_s-4.4.5-20110214.so.1 ???
0x388bc00000 – 0x388be5dfff libfreebl3.so ???
0x388c400000 – 0x388c608fff libcrypt-2.12.so ???
0x388e400000 – 0x388e6f0fff libstdc++.so.6.0.13 ???
0x7fffb1fff000 – 0x7fffb1ffffff linux-gate.so ???
可以看到,保留的信息还是相当丰富,足以用于诊断问题,并且文件很小(629k),相应的core file则有6g。
———————————-