This post will demonstrate a way of obtaining and examining a core dump on Fedora Linux. Core file is a snapshot of working memory of some process. Normally there’s not much use for such a thing, but when it comes to debugging software it’s more than useful. Especially for those hard-to-reproduce random bugs. When your program crashes in such a way, it might be your only source of information, since the problem doesn’t need to come up in the next million executions of your application.
The thing is, creation of core dumps is disabled by default in Fedora, which is fine since the user doesn’t want to have some magic file spawned in his home folder every time an app goes down. But we’re here to fix stuff, so how do you turn it on? Well, there’s couple of thing that might prevent the cores to appear.
1. Permissions
First, make sure that the program has writing permission for the directory it resides in. The core files are created in the directory of the executable. From my experience, core dump creation doesn’t work on programs executed from NTFS drives mounted through ntfs3g.
2. ulimit
This is the place where the core dump creation is disabled. You can see for yourself by using the ulimit command in bash:
astro@desktop:~$ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 15976 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
To enable core dumps set some reasonable size for core files. I usually opt for unlimited, since disk space is not an issue for me:
ulimit -c unlimited
This setting is local only for the current shell though. To keep this settings, you need to put the above line into your ~/.bashrc or (which is cleaner) adjust the limits in /etc/security/limits.conf.
3. Ruling out ABRT
In Fedora, cores are sent to the Automatic Bug Reporting Tool — ABRT. So they can be posted to the RedHat bugzilla to the developers to analyse. The kernel is configured so that all core dumps are pipelined right to abrt. This is set in /proc/sys/kernel/core_pattern. My settings look like this
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t %h %e 636f726500
That means, that all core files are passed to the standard input of abrt-hook-ccpp. Change this settings simply to “core” i.e.:
core
Then the core files will be stored in the same directory as the executable and will be called core.PID.
4. Send Right Signals
Not every process termination leads to dumping core. Keep in mind, that core file will be created only if the process receives this signals:
- SIGSEGV
- SIGFPE
- SIGABRT
- SIGILL
- SIGQUIT
Example program
Here’s a series of steps to test whether your configuration is valid and the cores will appear where they should. You can use this simple program to test it:
/* Print PID and loop. */
#include <stdio.h>
#include <unistd.h>
void infinite_loop(void)
{
while(1);
}
int main(void)
{
printf("PID: %d\n", getpid());
fflush(stdout);
infinite_loop();
return 0;
}
Compile the source, run the program and send a signal like following to get a memory dump:
gcc infinite.c astro@desktop:~$ ./a.out & [1] 19233 PID: 19233 astro@desktop:~$ kill -SEGV 19233 [1]+ Segmentation fault (core dumped) ./a.out astro@desktop:~$ ls core* core.19233
Analysing Core Files
If you already have a core, you can open it using GNU Debugger (gdb). For instance, to open the core file, that was created earlier in this post and displaying a backtrace, do the following:
astro@desktop:~$ gdb a.out core.19233 GNU gdb (GDB) Fedora (7.3.1-47.fc15) Copyright (C) 2011 Free Software Foundation, Inc. Reading symbols from /home/astro/a.out...(no debugging symbols found)...done. [New LWP 19233] Core was generated by `./a.out'. Program terminated with signal 11, Segmentation fault. #0 0x08048447 in infinite_loop () Missing separate debuginfos, use: debuginfo-install glibc-2.14.1-5.i686 (gdb) bt #0 0x08048447 in infinite_loop () #1 0x0804847a in main () (gdb)
Sources
Image may be NSFW.
Clik here to view.
Image may be NSFW.Clik here to view.
