When analysing and evaluating memory load of concurrent applications a lot of problems may occur: Retrieving information through operating system commands cyclicly you may miss certain peaks. Information functions provided by the runtime library (malloc_info) do not support multi-threading. The memory analysis tool from valgrind called massif scales so badly that it is getting hard to observe the actual application behaviour. Those are the reasons why we have developed a very simple and light-weight memory analysis tool which produces quite less interference. It automatically hooks calls to memory management functions and logs them or determins statistics such as maximum and total memory usage considering time and average memory usage.
This is a thin layer to memory management functions of malloc
to analyse memory management without any manual instrumentation.
This is achieved by small preloaded libraries which intercept calls to
malloc, free and similar.
The package provides two tools for memory analysis:
mmstats: During runtime mmstats gathers statistics to memory
management in the observed application.mmprof: This generates a function call trace with time stamps,
thread id, function id, parameters and return values.
It also includes an offline analysis tool:
mmoa: Reads an mma log file produced by mmprof and calculates the
same statistics as mmstats does, but post mortem.
Valgrind actually provides a tool called 'massif' which does similar things but it has the slightly disadvantage that it produces too much contention in parallel programs which results in bad scalability and therefore does not reflect the actual memory usage of the application.
> make depend > make
Result are two libraries called libmmprof.so and libmmstats.so and a
small test application called test_mmprof.
Once the sources have been compiled you get a library for each tool called libmmprof.so and libmmstats.so (more to come).
Both libraries have to be prepended to the libc e.g. by using the LD_PRELOAD environment variable. The script mma.sh does exactly this. It expects the first command line argument to be the name of the tool (mmprof or mmstats) and the remaining command line is interpreted as the command to be executed with its arguments.
SYNOPSIS:
mma.sh <tool> <command>
Example to use the tool 'mmprof' on the command 'ls *.c':
> mma.sh mmprof ls *.c
The scripts called 'mmprof.sh' and 'mmstats.sh' are just shortcuts to mma.sh
which preselect either mmprof or mmstat as tool.
SYNOPSIS:
mmprof.sh <command> mmstats.sh <command>
> mmprof.sh ls *.c
The selected tool automatically initialises itself and profiles function calls for:
When the last process of a group of related processes (i.e. parent or child process)
exits mmstats writes its statistics in a log file (default: ./mma.log):
The mmprof tool writes for each event the relevant data into the log file. Each line contains one log entry. A log entry describes one event. Events are calls to functions or a simple time stamp to be placed in the log file:
Each log entry consists of a header and the parameters of the event which is the corresponding function that has been called.
The header contains:
The parameters section contains the parameters and return values of the function that has been called or the empty set (nil) in case of the tmstmp event.
mmprof writes either just the value of each element or creates humanized output, which adds a name for each element in the log file (e.g.
<name>'='<value>).
The detailed format of a log file is:
logfile: (<entry> '\n')*
entry: <header> <parameters>
header: <time> ' ' <tid> ' ' <event_type>
parameters: <free_params> | <malloc_params> | <calloc_params> | <realloc_params> | <tmstmp_params>
free_params: <hex> | 'ptr='<hex>
malloc_params: <hex> ' ' <int> | 'ptr=<hex>' ' size='<int>
calloc_params: <hex> ' ' <int> ' ' <int> | 'ptr=' <hex> ' num=' <int> ' elem=' <int>
realloc_params: 'newptr=' <hex> ' ptr=' <hex> ' size=' <int>
tmstmp_params: {}
time: <int> | 'secs='<int> ' nsecs='<int>
tid: <int> | 'tid='<int>
event_type: <type_id> | <type_string>
type_id: {0..4}
type_string: 'free' | 'malloc' | 'calloc' | 'realloc' | 'tmstmp'
If you have access to the source code of the binary you are profiling then you can use the following two functions to suspend/resume profiling. Both functions will add a log entry to mark the time when profiling was suspended or resumed.
Refer to environment variable MMA_START_SUSPENDED to control
behaviour at startup.
int mma_suspend(void)
Suspends profiling.
int mma_resume(void)
Resumes profiling.
Behaviour of the mma tools is customized via environment variables. The following environment variables are considered:
MMA_OUTPUT_FILE:
Default: ./mma.log
Use this environment variable to set another file name for the output.
MMA_OUTPUT_HUMANIZED:
Default: 1
Use this environment to customize the output.
mma generates detailed 'humanized' output per default.
You can disable it by setting this environment variable to zero (0).
Humanized output means:
Non-Humanized output means:
MMA_LIBC_NAME:
Default: "libc.so.6"
Use this environment variable to customize the name of the libc library to be loaded and wrapped dynamically.
Note that you can customize the function names to be instrumented in the makefile!
MMA_START_SUSPENDED:
Default: 0 (false)
Use this environment variable to control, whether the library should suspend logging at start or not. You can manually resume/suspend profiling from source code (see Suspend or Resume Profiling).
To run a simple test you can profile the test application using the following command
> mmprof.sh ./test_mmprof
Note that the output is written to ./mma.log per default. See Section Cutomization above to change log file or format. If the log file is not in humanized form (i.e. MMA_OUTPUT_HUMANIZED=0), you can calculate statistics from the log file using mmoa:
> mmoa mma.log
Alternatively you can directly calculate statistics using mmstats:
> mmstats.sh ./test_mmprof
This software is provided under the GNU LESSER GENERAL PUBLIC LICENSE Version 3, 29 June 2007. A copy of the license statement can be found in the file LICENSE.