Memory leaks/corruption with libplist
Reported by Martin S. | April 24th, 2009 @ 03:31 PM
While working on mobile sync with Python, I now get regular memory corruption segfaults when attempting to use "com.apple.Contacts" but can not really locate the exact cause.
It is not related to a particular binary plist parsing failure, but seems to be due to some memory issue. As the mobile sync stuff needs many plists parsed, it might be due to some leak/corruption getting visible once operating on more than a single plist.
A backtrace tells me plist_new_plist_data is the bad guy:
...
*** glibc detected *** python: malloc(): memory corruption: 0xb7f00008 ***
======= Backtrace: =========
/lib/libc.so.6[0xb7e13654]
/lib/libc.so.6[0xb7e15c8a]
/lib/libc.so.6(__libc_calloc+0xf5)[0xb7e175a5]
/usr/lib/libplist.so.0(plist_new_plist_data+0x26)[0xb7b24396]
/usr/lib/libplist.so.0[0xb7b24f45]
/usr/lib/libplist.so.0[0xb7b254d1]
/usr/lib/libplist.so.0(plist_from_bin+0x33a)[0xb7b258fa]
/usr/lib/libiphone.so.0(iphone_msync_recv+0xff)[0xb78a085f]
/usr/lib/python2.6/site-packages/libiphone/_iPhone.so[0xb80d11ae]
/usr/lib/libpython2.6.so.1.0(PyCFunction_Call+0x85)[0xb7fb4875]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x2482)[0xb8008242]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x11b)[0xb800cb9b]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x7b4)[0xb8006574]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x147d)[0xb800723d]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x147d)[0xb800723d]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x11b)[0xb800cb9b]
/usr/lib/libpython2.6.so.1.0[0xb7fa1a29]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0[0xb7f8a9fe]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x4f)[0xb800574f]
/usr/lib/libpython2.6.so.1.0(PyInstance_New+0x6a)[0xb7f8bd1a]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x1f60)[0xb8007d20]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x2ff)[0xb800cd7f]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x63)[0xb8004c73]
/usr/lib/libpython2.6.so.1.0[0xb8026adc]
/usr/lib/libpython2.6.so.1.0(PyRun_FileExFlags+0x9b)[0xb8026b9b]
/usr/lib/libpython2.6.so.1.0(PyRun_SimpleFileExFlags+0x17f)[0xb802758f]
/usr/lib/libpython2.6.so.1.0(PyRun_AnyFileExFlags+0x48)[0xb8027818]
/usr/lib/libpython2.6.so.1.0(Py_Main+0x3b2)[0xb8033e12]
python(main+0x32)[0x8048692]
/lib/libc.so.6(__libc_start_main+0xe5)[0xb7dbd705]
python[0x80485c1]
gdb tells me the same:
# gdb --args python -v -d iphonesync.py com.apple.Contacts
...
Program received signal SIGSEGV, Segmentation fault.
0xb7d1a90e in ?? () from /lib/libc.so.6
(gdb) bt
#0 0xb7d1a90e in ?? () from /lib/libc.so.6
#1 0xb7d1c5a5 in calloc () from /lib/libc.so.6
#2 0xb7a29396 in plist_new_plist_data () from /usr/lib/libplist.so.0
#3 0xb7a29f45 in parse_bin_node () from /usr/lib/libplist.so.0
#4 0x00000019 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Python/SWIG docs say that this happens due to improper calloc/free usage where PyMem_Malloc should be used instead and malloc/calloc memory should never be passed to Python. Thus it might be caused by the memory handling of the bindings, too.
Also a valgrind session revealed some leaks exists in libplist related to the g_node_new() handling, but I am not sure if those are just bad positives due to the clever g_slice allocator.
Attached patch plugs some easy ones in plutil. I hope this ticket can plug more. :)
Comments and changes to this ticket
-
-
Jonathan Beck April 27th, 2009 @ 05:53 PM
- State changed from “new” to “open”
Thanks for the patch Martin, both are applied.
Unfortunately I cannot reproduce your memory corruption. How many contacts do you have on the phone ? Do you have big plist transfert ?
on what architecture do you have this corruption ?
I'm running ubuntu intrepid i386.
-
Martin S. April 28th, 2009 @ 02:46 PM
I actually just have 3 contacts on my iPod Touch, however all have a photo associated which gets transferred in a data section.
It does not seem to be a parsing issue as I have dumped all bplist packets to make sure and used plutil on them without issues.
The corruption happens on openSUSE 11.1/x86 with Python 2.6 and libxml2 2.7.1.
In order to come behind what's wrong I'll check on a different computer and a few different devices later.
As noted, valgrind still shows there are leaks with the g_node stuff.
Did you try to run my mobilesync Python stuff like "./iphonesync.py com.apple.Contacts"? Did it work ok and dump VCards? Do you have any photos associated with contacts?
-
Martin S. April 28th, 2009 @ 06:22 PM
Just tried it on an iPhone with loads of contacts and photos, worked without issues on the same system. :(
Another thought I had is that this might be an issue because I have some Unicode text in my contacts...
You seem to handle PLIST_UNICODE as an extra case in libplist aswell, just mind that from what I saw the Apple specs/tools for plists just treat those as a "regular" string.
Must dig deeper into getting something that can be reproduced...
-
Jonathan Beck April 28th, 2009 @ 06:25 PM
I did add pictures to some of my contacts (3 over 60) and i still don't get a memory corruptions.
for g_node stuff, try with
G_SLICE=always-mallocAnd see if the leaks remain.
I did run ./iphonesync.py com.apple.Contacts, and it dumped all my contacts, photo included. Nice work !
The only thing not working are unicode strings, which are not getting through the SWIG interface. I am looking on a generic way to do that.
-
Jonathan Beck April 28th, 2009 @ 08:31 PM
You can try with last libplist revision, which merge ascii and unicode string handling.
Surprisingly, data is processed well and i get all characters in the vcard dump, without changing anything in python-iphonesync.
However, i still do not have any segfaults :)
Warning : last libplist breaks API, so you need to recompile libiphone and iphone wrapper. Attached is a path that remove deprecated call to PLIST_UNICODE.
-
Martin S. April 29th, 2009 @ 09:49 AM
- State changed from “open” to “resolved”
First, G_SLICE=always-malloc makes valgrind report zero leaks, so there were just false positives due to the g_slice allocator.
Second, something in your last change for libplist regarding the Unicode stuff appears to have cured the memory corruption. Works fine now. Maybe I will try to cherrypick it at some time.
Awesome, thanks!
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.
Create new ticket
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
A project around supporting the iPhone in Linux.
See http://libimobiledevice.org
Jonathan Beck
Martin S.