Memory leaks/corruption with libplist
While working on mobile sync with Python, I now get regular memory corruption segfaults when attempting to use "com.apple.Contacts" but can not really locate the exact cause.
It is not related to a particular binary plist parsing failure, but seems to be due to some memory issue. As the mobile sync stuff needs many plists parsed, it might be due to some leak/corruption getting visible once operating on more than a single plist.
A backtrace tells me plist_new_plist_data is the bad guy:
...
*** glibc detected *** python: malloc(): memory corruption: 0xb7f00008 ***
======= Backtrace: =========
/lib/libc.so.6[0xb7e13654]
/lib/libc.so.6[0xb7e15c8a]
/lib/libc.so.6(__libc_calloc+0xf5)[0xb7e175a5]
/usr/lib/libplist.so.0(plist_new_plist_data+0x26)[0xb7b24396]
/usr/lib/libplist.so.0[0xb7b24f45]
/usr/lib/libplist.so.0[0xb7b254d1]
/usr/lib/libplist.so.0(plist_from_bin+0x33a)[0xb7b258fa]
/usr/lib/libiphone.so.0(iphone_msync_recv+0xff)[0xb78a085f]
/usr/lib/python2.6/site-packages/libiphone/_iPhone.so[0xb80d11ae]
/usr/lib/libpython2.6.so.1.0(PyCFunction_Call+0x85)[0xb7fb4875]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x2482)[0xb8008242]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x11b)[0xb800cb9b]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x7b4)[0xb8006574]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x147d)[0xb800723d]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x147d)[0xb800723d]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x11b)[0xb800cb9b]
/usr/lib/libpython2.6.so.1.0[0xb7fa1a29]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0[0xb7f8a9fe]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x4f)[0xb800574f]
/usr/lib/libpython2.6.so.1.0(PyInstance_New+0x6a)[0xb7f8bd1a]
/usr/lib/libpython2.6.so.1.0(PyObject_Call+0x5c)[0xb7f7cc1c]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x1f60)[0xb8007d20]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x2ff)[0xb800cd7f]
/usr/lib/libpython2.6.so.1.0(PyEval_EvalCode+0x63)[0xb8004c73]
/usr/lib/libpython2.6.so.1.0[0xb8026adc]
/usr/lib/libpython2.6.so.1.0(PyRun_FileExFlags+0x9b)[0xb8026b9b]
/usr/lib/libpython2.6.so.1.0(PyRun_SimpleFileExFlags+0x17f)[0xb802758f]
/usr/lib/libpython2.6.so.1.0(PyRun_AnyFileExFlags+0x48)[0xb8027818]
/usr/lib/libpython2.6.so.1.0(Py_Main+0x3b2)[0xb8033e12]
python(main+0x32)[0x8048692]
/lib/libc.so.6(__libc_start_main+0xe5)[0xb7dbd705]
python[0x80485c1]
gdb tells me the same:
# gdb --args python -v -d iphonesync.py com.apple.Contacts
...
Program received signal SIGSEGV, Segmentation fault.
0xb7d1a90e in ?? () from /lib/libc.so.6
(gdb) bt
#0 0xb7d1a90e in ?? () from /lib/libc.so.6
#1 0xb7d1c5a5 in calloc () from /lib/libc.so.6
#2 0xb7a29396 in plist_new_plist_data () from /usr/lib/libplist.so.0
#3 0xb7a29f45 in parse_bin_node () from /usr/lib/libplist.so.0
#4 0x00000019 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Python/SWIG docs say that this happens due to improper calloc/free usage where PyMem_Malloc should be used instead and malloc/calloc memory should never be passed to Python. Thus it might be caused by the memory handling of the bindings, too.
Also a valgrind session revealed some leaks exists in libplist related to the g_node_new() handling, but I am not sure if those are just bad positives due to the clever g_slice allocator.
Attached patch plugs some easy ones in plutil. I hope this ticket can plug more. :)
-
Martin S.Below patch silences 3 warnings.
-
Jonathan Beck- State changed from new to open
Thanks for the patch Martin, both are applied.
Unfortunately I cannot reproduce your memory corruption. How many contacts do you have on the phone ? Do you have big plist transfert ?
on what architecture do you have this corruption ?
I'm running ubuntu intrepid i386.
-
Martin S.
I actually just have 3 contacts on my iPod Touch, however all have a photo associated which gets transferred in a data section.
It does not seem to be a parsing issue as I have dumped all bplist packets to make sure and used plutil on them without issues.
The corruption happens on openSUSE 11.1/x86 with Python 2.6 and libxml2 2.7.1.
In order to come behind what's wrong I'll check on a different computer and a few different devices later.
As noted, valgrind still shows there are leaks with the g_node stuff.
Did you try to run my mobilesync Python stuff like "./iphonesync.py com.apple.Contacts"? Did it work ok and dump VCards? Do you have any photos associated with contacts?
-
Martin S.
Just tried it on an iPhone with loads of contacts and photos, worked without issues on the same system. :(
Another thought I had is that this might be an issue because I have some Unicode text in my contacts...
You seem to handle PLIST_UNICODE as an extra case in libplist aswell, just mind that from what I saw the Apple specs/tools for plists just treat those as a "regular" string.
Must dig deeper into getting something that can be reproduced...
-
Jonathan Beck
I did add pictures to some of my contacts (3 over 60) and i still don't get a memory corruptions.
for g_node stuff, try with
G_SLICE=always-mallocAnd see if the leaks remain.
I did run ./iphonesync.py com.apple.Contacts, and it dumped all my contacts, photo included. Nice work !
The only thing not working are unicode strings, which are not getting through the SWIG interface. I am looking on a generic way to do that.
-
Jonathan Beck
You can try with last libplist revision, which merge ascii and unicode string handling.
Surprisingly, data is processed well and i get all characters in the vcard dump, without changing anything in python-iphonesync.
However, i still do not have any segfaults :)
Warning : last libplist breaks API, so you need to recompile libiphone and iphone wrapper. Attached is a path that remove deprecated call to PLIST_UNICODE.
-
Martin S.
- State changed from open to resolved
First, G_SLICE=always-malloc makes valgrind report zero leaks, so there were just false positives due to the g_slice allocator.
Second, something in your last change for libplist regarding the Unicode stuff appears to have cured the memory corruption. Works fine now. Maybe I will try to cherrypick it at some time.
Awesome, thanks!
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.