libumem - Memory Debugger, Memory Corruption and Memory Leaks

I am currently entangled with a particularly nasty memory corruption on my application. So I am using the libumem memory allocation utility to try and track it down. Since I am not an expert in this area I am going to write this blog in the hope that it helps me retain some information as well as proving useful to others out there.

The first thing I found to be of use was the following blog on sun.com. This gave a decent overview of memory buffer structures.

http://blogs.sun.com/amith/entry/detecting_memory_corruption_with_libumem

My problem manifested itself as a core dump in my application, written in C. I had turned the libumem utility to help me analyse the problem. Here are the steps I followed:-

1) Ran my test.

2) Generated a core file for analysis

gcore

3) Opened the core file for analysis
mdb

4) Ran the ::umem_verify utility and got the following (output here is truncated for clarity):-

umem_alloc_80 72a708 clean
umem_alloc_96 72aa88 1 corrupt buffer
umem_alloc_112 72ae08 clean

5) so I looked a little closer at the corrupted address :-

> 72aa88::umem_verify
Summary for cache 'umem_alloc_96'
buffer fcbe40 (free) seems corrupted, at 0

6) And so I looked event further into the buffer address provided by this command.

7) fcbe40::bufctl_audit

ADDR BUFADDR TIMESTAMP THR LASTLOG CONTENTS CACHE SLAB NEXT DEPTH
00fcbe40 deadbeef deadbeef00000000 deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
0xdeadbeef
libACE.so.5.4.0`__1cGACE_OSEexit6Fi_v_+0x2

8) The following command was then used to print out 10 4-byte hex values starting at the memory address 100197f60 with X setting the format to hex> fcbe40/24X
0xfcbe40: deadbeef deadbeef deadbeef deadbeef deadbeef 0 deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef

Ok, the deadbeef suggests that the memory area has been freed but the '0' which stands out like a sore thumb here suggests that a '0' has been allocated to this area of memory after it has been.

We can ignore the first two 'deadbeefs' here as that represents the metadata part of the memory buffer.

9) What I then did was print out the umalog to see what actions had been carried out on this buffer. The first line here will redirect output to a file (mdb.log)

> ::log -e mdb.log
> ::umalog

10) I then grepped the file for the buffer fcbe40. This actually showed me where the area of memory was last freed. I looked at the structure which was being freed. Each field in the structure was 4 bytes long. So I looked at the fourth field (1 - deadbeef, 2 - deadbeef, 3 - deadbeef, 4 - 0) and then looked at my code to see where it was being used.

Lo and behold, I found that the field was being set after the memory was being freed.

Comodo vs ZoneAlarm firewalls

Well I recently patched ZoneAlarm on my Windows PC (running Windows XP Professional). This was as a result of being informed by ZoneAlarm that a patch was available and should be installed. I duly obliged.

Straight away a major problem was uncovered. The machine refused to boot and always reported an error. The error was NO_MORE_IRP_STACK_LOCATIONS reported via the Blue Screen of Death.

To resolve the issue I carried out the following.

1) Booted Windows XP in Safe Mode
2) Uninstalled ZoneAlarm (simple enough - the option is available under the Programs->ZoneAlarm drop down list).
3) Restarted computer.

What I noticed IMMEDIATELY was my machine ran MUCH faster than it has done for a long time. Now, I simply put my machine's poor performance down to an old processor being asked to do too much but it is now very apparent that it was ZoneAlarm that was causing the grind all along.

Of course, not wanting to be without a software firewall I hunted around and installed Comodo. Even with Comodo in place my old PC is now enjoying a new lease of life.

I would recommend that anyone who is having performance issues with a PC running ZoneAlarm, that they uninstall ZoneAlarm and see for themselves the difference it will make....

The Rich Jerk Scam

It seems that the Rich Jerk (Kelly Felix by name apparently) Scam is so subtle that they are even trying to put fake positive reviews online.

Here are a few links to what seems to be genuine reviews of the 'product'.

Again, the old maxim "If it sounds to good to be true it probably is" is as solid today as it ever was.

Rich Jerk Fake Reviews

Issue with O2's 3G network and the Palm Pre

I have experienced a new(ish) problem with my Palm Pre. It only seems to occur when the Pre has registered with O2's 3G network. The reception for the 3G network in Belfast is flaky to say the least and the problems I am seeing manifest themselves as follows:-

1) Inability to send SMS'. I get the following error message (which is a GSM timeout code I believe).

'Message failed to:XXXXXXXXXXX Could not send your message due to network error. Try again. (code:31)'

2) Inability to receive calls - all incoming calls are diverted straight to voicemail.

3) Inability to make calls - nothing but 'dead air' is heard on the Pre.

The issue is currently being actively discussed on two forums. It has been mentioned on the Palm Pre forum (by yours truly) but no other posters have replied.

PreCentral Forum

O2 Forum

Palm Forum

O2 admit poor customer care...

...well not quite. They didn't explicitly say they were at fault but they did give me a £70 credit on my account. Which is nice but I will still be leaving once my contract expires....in 23 months!

Calls to Palm Pre Going Straight to Voicemail/Unable to send SMS

This is a problem I experienced early on in my use of the Palm. Here is a posting on the (spit) O2 forum that I just sent in to explain what was happening.


I think I may have identified the issue.

I suspect the voicemail/SMS issue is happening because the phone is trying to synchronise with your e-mail accounts as set up on your phone.

There is a section in the Palm Pre user guide (for O2) called "Use another application while on a call" (see page 52).

It basically implies that if you are using a GPRS network (the G icon appears in the top right hand corner of your phone) then there are certain types of applications that cannot be used when you are on a call/attempting to make a call or someone is trying to call you. One of the apps mentioned is e-mail! To be able to make/take calls AND use e-mail you need to be on a Wi-Fi network or on a 3G network.

I tested this this morning. I manually synch'd with my e-mail accounts. At the same time I placed a call to my phone. And guess what - straight to voicemail.

I haven't had as much problems with this issue since I reduced my synch time from 15 minutes to 6 hours. If you are mobile and need constant updates from your e-mail account you may continue to have problems unless you are on 3G and/or have a Wi-Fi connection.

Palm Pre - webOS 1.3.1 upgrade

My Palm Pre successfully updated using my Wifi network at home on Monday night.

All seemed to go smoothly although it seemed to take an age for the update to complete - 30 minutes
perhaps.

Anyway, here are the release notes here for v1.3.1 of webOS released for (spit) O2.

http://kb.palm.com/wps/portal/kb/na/pre/p100eww/o2/solutions/article/71380_en.html#131

Sed script for swapping 2 strings

This is quite a nifty wee sed script - it simply swaps text strings in targeted files within the current directory.

In this case, the files are all C code files (.c extension) and we are swapping TRACE_LOW string and TRACE_HIGH string. TRACE_REPLACE is a temporary string used to help with this.

#!/bin/sh

#for each .c file in the current directory carry out the replacement.
for fl in *.c;
do
echo $fl
echo "Replacing text"

#Take a backup of the file first and then replace LOW with REPLACE
sed -i.backup -e 's/TRACE_LOW/TRACE_REPLACE/g' $fl

#Next, replace HIGH with LOW
sed -i -e 's/TRACE_HIGH/TRACE_LOW/g' $fl

#Finally, replace REPLACE with HIGH
sed -i -e 's/TRACE_REPLACE/TRACE_HIGH/g' $fl
echo "Replacing text...complete"
done

O2 Customer Complaints

I have just sent a letter of complaint to O2.

The addresses it was sent to was:-

ComplaintReviewService@O2.COM.
mycare@o2mail.co.uk

Battery Life on the Palm Pre

I have read some reports that the Palm Pre battery life is quite poor. Indeed one report stated that the user had gone to bed with the phone fully charged but had woken up to find it dead.

Out of interest I noted the following:-
1) Battery charge at 87% at 10.30 pm.
2) Battery charge at 66% at 6.30 am.

Now, I can't explain why people are experiencing bad battery performance in every case but I would suggest the following.

1) If you don't need Wi-Fi or Bluetooth on during the night the turn them both off.
2) Unless you deem it absolutely necessary, there is no need to synchronise with your configured mail accounts every 15 minutes during the night. I have set mine to 6 hours for the night-time period.

Kjots

I am becoming something of a fan of this useful little utility.

Almost like a beefed up Notepad, it is very useful for cutting and pasting those little commands and tit-bits of info that make a developer/administrator/engineer's life a little bit easier.
Especially those commands that you only need every 6 months or so.

I have books filled with such information but hopefully kjots will make it all easier to manager.

http://en.wikipedia.org/wiki/KJots

Reading material

I am currently reading the following books:-

1) Mobile Web 2.0 - The innovator's guide to developing and marketing next generation wireless/mobile applications
2) C++ for Dummies (!)

I am reading the first since the telecommunications industry is constantly evolving and this books comes highly recommended for those who want to understand where mobile communications is heading. In particular it has some focus on the changes in thinking that will be required by the Teleco's if they want to become something other than a mere bit-pipe.

As for the second book I am reading - sometimes it doesn't hurt to read around the fundamentals once in a while :-)

I bought the dummies book a long time ago and never completed it (I treated it more as a reference book back then). Not going through a book cover to cover irks me.....

Palm Pre is in the country

According to the UPS tracking system, my Palm Pre is in Northern Ireland but has yet to leave the depot.

Will I receive it today? Who knows...

Palm Pre User Guide

It came as a bit of a surprise to me that the Palm Pre only came with a quick start guide. If (like me), you like your user manuals to be large, chunky and full of technical detail then you might want to consider downloading something with a bit more meat on it's bones.

http://www.o2.co.uk/deviceinfo/device-pdfs/palmpreeng.pdf

Ok, so its an O2 link and it goes against my beliefs to use anything from O2 but really, the user guide is remarkably similar to several versions out there on the Web.

Palm Pre to Arrive Today

It just so happens I am at home today. Palm have stated that my replacement Pre should arrive today. It feels a little like Xmas as a kid :-)

O2 Customer Care and the Palm Pre Cracked Screen debacle

I decided my first blog should be a rant. O2 have started to sell the Palm Pre smart phone. This highly publicized device has been on the market for about a month now. I bought mine on Saturday the 31st of October. By Friday the 6th of November, 6.30 pm, a number of cracks had developed on the screen as a result of RESTING IN MY FRONT POCKET! The phone has now been rendered useless as the touch screen will no longer work.














The course of events that then took place are as follows:

1) Straight on to Google. It turns out that this is a well known issue with a small number of the Palm Pre devices in the US - where it has been available with Sprint for a number of months.

http://forums.palm.com/palm/board/message?board.id=weboshardware&thread.id=145&view=by_date_ascending&page=29

The Daily Telegraph had also ran an article about the same issue in July.

http://www.telegraph.co.uk/technology/mobile-phones/5731369/Palm-Pre-users-in-US-reporting-handset-problems.html

2) I then contacted O2 Customer Service on 202. I was immediately told this was obviously a case of user abuse and that O2 would not exchange the phone.

3) I then contacted the O2 Store on Castle Lane in Belfast (I think that's the street). I was told by some herbert who sat under a sign crying 'GURU' that this was not a known issue. I then handed him the print out from Google. He said 'but these are just forums'.

Exactly my good man - user reviews posted on a respected forum.

I wonder if he would ignore the same cases that are being documented in O2 OWN forum now.

http://forum.o2.co.uk/viewtopic.php?t=30038&highlight=

I was again told in a rude manner that the phone would not be exchanged.

So O2 now have me tied in to a 2 year deal for a broken phone.


4) I turned to Palm in desperation. They were more helpful and at least not immediately dismissive of me as a customer than O2 were. Currently pictures of the phone are with Palm who are investigating. Given their professional attitude so far I have high hopes of having the Palm Pre replaced.

I will keep the blog updated as I received updates.

But I will sign off by saying that dealing with O2 was, without doubt, the worst customer care experience I have ever had in my life as a consumer.



Update: A few minutes after this post I got a call from Palm.....

Palm have agreed to replace the unit. Well done Palm - customer service as it should be. Professional and thorough - unlike O2 who were simply shocking.

I hope everyone who is in the same boat gets a satisfactory conclusion.

I have also purchase the following to help prevent the same thing happening again.

http://www.amazon.co.uk/exec/obidos/ASIN/B002KQ5EF8/ref=ox_ya_os_product

I think I might devote this blog to both Palm and Palm Pre.....

Update: Out of badness I rang another O2 shop and posed as a potential Palm Pre customer.
I said I had read online about the cracked screen problem. The guy at the other end said he had not heard of the issue but if it happened to me then they would take the phone away for investigation and probably replace it as it sounded like an inherent fault. Interesting. Was this just a line to get me in through the door or was this his genuine position. It certainly was not at the O2 store I originally attended.....

A strong letter of complaint will be winging its way towards O2 very shortl y.