I begin by noting that Skype has been very important to me in the past six months. It is good enough for the calling I do and has literally saved thousands of dollars for our small business - "free" is a good price for multiperson four hour conference calls to Taiwan. It creates opportunity as we do things that were too expensive in the past.
But there are any number of problems with the service. The voice quality of SkypeIn and SkypeOut is poor - much of it strikes me as mobile phone quality where transcoding is taking place. There have been an increasing number of service problems recently. Second attempts have always worked, but it is clearly an issue.
And then there is the interface. I'm always amazed when people sing its praises - particularly when they praise its IM interface. I can only speak to the OS X interface, but would suggest that not a lot of work went into the UI design. The good news is this can be fixed. Skype is "hot' enough these days that someone should be fixing it (actually - the best thing for the OS X client might be for Apple to license it and integrate it with iChat). Particularly maddening is the poor support of headsets.
Martin Geddes has posted a list of features he would like to see..