Work Items are a permanent record
We don't currently provide any means to obliterate individual work items from a project - the only mechanism is to delete an entire project using the DeleteTeamProject tool. This is is largely to support the rich auditing functionality built into Team System, and it also ensures that any links between work items are always preserved.
Historically, this is the way we've dealt with bugs & work items at Microsoft. Work Items are permanent record of work done in a project. It's a way we can be 100% certain that any information gathered regarding the project is always available. For example, we don't rely on email discussions being permanently available - if we have important email discussions about bugs, we'll frequently cut & paste them into the Work Item to ensure that we'll always be able to refer back to the discussion. One of the nice features of Team Foundation over our current internal tool is that formatting is preserved when you paste from e-mail in Team Foundation (our internal tool is plain text) which can make it much easier to review the thread.
Obviously keeping the work items around as a permanent record means that you have to be careful about what you say in a work item - you can never take it back! (I remember this being something impressed upon me very strongly when I first joined Microsoft)
I once heard that someone had proposed marriage via a work item on our internal tool, but it sounds like it could be an urban myth...