Windows provides a recycle bin (AKA waste bin).  It is that little icon on your screen that looks like one those you have around your home or office.  Most (but not all) programs use it.

I like to keep mine in the lower left apart from others on my desktop.  So, if I have worked on your computer, I may have put it there.

When you delete something, it goes into that recycle bin.  If you wished you had not deleted something, it may be there.  Double-click on the icon, find the item, select it, and click Restore this item.  It will go back where it came from.  If the item you delete is really big, it may pop a message to tell you it is to big for the recylce bin.  If it does that, it will ask you if you are sure because it will really remove it.

You can empty your recycle bin by simply right-clicking on it and choosing Empty.   When you do this, there is no return.

Windows Live Mail does not use the Windows Recycle bin.  It has its own “Deleted items” folder.  Find the mail item there and simply drag it back to the folder you want it in.   You need to keep this folder pruned once every 3 to 12 months or so.  When you delete something from this folder, you will be warned and it will be truly gone forever.