~/ means the root of the site so if you are storing it in the skin folder you would need to use the full path to that folder like ~/Data/Sites/1/skins/[skinname]/imagename.gif"
However, we also have a convenience control that you can use instead of <asp:Image
You can use:
<portal:SkinFolderImage id="si1" runat="server" ImageFileName="imagename.gif" />
and it will work if imagename.gif is in the skin folder, it will resolve the path to the skin folder for you.
Hope that helps,