Ok, I mispoke, you can also delete a module from the module settings page. But understand this, if you want to remove it from a single page then you can un-publish it from /Admin/ContentManager.aspx using the grid, or delete it completely using the delete button (same as delete button from module settings). Un-publish is the same as deleting it from a page using PageLayout.aspx, ie deleting it from the page in PageLayout.aspx is the same as un-publishing it from the page in content manager, it does not delete the module instance. But the delete button in module settings or the delete button in content manager can completely delete a module instance.
Hope it helps,