yes by design you can recover an instance because of this. removing a content instance from a page or deleting a  page does not delete the content instance. There are 2 places where you can completely delete a content instance, from the content instance settings page ModuleSettings.aspx or from Administration > Content Manager. 

However a content feature may also have its own tables and data. In a custom feature if you want to clean up data in the content feature specific tables you need to implement a content delete handler that will be called when the instance is really deleted. You can look at examples of this in the code for included features.