You did not mention if you rebuilt the search index, you must do that to remove content for other features that was indexed before you deleted the files. It takes several minutes for the index to be rebuilt so don't expect instant results.
<add key="DisableSearchFeatureFilters" value="true" /> should remove the feature dropdown from the search page.
We do not have a way to prevent unauthenticated users from searching, however search results are filtered according to the roles that protect the page or feature instance. ie if you set the view permissions in page settings to "Authenticated Users" then the search index will be updated with that role (it may take a few minutes, it doesn't happen instantly). Once the search index is updated with the allowed view roles it will filter out the result if the user is not in the role. So the user can still search but he will not be able to find protected content.
When new users register on the site they are automatically added to the role "Authenticated Users". You can also create custom roles and use those to protect content but then you would have to add users to the roles if you want them to be able to access the content.
Similarly, the menu is filtered according to view roles, so if you protect a page by roles then only users who are in the roles will see the menu item for the page.
Hope that helps,