If you remove it from the config files before installation it will not be there but it was already installed when you removed the config files. You can first delete any instances of it from content manager then delete the feature itself from under Admin > Advanced Tools > Feature Installation
However it won't uninstall the tables and stored procedures from the db.
In a multi site installation you could also make the first site an admin site maybe with an url like admin.yourdomain.com and from there create a child site for www.yourdomain.com and from the admin site you can choose which features are "installed" in child sites.
So really the best way if you don't want the feature at all is to build your own package from source code after having removed the config files for the features you don't want and then package it with UnleashIt
Keep in mind that the files come from mojoPortal.Features.UI/Setup and are copied by post build events to the Web/Setup tree. So if you delete them from Web and not from mojoPortal.Features they will return on post build.
Hope it helps,