We don't currently have a way for you to edit the product urls from the ui. If you create new products after changing to extensionless urls the new products will not have the .aspx.
You could actually edit the friendly url record for the products under Administration > Advanced Tools > Url Manager. But you would need to make corresponding changes directly in the database in the ws_Product table.
Hope that helps,