Many thanks for the generous beers! Much appreciated.
Basically you need to migrate the files and the database from your local vms to Azure websites.
You can use ftp to upload the files to Azure web sites, and you will need to use a tool like the Sql Azure Migration Wizard to migrate your database to Sql Azure.
Once the files and database are migrated you just change the connection string in the user.config file to point to the new sql azure database.
I'm not sure about running a multi site installation from a single Azure web site. In theory host name based multi sites should be possible as long as the dns /ip for both/each domains can point to the same Azure website installation. But I'm fairly certain the ip addresses are shared with other sites so there would need to be host name header in IIS in Azure for each host. There is probably a way to do that or to get a dedicated fixed ip address, but I can't say that I know off the top of my head.
Folder based multi sites should just work so long as the first site is a root site level site.
In general though, I think Azure hosting is relatively expensive vs traditional shared hosting. In some accounts they give you a free Azure web site but the Sql Azure is not free and tends to be expensive in my experience. If these are low traffic brochure type sites I would probably go with traditional shared hosting. You get more scalability with Azure but it does tend to cost more as a result and if you don't really need the scale then why pay for it.
With traditional hosting you can make a local backup of the db and ask the host to restore it and re-attach a sql user, or you can generate scripts that can be used to migrate both the structure and data into sql scripts that can be executed on the new db.
Hope that helps,