SEO strategy where page content comes from a web service

This forum is only for questions or discussions about working with the mojoPortal source code in Visual Studio, obtaining the source code from the repository, developing custom features, etc. If your question is not along these lines this is not the right forum. Please try to post your question in the appropriate forum.

Please do not post questions about design, CSS, or skinning here. Use the Help With Skins Forum for those questions.

This forum is for discussing mojoPortal development

This forum is only for questions or discussions about working with the mojoPortal source code in Visual Studio, obtaining the source code from the repository, developing custom features, etc. If your question is not along these lines this is not the right forum. Please try to post your question in the appropriate forum.

You can monitor commits to the repository from this page. We also recommend developers to subscribe to email notifications in the developer forum as occasionally important things are announced.

Before posting questions here you might want to review the developer documentation.

Do not post questions about design, CSS, or skinning here. Use the Help With Skins Forum for those questions.
This thread is closed to new posts. You must sign in to post in the forums.
11/22/2012 2:13:46 AM
Gravatar
Total Posts 537
feet planted firmly on the ground

SEO strategy where page content comes from a web service

Looking for some advice where a mojoPortal site has custom features to search and display results from a remote web service. We have a feature that collects search criteria, another that parses those criteria, queries the remote web service and displays pages list of results, and another feature that displays each record, again fetching the data from the web service with a record ID. All criteria are passed between the features as URL params, and each feature is configured as to which page contains the feature it needs to redirect to.

But this is not optimum for SEO - there are few direct entry points to the records, and most are behind the search feature which is probably opaque to bots.  We want every record to be googleable. Each record has the same URL with a param for the recordID (though we can also use mp_friendlyurls table to give each an address without querystring params).

We have found that we can gradually seed the mp_pages and mp_friendlyurls tables with the records once they are retrieved from searches. Adding them to mp_pages makes them appear in sitemap.ashx and then presumably get crawled. However there may be 50-100,000 records, and we are concerned this may make other aspects of the system unusable (e.g. Add/Edit Pages would ultimately have 100,000 entries in the list).

We can also/alternatively make our search feature more bot-friendly, so it can find its way to paged results of the entire dataset, then use "prev" and "next" links to help the bots crawl the pages of records. But without putting them in mp_pages, they will not be in the sitemap, so will the SE really index the pages?

Any opinions on good strategies, and whether seeding the mp_pages table is a good idea?

PS another strategy we have considered is writing our own site map handler for these pages, getting its data from a custom table that we seed when records are fetched (so meaning they do not have to added as records in mp_pages).

 

 

11/23/2012 9:13:15 AM
Gravatar
Total Posts 18439

Re: SEO strategy where page content comes from a web service

Hi,

I don't recommend implementing separate features that pass paramters to each other or have any cross talk between them.

I recommend implement a single feature with a module that loads on a cms page and is the entry point to the feature. Then from there you link to supporting pages (physical .aspx files) passing in the page id and module id and any other params you need. And implement a separate sitemap for your feature instead of trying to piggy back on the site map for cms pages. Your site map would have urls to the supporting page that shows the item detail. It sounds like what you have done is try to avoid creating supporting pages and just using more modules on cms pages and linking to cms pages instead of your own supporting pages, that isn't a good way to do it.

This is how all existing features in mojoPortal work. For example study the forums feature, it has several levels of supporting pages, and then the thread.aspx page is the thread detail and we build a site map with the urls for each thread. You could implement freindly urls but that is optional, the forums don't use them, but the blog does. In the case where a feature does use a friendly url the item must also know what the friendly url is so it can use it for its own links, ie the blog post has a database field where is keeps its own copy of the friendly url for the blog post, it doesn't know about the mp_FriendlyUrls table but there must be a row there so that the url rewriter knows to rewrite the friendly url to the real url, which in the case of the blog is /Blog/BlogView.aspx?pageid=x&mid=y&ItemID=x

So you should have a supporting page in your feature that expects all the url params to show a single item, whether you map to that with a friendly url or not is your own choice but is not required in order to have a google site map. If you do use friendly urls then your sitemap should use them like the blog site map does, but the forums for example does not and its sitemap uses the real url with the params though what we did there was combine a few params together since google recommends keeping it to just a few params. So for example now the forums passes only pageid and one extra param that has the thread id and pagenumber of the thread separated by a "~" the page number is just for pagination of longer threads. We derive the module id and forum id from the thread id and then verify those are valid and that the page actually contains the module to enforce security. In older versions we were passing module id, forum id and thread id which was not quite as google friendly.

Hope that helps,

Joe

11/23/2012 10:09:22 AM
Gravatar
Total Posts 537
feet planted firmly on the ground

Re: SEO strategy where page content comes from a web service

Thanks Joe - very useful food for thought.

You must sign in to post in the forums. This thread is closed to new posts.