Export then Import site content

If you have questions about using mojoPortal, you can post them here.

You may want to first review our site administration documentation to see if your question is answered there.

This thread is closed to new posts. You must sign in to post in the forums.
1/17/2009 12:27:59 AM
Gravatar
Total Posts 5

#1 for IT Support in Brisbane

Export then Import site content

Hi everyone

Sorry if this has been answered before, but I couldn't find any similar topics.

We have a need to move content between hosts.  The easy way to descibe this is that content will be developed on a content developers workstation and then when done needs to be then moved to the live web site.

Is there currently any easy way of doing this?

Thanks

Lachlan McIntosh

1/17/2009 1:05:44 AM
Gravatar
Total Posts 5

#1 for IT Support in Brisbane

Re: Export then Import site content

It seems that there was a similar request here:
http://www.mojoportal.com/Forums/Thread.aspx?thread=1708&mid=34&pageid=5&ItemID=5

So I guess that no there isn't an easy way at the moment.

If thats the case then if someone could let me know what tables we need to consider in such a migration then I may look at writing a simple utility page for this task myself.

At this point I believe that these would be the main tables:
Modules
ModuleSettings
PageModules
Pages

Plus any images etc.

ID's would need to be remapped based on the PageGuid and ModuleGuid

Would there be anything else?

 

1/17/2009 6:16:10 AM
Gravatar
Total Posts 18439

Re: Export then Import site content

It would need a lot more than that, you would have to do it for feature tables like mp_HtmlContent, mp_Blogs etc., pretty much most tables. I think it will be an overly complex solution and will lead to problems that are hard to figure out.

mojoPortal was not deisgned for pushing content from one server to another. You can stage content right in the site by creating a section of the site that is only visible to content authors and approvers then push the content from a non public page to a public page (ie visible to all users).

I recommend create all content on the production server. For testing get a backup of the production db and files and restore it on a dev machine.

I do not ever recommend pushing from dev or staging to production, production should be the definitive db and dev should use a restored copy of production.

Hope it helps,

Joe

1/17/2009 7:38:44 AM
Gravatar
Total Posts 5

#1 for IT Support in Brisbane

Re: Export then Import site content

 

I’m sure in the days of ITIL becoming flavour of the month, that refusal to even consider a deployment API will disqualify your fine content management system from use by most IT shops.

Every test manager I know would have a brain hemorrhage at the suggestion of making untested changes to a production system and then restoring to development machines for testing. And to be quite honest, right now I'm not far away from one myself!!

If the sites are smaller then I guess that your suggestion would be ok. Maybe I missed something but the impression I got was that mojoportal had larger ambitions.

 

 

 

 

 

1/17/2009 8:04:59 AM
Gravatar
Total Posts 18439

Re: Export then Import site content

To me when discussing testing I'm talking about software testing and there is nothing unusual or unnatural about testing on a copy of production data. In fact in my experience that is the norm.

Now in terms of content review and publishing yes there are systems that do it as you would like, but that involves a good deal more complexity. From a users point of view as long as content can be reviewed and approved I don't see where a multi server or multi installation architecture for publishing is to be considered always better. The vast majority of companies want simplicity and are not running sites on web farms and those are my primary targets. If some mega company wants to use mojoportal and they need this kind of formality of architecture they can sponsor development to get what they need or pursue other solutions. I certainly don't say mojoportal is the best solution for every situation, but its a damned good solution for most businesses.

If you'd like to build an API to deploy content in that fashion feel free, I'm not opposed to it but I think it will be challenging. Perhaps it should just be for the Html module to keep it simpler. Other features like blogs and forums it just doesn't seem to make sense for as the content often comes from the community and enters the system naturally through the production site. So definitely if using those you would want to sync from production to test or staging.

The main challenges as I see them are the fact that images are uploaded to the file system and the urls which are part of the content may be different for the dev and production environments unless you structure the staging site very similar. It does use releative urls for the html module, but the blog for example uses fully qualified urls because theyare needed for outbound rss feeds to correctly show them.

The other challenge that comes to mind is keeping the search index synchronized as it is also file system based.

These same issues also make mojoPortal not quite suitable for web farm deployment. I plan to implement a solution for web farm support but it will be an add on product and will not be free because it will reqwuire a lot of work and most people won't need it.

Best,

Joe

1/17/2009 8:25:24 AM
Gravatar
Total Posts 5

#1 for IT Support in Brisbane

Re: Export then Import site content

It's exactly the community/customer generated content that necessitates the need for a proper deployment API!! 

Imagine....

A marketing content developer, working late on a Friday.

He screws up some new content that they had been working on all week. But thats ok, its not visible to the customers. The go home for the weekend and don't think any more about it.

On Monday they come in and as their friendly IT geek if they can restore their content from last Thursday nights backup.

She says "Sure! But you DO realise that all the blog and forum posts from the weekend will be lost when the backup overwrites the live database! Is that OK?"

You can imaging the look on the marketing person's face.  It would be something like 

I'd hate to think what would happen if multiple customers sites were in a single database and one of them had to have a restore.... 

OK we had planned to write something in any case.  

Basically I'm of the opinion that when a site is operating in this mode that a new release would nuke the existing HTML content WITHOUT touching the community generated content.  

If we didn't care about the community content, we would just restore the staging database over the production database!

However with the advent of platforms such as Azure, I think we all might be having to think much more closely about such issues in the future....

 

 

1/17/2009 8:26:21 AM
Gravatar
Total Posts 18439

Re: Export then Import site content

I think if I were going to implement this the way I would go about it is to implement RESTful web services for the features that need to be published this way. This would make the image handling and search index sync much easier because content coming in via the web service can be handled the same as from the web UI. Then a cross site publishing can be implemented by site to site web service requests and its very de-coupled and much less complex than directly talking to another site's db.

Best,

Joe

1/17/2009 8:39:52 AM
Gravatar
Total Posts 18439

Re: Export then Import site content

To me its exactly the community content and other incoming data that forbid ever restoring data from test to production unless its done very granularly so it can't be done by restoring a db, it would need an api and it would need to be very selective.

Things like new users, web store orders, forum posts all come in from production and should never be restored over from a test machine.

If there is a review process in place content mistakes are going to be minimized, but no matter what there is always a chance of mistakes. If it gets past the review process the mistake still occurs no matter the deployment architecture. If there is a mistake it just needs to be corrected easily as soon as possible. One of the things on my near term road map is content versioning where every edit can be kept and restored and this can mitigate any big content disasters of losing content that had a lot of work put into it.

I also think a more frequent automatic database backup can easily be put in place to mitigate any big disaster and should certainly always be in place for any important site. For large scale mission critical sites real time replication would be in order to ensure no data loss.

Best,

Joe

1/17/2009 9:47:16 AM
Gravatar
Total Posts 18439

Re: Export then Import site content

Sounds like we are in agreement about never restoring a db from dev to production and really have the same concerns but talk about it from different angles.

I am not opposed to an API for cross site publishing workflow for specific content features. We don't have it now but if anyone wants to work on it and willing to implement it as RESTful web services they will have my support. I would even consider implementing it myself if someone wants to sponsor it financially, but short of that its not a high priority for me.

Content versioning is coming in the near future and will make it possible to mitigate most concerns about losing any previous version of content. It will be easy to preview and restore any previous version.

Also a better approval workflow system is planned but currently it is envisioned as self contained in a single site. The combination of this with content versioning to me seems like it would meet most requirements. I will think about how to do this as web services and then maybe the same solution can be configured for cross site workflow.

Best,

Joe

1/17/2009 3:25:03 PM
Gravatar
Total Posts 18439

Re: Export then Import site content

Just for interest I went googling to find out whether other well known open source CMS systems support this kind of publishing scenario from staging to production. It definitely seems these conversations have been going on and its like 2 camps come to a table with different ideas about the way staging is used. This thread in the Drupal forums is quite interesting and parrallels our discussion. One camp, the one apparently I've been in is of the mind that staging is where you test upgrades and can generally experiment and mess around mainly to test the software itself before pushing a software upgrade to production and others are wanting to have staged publishing from staging to production where staging is used primarily to facilitate content review. In any case both camps seem reasonable but nevertheless quite different in the way they are looking at it. Seems like maybe its possible to do this with Drupal if you obtain the right add ons, but they are a much more established project with a much huger community so I don't feel like mojoportal is behind in not having support for this kind of work flow as they don't really have it in the core of Drupal either.

But your points have definitely got me thinking about this scenario and how to suport it efficiently. Perhaps having this functionality would be a way to get a leg up on the competition.

Best,

Joe

1/18/2011 12:15:46 PM
Gravatar
Total Posts 17

Re: Export then Import site content

Joe,

I was scouring your forums on this issue with wanting to export / import a specific site from a multi-site setup to another platform.  I am in the same boat with wanting a feature like this.  Has any feature grown out of this request?  I would tackle this myself, but it seems like a nightmare.

Thanks in advance,

Steve

You must sign in to post in the forums. This thread is closed to new posts.