Web Farm Support

12/21/2009 10:01:48 PM

davidgdean

Total Posts 131

View Posts

Mercatus3

I'm getting close to where I will need web farm support. Joe, I know this has been on your roadmap. Just wondering if it's something you are looking to do in the near future, if not, what it may take to sponsor (you could contact me directly on that one), and if neither of these are possible, what are the areas I need to look to update to code it myself? I'm thinking the issues are around the DB caching but am not sure. I could also use a load balancer that forces sticky sessions. Would that solve many of the issues?

Thanks

12/22/2009 6:49:58 AM

Joe Audette

Total Posts 18439

View Posts

Re: Web Farm Support

Hi David,

There is no issue with database because all nodes would use the same database.

The issues are with the file system, if each node in the farm uses the same file system it should work fine but if they each have different copies there are many issues to solve and they will not be easy to solve. Especially difficult will be solving the search index which is file system based, other issues include user uploads and cache dependency files where we clear the cache by touching files.

I hope to complete web farm support solving these issues sometime in 2010 but it is not coming any time real soon as it is not my top priority.

I would go with as much scaling up with processors and memory on the web server as possible before trying to scale out to more servers. Typically first step is have the db on a separate server cluster as the db is usually a bottle neck long before the web code becomes a bottle neck, and you can scale out the db separately with no issues that I know of. All the challenges t be solved are for scaling out the web server.

Best,

Joe

7/11/2010 10:28:08 PM

bprado

Total Posts 70

View Posts

Re: Web Farm Support

Hi David,

I Did some "digging" in Developers Forums but found no recent news about WebFarm support for mojoportal. Did you make any progress? What exactly is/was your bottleneck (cpu, memory, database)?

7/12/2010 8:32:39 AM

davidgdean

Total Posts 131

View Posts

Mercatus3

Re: Web Farm Support

I was just planning ahead. I've been able to run up to a 200 concurrent users without issue so i put this on hold. Some day I will move to a web farm just for redundancy and eventually increased capacity, but now I'm fine.

2/3/2011 3:03:16 PM

davidgdean

Total Posts 131

View Posts

Mercatus3

Re: Web Farm Support

So. it's months later and it's time for me to implement web farm support. Couple of questions as I go about this:

1) is there a setting in web config or elsewhere that allows me to point the data directory somewhere else (like a network share)? This would allow all servers to access the same data directory.

2) When using a web farm, the SSL is placed on the Load Balancer, and the request, even if secure from the outside to the LB is sent in clear text on the local network. I believe, but have yet to test this. If my assumption is correct, the local server will think the request was not secure even though it is. Is there a way to disable the SSL message (other than to just blank it out) in the web store if no SSL is present on the local server?

Thanks

2/4/2011 1:12:39 PM

Joe Audette

Total Posts 18439

View Posts

Re: Web Farm Support

1. No, but seems like you could create a virtual directory for /Data in IIS and have it point to a network share.

2. I don't see how that can work if the site has no way to know if the request is secured or not, we need to be able to rely on Request.IsSecureConnection. I would think one would install the SSL certificate on each IIS node, not on the load balancer. Of course there may be lots of vendor specific ways of doing things and they may not all work with mojoPortal.

I did not get to working on web farm support in 2010 s I had hoped and I'm still not sure when I will get to it, maybe this year but it is not a top priority for me at the moment.

Best,

Joe

8/1/2011 7:48:14 PM

Winston

Total Posts 1

View Posts

Re: Web Farm Support

Joe,

I'm running into the same issues with the web farm, but thanks to being able to point a virtual Data directory to a folder on a shared drive, mojoPortal works. My question is if it is only the cache dependency file that need to be deleted, can I just have a virtual directory for "system files" pointing to: ~\Data\Sites\1\systemfiles?

I'm not sure if having this structure will affect any other parts of the site. We have an automated build that replaces our skins, so we don't want to have the whole Data directory as shared.

8/2/2011 7:37:42 AM

Joe Audette

Total Posts 18439

View Posts

Re: Web Farm Support

Hi Winston,

Yes that should work as a workaround for now, see a similar discussion on this thread.

I'm actually currently working on changes for better web farm support. I've already completed changes to allow logging to the database instead of the file system and I'm currently working on reducing/eliminating the use of dependency files for clearing the cache.

Best,

Joe

8/9/2011 4:55:04 PM

Lillnor

Total Posts 123

View Posts

/Mats Lillnor

Re: Web Farm Support

Hi!

I just want to share the farm configuration I use with mojoPortal and also other CMS-systems.

I did the infrastructure design for a web-hotel and they now run over 600 websites in this configuration and has done so for more than a year now and it works absolutely great!

There are 2 very powerful physical 2008 R2/IIS 7.5 webservers running in a NLB-cluster. 10 IP-adresses are load-balanced and also there are FTP-sites for each customer that are running in the same NLB-cluster, but with single node failover mode.

Data are replicated between the servers with File Replication Service.

So nice, just upload files to any node and they are instantly replicated across.

mojoportal works flawlessly in this configuration. Some DNN-sites had problems because of some caching issues so the site on one node where not aware of new files that had replicated there, but the DNN-guys fixed that (not sure about what was the problem).

FRS is really nice and with a dedicated Gb-network between the nodes it is rellay quick as well and handles all kind of conflicts.

This is a rather special infrastructure setup and we have created a comprehensive set of custom managing-scripts so they can easily provision new sites and force replication and so on.

/Mats

8/9/2011 4:59:50 PM

Lillnor

Total Posts 123

View Posts

/Mats Lillnor

Re: Web Farm Support

Addition to what i just wrote regarding https-sites.

SSL works perfectly as well with termination on the web-servers. just install the certificates on each node. With NLB there is no loadbalancer that intercepts and dispatches packets. No need to terminate SSL on an ISA-server and forward to the sites over http or anything else like that, just plain SSL that works great. You have to export and import the certificaes of course, but no problems with this at all.

/Mats

8/10/2011 7:01:44 AM

Joe Audette

Total Posts 18439

View Posts

Re: Web Farm Support

Hi Mats,

That's good stuff! I've been surprised how many people are already running mojoPortal in various web farm configurations. That is why I decided its time to work on officially supporting web farms, and that is what I'm focused on right now. I'll be working on documentation also for web farm configurations and would be glad to have help if any community members would like to write tutorials about how they have configured things.

Best,

Joe

9/19/2011 8:53:05 AM

John S

Total Posts 190

View Posts

Re: Web Farm Support

Hi All,

I've done the same thing here (http://www.metrotransit.org) as Mats described, almost. We have the site on two load balanced servers with the MSSQL database on a two node cluster. File Replication Service is the best way to go but wasn't available to me when I set this up, so I used SyncToy to replicate files between the servers, run by a scheduled task once a minute, and it's been working great. Rather than replicating the entire Data folder, I have just a couple of sub folders replicating. The only drawback I've found in hosting on two servers is that each server keeps its own log file, so you just can't go to the logging in the admin and see entries for all servers at the same time. Not really a big deal if you can log into each server to review logs when it's really necessary. (hosts file entries make sure servers resolve to themselves.) I'll be setting up FRS as soon the domain controller on our DMZ gets upgraded. The replication is almost instant, but in the meantime SyncToy has worked like a charm.

John

9/19/2011 9:00:35 AM

Joe Audette

Total Posts 18439

View Posts

Re: Web Farm Support

Hi John,

fyi, the next release of mojoPortal will have support for logging to the database so that both nodes can log to the same place. I'm already using logging to the database on this site.

Best,

Joe

9/21/2011 5:30:33 PM

Lillnor

Total Posts 123

View Posts

/Mats Lillnor

Re: Web Farm Support

We had the same issue with the log-files. We solved it by creating a folder structure for logs with two sub folders, one for each node and configure the sites on the different nodes to log to the specific sub-directory.

Joe. I was wondering about the log to db-option in next release of mojoPortal you mentioned.

How can that be made since logging is part of IIS outside of the application. Can that really be managed inside the application?

How do we access the log-data to use an external log-analyzer? Guess that there will be some configurable statistics pages within the admin sections of mojoPortal?

Will it be possible to log both to file and to the DB?

/Mats

9/22/2011 8:03:39 AM

Joe Audette

Total Posts 18439

View Posts

Re: Web Farm Support

Hi Mats,

I guess you and maybe also John are talking about the IIS logs. I have no solution for that, it is outside the scope of mojoPortal. When I said I have implemented database logging I was talking of the mojoPortal log which only logs errors and other information logged from code. We use log4net for such logging and historically the logging has to been to a file at /Data/currentlog.config and is viewable from Administration > System Log

All I have done is implement a custom logging appender for log4net that logs to the database so that we can have one central error log for all nodes.

From a quick research it doesn't sound like there is a way to log multiple web site nodes to the same IIS log. The suggestions seem to be that one could use logparser to process the separate log files into a database. In fact they say it is better for each node to log separately so that if there is a problem on a specific node it can be identified. Even if there were a way to do it, it would have to be something outside of mojoPortal.

Just on a side note, one interesting tool I've seen lately is this free Log Parser Lizard that provides a nice GUI on top of the Microsoft log parser and allows querying against the IIS log files. Scott Hanselman has a nice blog post about it.

Best,

Joe

12/14/2011 10:15:40 AM

John S

Total Posts 190

View Posts

Re: Web Farm Support

This is kind of a late response, but I was also referring to the mojoPortal log files. The database logging solution has been working great for me. This solves the only unresolved issue I had with the site on two web servers. As for the IIS logs, I use AWStats and merge logs from two different proxy servers for traffic reports.

John