Web Services Application Pool Recycling

Overview

When you create a new Application Pool in IIS, the default 'recycling' rules are as follows (I have included the 'Process Model' and 'Rapid Fail Protection' sections since I'll be referring to them later):

This means that every 1740 minutes (29 hours), IIS will create a new Application Pool to service requests and, at the same time, attempt to stop the existing pool. This means that for an indefinite amount of time (up to 90 seconds by default), there may be more that one Application Pool servicing the web application. This can be seen from the 'Worker Processes' view in IIS Manager.

Memory Usage

Due to the way the finPOWER Connect Scripting model works, dynamic, in-memory, .NET assemblies are created for each Script (including Portals, Documents etc).

Microsoft's .NET framework does not allow 'unloading' of these dynamically created assemblies from memory hence, if a change is made to a Script, the finPOWER Connect business layer will simply create an additional in-memory assembly to use but, the memory from the old Script's assembly will never be unloaded.

In a desktop scenario, this is rarely likely to cause any issues since there is only ever likely to be one instance of finPOWER Connect running and the user is likely to close down finPOWER Connect at the end of their day.

This is not the case for Web Services though. Business layer pooling means that there may be many instances of the business layer in memory (based on the maximum configured pool size, this could be up to 20). Also, the web server is not likely to be shut down and restarted every day.

By default, as we have seen above, IIS should restart the Application Pool every 29 hours. In many installations, this may be sufficient to prevent memory leakage becoming an issue.

However, it may be that a particular installation has larger, or more dynamic, memory requirements that others. This may be caused by:

• Large global collections such as External Parties
• This simply means that the finPOWER Connect business layer has a larger memory footprint
• Frequent editing of Scripts, Documents or Portals in the production database
• This leads to memory leaks since dynamic .NET assemblies cannot be unloaded from memory

Recycling based on 'Private Memory Limit'

IIS provides a way to recycle an Application Pool when its memory usage exceeds a certain number of Kilobytes (KB).

This limit can be configured to ensure that the Web Services Application Pool never gets out-of-hand.

The figure you set this limit to should be explored by a network administrator and will be based upon factors such as:

• Physical memory size of the web server
• Memory footprint of a single finPOWER Connect business layer
• Certain factors such as a large number of External Parties can affect this significantly
• The configured maximum Web Services pool size

The IIS Worker Processes view can be used to monitor memory usage (note that you must click the refresh button at the top for the figures to update and there are almost certainly better tools for achieving this).

IIS Rapid-Fail Protection

In the first screenshot, there is a section named 'Rapid-Fail Protection'. By default, this is configured as follows:

What this means is (when interpreted using the 'Shutdown Time Limit'):

1. When the application (i.e., Web Services), fails to shut down cleanly within 90 seconds, consider this a 'Fail'
2. If 5 failures (the Maximum Failures value) happen within the space of 5 minutes (the Failure Interval value) then stop the Application Pool!

Now, neither the Web Services or finPOWER Connect Cloud applications currently shut down cleanly (as to why this is is currently under investigation) so, there is a chance that, if the Application Pool is recycled a few times (e.g., 5) in a short period of time, the Application Pool itself with be stopped. Obviously, this is not ideal.

Therefore, with the current versions of Web Services and finPOWER Connect Cloud, it is recommended that 'Rapid-Fail Protection' is disabled.

Conclusion

Web applications are complicated. The finPOWER Connect Web Services, in particular, partly due to their dynamic scripting model, are complicated.

Each installation of Web Services will be different, e.g., different database sizes, different web server memory and processor configurations, therefore there is not a one-size-fits-all solution to configuring Application Pools.

However, based on testing, we recommend the following:

• Web Services and finPOWER Connect Cloud use their own, independent, Application Pools
• The default Recycling time should probably be changed from 1740 minutes to use 'Specific Times', e.g., so in the middle of the night
• Use the IIS 'Worker Processes' view (or other 3rd party tools) to determine how much private memory Web Services consumes and use this figure, in conjunction with the web server's physical memory, to configure the Application Pool's 'Physical Memory Limit' value
• Disable 'Rapid-Fail Protection'