WCF Hosting in IIS and serviceHost.Open()

We recently had a valid reason for *looking* to see if we could (when hosting WCF in IIS) override the ServiceHost.Open() method - specifically we were looking at the possibility of implementing a retry for certain exceptions thrown on open.


The most obvious course of action was to look at declaring a custom service host factory, reference this from the "factory" attribute in the .svc markup and implement our own "ServiceHost" implementation derived from ServiceHostBase.


The problem is that "open" is defined on the communication channel class and it's not virtual, we thought about redefinition by hiding (declaring a reimplementation of "open" with the the "new" keyword in our service host class) but it would appear that IIS internally takes a reference to serviceHostBase - and redefining that would be a step too far IMO.


So in short it doesn't look like it's (easily) possible, if at all, I guess not many people have a reason to consider doing this with IIS hosted services.


(We did also take a quick look at what might be possible if we hooked up to some of the events around open, but this "felt" wrong then and still does, even though it *might*  be technically possible).


This was all in context of "on-premise" services connecting to azure via the Azure "ServiceBus", in some circumstances we were getting "AddressAlreadyInUseException" - the discussion was around what it might take to implement  a "retry" if the serviceHost.Open() failed with that exception; subsequently we understand that MS intend to release changes to support load balancing in SB - in other words allow more than one connection per URI, so our issue should be resolved with this change.

Msbuild and VS solution/project "platform" inconsistency

If you want to build a solution using MSBUILD under a specific configuration you have to specify a valid combination of build-configuration and platform.

For example "Debug|Any CPU"

This is fine, unless you are using a composite target that works on both solution and projects; in our case we were using the solution to build some web projects and their dependent projects and then calling a target to build the web project hosts. (An alternative would have been to build from a project list, but it's convenient to utilise a solution where dependants are resolved auto-magically!)

The problem was that introducing "platform" as a parameter worked for the "build solutions" target - we build under the correct configuration, but it broke the "build web-project-package" target.

The reason is an inconsistency between solutions and projects

Solutions platform Any cpu = "Any CPU", but for projects it's "AnyCPU"

See here

As a side note:

If the solution contains an azure web role project and this project has it's own service configuration then the .csfg file is likely to be called "ServiceConfiguration.[my config name].csfg"

If this is the case you need to pass the parameter "TargetProfile=[My config name]" when building the solution, or else remove the azure web role project from the build list, which may not be desirable.

There is some default behaviour with the azure web role project build target, if a valid value for parameter "TargetProfile" is not passed then the default behaviour is to look for "ServiceConfiguration.Cloud.csfg" first, followed by "ServiceConfiguration.csfg" if neither is found there's an error.

Re-connecting service-bus endpoint - Unexpected - the specified address already exists

Consider two"on premise" WCF service endpoints pointing at a azure service-bus namespace to make them "address-able" to trusted callers going via azure.

Note that the URI are almost the same, except MyContractA has an additional qualification
(a)

<endpoint contract="MyContractA"
binding="netTcpRelayBinding"          address="sb://MyNS.servicebus.windows.net/Company/Area/Service"/>

(b)

<endpoint contract="MyContractB"
binding="netTcpRelayBinding"          address="sb://MyNS.servicebus.windows.net/Company/Area"/>

Remember that each on-premise service that should be address-able via service bus need to be started to establish the connection to the sb relay(which if hosting in app fabric can be accomplished via the warm start feature), but after long periods of inactivity, the connection may be dropped and can be restored by issuing an HTTP Get to each (to re-establish the outbound connection to azure).

What we found was that we'd occasionally get this error trying to re-establish a connection (the services are hosted only once, because at the time of writing Azure SB supports only a single connection per endpoint)

The specified address already exists. Address sb://MyNS.servicebus.windows.net/Company/Area/Service/ already in use.

Which didn't make sense at all in this case.

The thought was that the order in which we "connected, or re-connected" the two services was important, if we started the service with the longer address (a) in example above, first, then no error.



Thanks to Paolo for finding us an explanation of this

Extract from that:-

"Suppose we need to expose the url i.e. just http://.servicebus.windows.net/ e.g. to expose client access policy file through services bus. Then once the base url (as mentioned earlier) is registered, for any further url registration like http://.servicebus.windows.net// appfabric will through the below mentioned error message even if the url is tried to register for the first time.
The specified address already exists.Address sb://.servicebus.windows.net// already in use.
So the resolution is, if needed then first register url like http://.servicebus.windows.net// and then only register the url http://.servicebus.windows.net/ for any appfabric service namespace. "

IIS Express and SSL

There's plenty about this, but still I got caught out for a while this afternoon...

When using IIS express you can host a service expecting SSL, if you want to make calls into the local service it looks like you do have to trust the default localhost cert installed for IISExpress (see inetmgr "server certificates", this can be exported and imported into certmgr.msc)

See here for enabling SSL and a neat workaround for port numbers (iis express doesn't use 443 by default)

And
Jon Simpsons nice post for sending client credentials to a service which then uses ASP net authentication


Update: Another way (but not that safe) is to ingore the local cert trust issue in the client by adding this (in vb in this example)

System.Net.ServicePointManager.ServerCertificateValidationCallback = New System.Net.Security.RemoteCertificateValidationCallback(Function() True)

TCP Relay Binding (azure / service bus)

The azure service bus has 3 modes of operation, tcp, http and auto-detect (correction:- tcp is the default)... This initially confused me; we were testing a simple scenario hosting a service on-premise and invoking it from the cloud via the service bus and relay. We need tcp but were expecting the connection to be refused as the ports were not yet open.
Whilst our firewall logs showed the tcp connection was refused as expected - the connection was then established over http.
Because we need tcp in our scenario, we're going to need to configure the service bus system connectivity mode to tcp (and not the default auto detect).

You can do this in the global asax

ServiceBusEnvironment.SystemConnectivity.Mode = ConnectivityMode.Tcp;


(At time of writing this will avoid a ping to the MS watchdog address when the service starts)


If you're hosting in WCF and can't add global asax then add a custom service behaviour extension

If you have a requirement to authenticate client credentials check out

this post

Which explains how to configure the service to flow client credentials from caller to service where a custom password validator component (very easy to write) can be configured to provide the validation

IIS hosted service "warm up" or "auto start"

The beta extension for IIS has been removed, so far as I can see the other option is calling the service method manually (script) or using the built in feature of "Windows Server App Fabric" if hosting for WF/WCF

See here - iis-application-warm-up-module

Beta site (now closed)

The reason for looking? If you have a service acting as an on-premise end point to be called from a cloud service via the service bus using a relay-binding, that on-premise end point needs to be active (I won't say listening because it's not exactly).

Unhandled Exceptions in IIS

Any single unhandled exception (not an exception thrown in code) - an exception at the CLR/Framework level that occurs during service execution, will not be caught and wrapped as a fault by IIS. Instead the w3wp.exe process will be torn down and any app domains inside will be lost, the app pool will keep running under default settings - the default behaviour of IIS is to re-spawn a new worker process (which is expensive). In this case the client would see ---- not much, a socket error or timeout.



This behaviour was new for .net version 2 and deliberate, it prevents unhandled exceptions of this type getting lost (or “swallowed”), the OS will also attempt 11th hour logging – to the event log – so that you have information to help you debug


For more Info

See here

And here


I didn’t know about the setting: IIS, App pool, Advanced Settings, Rapid-fail protection.


This exists in IIS6 and 7 and it’s designed to protect your server, in the possible event that any “unhandled exception” isn’t a one off, or glitch, after a set number IIS stops the app pool based on those user settings.


The process of recycling is very expensive which means this behaviour is obviously useful on a number of levels. The caller gets “HTTP Error 503. The service is unavailable” once the app pool stops.


It’s actually quite difficult to test this, after all just throwing an exception won’t work, don’t want a fault, and forcing CLR/.net to throw an exception is contrived, but you can test this behaviour in a deployed service by adding the following line into your service

            Environment.FailFast("My message: Killing host");


Note: You can add a catch for AppDomain UnhandledException but (again from .net 2) this won’t stop the app domain crashing.

Notes:

Info on “Watson bucket” problem signatures