Showing posts with label Azure. Show all posts
Showing posts with label Azure. Show all posts

Wednesday, 1 July 2015

Aspnet autoconfig

The defaults of max connection allowed in an asp.net hosted process with machine config autoconfig="true" appears to have changed post .net 4.

Max connections can be set using the system.net max connections element 

  <system.net>
    <connectionManagement>
      <add address="*" maxconnection="50"/>
    </connectionManagement>
  </system.net>


or through System.Net.ServicePointManager.DefaultConnectionLimit in code 

Changes are honored ONLY once autoconfig has been turned OFF 

With autoconfig="true" the max connections now appears to default to max.int and no longer limited to 12*number-of-cores which was the original default.


Monday, 9 March 2015

Azure API Management - APIM, consuming a SOAP WCF service over HTTP


Update 26/March/20 - In the intervening years Azure APIM has come a long way - it's now quite common to see customers of APIM facading SOAP webservices - using both Liquid and XSLT to transform legacy webservices to a more modern RESTful experience. 
Please do checkout the Azure APIM docs.

***

At the time of writing:

I've been looking at the excellent API-Management feature of Azure .. I understand this functionality has come about through an acquisition (Apiphany). 

I think Azure APIM could offer some great benefits. 

After becoming familiar with the product through the documentation I was still a little confused about how or in fact whether a WCF-SOAP service was supported as the back-end service. The APIM documentation focuses very much on a REST based example back-end service.

(Note: I assume the readers familiarity with APIM and a previous walk through of the documented examples).

To add to my confusion one of the feedback comments for the APIM product appeared to suggest SOAP was not in fact YET supported: feedback-for-api-management-suggestions 

However logic suggests it should work, a SOAP envelope submission over HTTP is via a POST, there are really only three prerequisites 

1. Valid SOAP envelope XML which contains an XML body which the target Web service will be able to de-serialize. 
2. A matching SOAPAction request header
3. A Content-Type header of "text/xml"

One further consideration is how the consumer obtains the contract for the service. It's an implementation detail, but it's possible to define an http and/or https endpoint from which the consumer of the service can acquire the service definition in the form of a WSDL. From .net v4.5 onward a "singleWSDL" option is supported which I believe to be generally preferable and better for interop.

There were a couple of goals:

1. Firstly I wanted to look at options around surfacing some existing internally facing WCF implemented basic-http (SOAP) services for the future

2. I wanted policy and the end user experience to centre around both endpoint and operation.

In my test I published a simple test WCF service to a public facing server, I didn't add security to this service - wishing to keep it simple initially - but the service was restricted to TLS from the front end load balancer out. So any test would be over https.

The URL for my test back-end service was something like this

https://ourdomain.com/helloworld/service1.svc?singlewsdl

The service had two methods:

[ServiceContract]
public interface IService1
{
    [OperationContract]
    string HelloWorldOperation1(string value1, int value2, bool value3);
 
    [OperationContract]
    CompositeType HelloWorldOperation2(CompositeType composite);
}

(Composite type wrapped the same basic types passed in operation 1 as params)

After creating an API instance on Azure within our account I added some users, a product and a new API!

1. Publisher Portal, API Management, APIs: Settings

The settings of the API were very basic - I did try and import the WSDL definition from file, out of curiosity and after playing with the encoding to get that right (it didn't like the initial file we copied out of Visual Studio). However quickly got an error around the WSDL not being understood. 
It could have been me, so it might work, I didn't take more than a couple of minutes trying. Certainly at the time of writing WADL and SWAGGER were the two radio option buttons to describe the definition contents type.

Web API Name = "Hello World" 
Description =
Connection = Directly (in a real scenario I'd explore VPN between APIM and back-end, which is beyond this post)

Web Service URLhttps://ourdomain.com/helloworld/service1.svc
Web API URL Suffix = service1Endpoint
Https = [ticked]

This was the resulting Web API URL

https://[ourazuredomain]helloworld.azure-api.net/service1Endpoint

2. Publisher Portal, API Management, APIs: Operations

Here's the interesting bit, APIM really doesn't do much until you add at least one operation and doesn't seem to surface at all right now if you publish an API without any (which is logical).

This is the bit I got stuck with, what to put in the operation section for a SOAP endpoint method?

What I began by doing was adding a single POST method to cover all endpoint methods, thus:

Http Verb = "POST" (I want to post my soap envelope)
Url Template = "/"
Rewrite URL tempate = ""
Display Name = "All operations"

But, I want to be in a position to add a POST operation to the API for each endpoint method, and this endpoint encapsulates ALL the methods, it's the SOAPAction at the back-end service which determines which method gets invoked on the endpoint and then the request XML within the SOAP body must match!

So I went back to the Web Service URL from step 1 and changed it to this:

Web Service URL = https://ourdomain.com/helloworld

(note I've lopped off the "/service1.svc" endpoint part)

Then I went back and edited my API method to look like this:-

Http Verb = "POST" 
Url Template = "/operation1"
Rewrite URL tempate = "/service1.svc"
Display Name = "Operation 1"

The key point here is that we now specify a URL segment to represent the method but we ALWAYS rewrite it to the service endpoint.

The reason for doing this is so that I can specify distinct operations.

But now I've got a further problem - there's nothing that can enforce the fact that I need the right payload for this operation to go to the right method....Yet...

Next I added an additional operation in for the second web service method, identical to that above except for the URL Template ("/operation2") and the Display name "Operation 2".

Finally I added a third method, a GET this time, to allow my consumers a chance to see the WSDL

Http Verb = "GET" 
Url Template = "/GetWsdl"
Rewrite URL tempate = "/service1.svc?wsdl" 
Display Name = "Get WSDL"

(Note: My test server was net4.0 otherwise it'd have been ?singleWsdl)
The important point is that my GET to the exposed URL segment "/GetWsdl" will always return the wsdl for this endpoint as it's re-rewritten as "/service1.svc?wsdl". I.e. with the wsdl URL query param.

Finally, one more step before I publish; need a way to associate the SOAP header to the correct operation - and one more thing I need to ensure I'm sending and receiving with a content type of "text/xml". And not "text/plain".

3. Publisher Portal, API Management, Policies: Policy Scope

I added a policy using the wizard for each of the endpoint operations defined in the steps above, the XML ended up looking like this:

GetWsdl
  


<policies>
 <inbound>
  <set-header exists-action="override" name="content-type">
   <value>text/xml</value>
  </set-header>
  <base></base>
  <rewrite-uri template="/service1.svc?wsdl">
 </rewrite-uri></inbound>
 <outbound>
  <set-header exists-action="override" name="content-type">
   <value>text/xml</value>
  </set-header>
  <base></base>
 </outbound>
</policies>





Operation1
  


<policies>
 <inbound>
  <set-header exists-action="override" name="content-type">
   <value>text/xml</value>
  </set-header>
  <set-header exists-action="override" name="SOAPAction">
   <value>http://tempuri.org/IService1/HelloWorldOperation1</value>
  </set-header>
  <base></base>
  <rewrite-uri template="/service1.svc">
 </rewrite-uri></inbound>
 <outbound>
  <set-header exists-action="override" name="content-type">
   <value>text/xml</value>
  </set-header>
  <base></base>
 </outbound>
</policies>




Operation2
  

<policies>
 <inbound>
  <base></base>
  <set-header exists-action="override" name="content-type">
   <value>text/xml</value>
  </set-header>
  <set-header exists-action="override" name="SOAPAction">
   <value>http://tempuri.org/IService1/HelloWorldOperation2</value>
  </set-header>
  <rewrite-uri template="/service1.svc">
 </rewrite-uri></inbound>
 <outbound>
  <set-header exists-action="override" name="content-type">
   <value>text/xml</value>
  </set-header>
  <base></base>
 </outbound>
</policies>




Given the fact the policies are hierarchical I could have optimized this by moving the content type header up a level.

The next most important part is the SOAPAction for the POST operations.

This ensures the right action goes with the right operation. There's nothing stopping the end user from sending a different action and request content but note the policy is set to OVERRIDE whatever the users sending which means they'd get an error if they sent the wrong payload to the wrong URL.

Clearly there's one quite big problem with all of this: When the user imports the WSDL into a .net application it's going to want to do so under a single endpoint - whereas APIM will want to a distinct URL per method. So there's a mismatch.

One approach could be to define the operations with the differentiator as a query string argument (of the SOAPAction name) instead of the URL segment, here's how operation1 might look like

Http Verb = "POST" 
Url Template = "/?op=HelloWorldOperation1"
Rewrite URL tempate = "/service1.svc"
Display Name = "Operation 1"


Later on that would be called, along with the subscription key instead of:

https://[ourazuresomain]helloworld.azure-api.net/service1Endpoint/operation1/?subscription-key=[ourkey]

It would be


https://[ourazuredomain]helloworld.azure-api.net/service1Endpoint/?op=HelloWorldOperation1&subscription-key=[ourkey]


Now the caller only has to manage the one URL in their web.config (let's assume they're using .net for now).

However this raises yet another point, which is how does the customer ensure that the subscription key gets added in the first place so the APIM façade allows the call through... 

We could ditch the subscription key, it's optional within APIM, but it's one of the things I'd want to use.

As it turns out, the subscription key doesn't have to be passed as a query string param, it can instead be passed via a request header.
The subscription key request header is "ocp-apim-subscription-key".

This might make more sense for a consuming customer, who could configure a set outbound header against their caller/client app to be attached for outgoing requests to our API.

In Summary
The exercise has proved we can relatively easily façade a wcf http SOAP service through APIM and go onto craft a neat developer portal experience, but, it doesn't feel quite right yet. The APIM is currently REST centric and requires a unique URL per operation. This is at odds with SOAP which is a single URL and utilises a SOAP Action to route to the correct operation for the contract the endpoint URL represents. 
The steps detailed above put the onus on the APIM façade consumer to modify the outbound call from client to service [façade] and that doesn't feel right. Also posted a comment here: feedback-azure-api-management-suggestions-soap-support



Wednesday, 18 April 2012

Federated custom tcp binding (and http binding)

This is a work in progress, but so far it allows me to take a token from ACS (Azure Access Control Services) - I followed the basic SDK/MSDN advice to set up a relying party, provider and rules - and send it to a service (acting as Relying Party) by way of authentication.

I was particularly interested to try this with TCP binding and it looks like custom binding is the only way.

For the http I started with the Federated HTTP binding, but since the custom binding took shape, you get more control this way, so I formulated an https equivalent here too.

Note this is for illustration purposes it's certainly not production ready.

1. Get token from ACS
  

        private static SecurityToken GetIdentityProviderToken(string acsEndpoint, string serviceEndpoint)
        {
            var factory =
                new WSTrustChannelFactory(new UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential), acsEndpoint)
                {
                    TrustVersion = TrustVersion.WSTrust13
                };

            factory.Credentials.ClientCertificate.SetCertificate(
                StoreLocation.LocalMachine,
                StoreName.My,
                X509FindType.FindBySubjectName,
                "[cert dns/hostname]");

            var rst = new RequestSecurityToken
            {
                RequestType = RequestTypes.Issue,
                AppliesTo = new EndpointAddress(serviceEndpoint),
                //specify URI realm that ACS token will apply to
                //AppliesTo = new EndpointAddress( new Uri( "urn:federation:customer:222:agent:11" ) ),
                KeyType = KeyTypes.Symmetric
            };

            factory.Credentials.UserName.UserName = ClientUsername;
            factory.Credentials.UserName.Password = ClientPassword;
            var channel = factory.CreateChannel();

            return channel.Issue(rst);
        } 


2. Http Client config (code)
 

        private static ChannelFactory GetCustomHttpBoundService(SecurityToken token, string address)
        {


            var securityBootStrap = SecurityBindingElement.CreateIssuedTokenForCertificateBindingElement(new IssuedSecurityTokenParameters());
            var security = SecurityBindingElement.CreateSecureConversationBindingElement(securityBootStrap, requireCancellation: true);
            
            Console.WriteLine("Include timestamp " + security.IncludeTimestamp);
            Console.WriteLine("Allow insecure transport " + security.AllowInsecureTransport);
            Console.WriteLine("Client: Detect replays " + security.LocalClientSettings.DetectReplays);
            Console.WriteLine("Client: Max clock skew " + security.LocalClientSettings.MaxClockSkew);
            Console.WriteLine("Server: Detect replays " + security.LocalServiceSettings.DetectReplays);
            Console.WriteLine("Server: Max clock skew " + security.LocalServiceSettings.MaxClockSkew);


            var customBinding = new CustomBinding(new List
            {  
                security,
                new BinaryMessageEncodingBindingElement(),
                new HttpsTransportBindingElement()
            });

            var factory = new ChannelFactory(customBinding,
                new EndpointAddress(new Uri(address), EndpointIdentity.CreateDnsIdentity("[cert dns/hostname]")));
            factory.ConfigureChannelFactory();

            Debug.Assert(factory.Credentials != null);
            factory.Credentials.SupportInteractive = false;
            factory.Credentials.ServiceCertificate.SetDefaultCertificate(StoreLocation.LocalMachine,
                StoreName.My,
                X509FindType.FindBySubjectName,
                "[cert dns/hostname]");

            return factory;
        }

3. TCP Client config (code)
  

        private static ChannelFactory GetCustomTcpBoundService(SecurityToken token, string address)
        {

            var securityBootStrap = SecurityBindingElement.CreateIssuedTokenForCertificateBindingElement(new IssuedSecurityTokenParameters());
            var security = SecurityBindingElement.CreateSecureConversationBindingElement(securityBootStrap, requireCancellation: true);
      
            Console.WriteLine("Include timestamp " + security.IncludeTimestamp);
            Console.WriteLine("Allow insecure transport " + security.AllowInsecureTransport);
            Console.WriteLine("Client: Detect replays " + security.LocalClientSettings.DetectReplays);
            Console.WriteLine("Client: Max clock skew " + security.LocalClientSettings.MaxClockSkew);
            Console.WriteLine("Server: Detect replays " + security.LocalServiceSettings.DetectReplays);
            Console.WriteLine("Server: Max clock skew " + security.LocalServiceSettings.MaxClockSkew);

            var customBinding = new CustomBinding(new List
            {  
                security,
                new BinaryMessageEncodingBindingElement(),
                new SslStreamSecurityBindingElement {RequireClientCertificate = false},
                new TcpTransportBindingElement()
            });

            var factory = new ChannelFactory(customBinding,
                new EndpointAddress( new Uri(address),  EndpointIdentity.CreateDnsIdentity("[cert dns/hostname]")));
            factory.ConfigureChannelFactory();
       
            Debug.Assert(factory.Credentials != null);
            factory.Credentials.SupportInteractive = false;
            factory.Credentials.ServiceCertificate.SetDefaultCertificate(StoreLocation.LocalMachine,
                StoreName.My,
                X509FindType.FindBySubjectName,
                "[cert dns/hostname]");

            return factory;
        }


4. Http Server config
  


        <binding name="customfedhttps">
          <security authenticationmode="SecureConversation" requiresecuritycontextcancellation="true">
            <secureconversationbootstrap authenticationmode="IssuedTokenForCertificate">
            </secureconversationbootstrap>
          </security>
          <binarymessageencoding>
          <httpstransport requireclientcertificate="false">
        </httpstransport></binarymessageencoding>
       </binding>

5. TCP Server config
  


        <binding name="customfedtcp">
          <security authenticationmode="SecureConversation" requiresecuritycontextcancellation="true">
            <secureconversationbootstrap authenticationmode="IssuedTokenForCertificate">
            </secureconversationbootstrap>
          </security>
          <binarymessageencoding>
          <sslstreamsecurity requireclientcertificate="false">
          <tcptransport>
        </tcptransport></sslstreamsecurity></binarymessageencoding>
        </binding>

6. Server behaviour
  

    <behaviors>
      <servicebehaviors>
        <behavior name="fedbehaviour">

          <servicemetadata httpsgetenabled="true">
          <federatedservicehostconfiguration>
          
        </federatedservicehostconfiguration></servicemetadata></behavior>
      </servicebehaviors>
    </behaviors>

    <extensions>
      <behaviorextensions>
        <add name="federatedServiceHostConfiguration" type="Microsoft.IdentityModel.Configuration.ConfigureServiceHostBehaviorExtensionElement, Microsoft.IdentityModel, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
      </add></behaviorextensions>
    </extensions>


7. Identity model config
  

  <microsoft.identitymodel>
    <service>
      <audienceuris>
        <add value="https://localhost/Service1.svc">
        <add value="net.tcp://localhost:997/Service2.svc">
      </add></add></audienceuris>

      <servicecertificate>
        <certificatereference findvalue="[cert dns/hostname]" storelocation="LocalMachine" storename="My" x509findtype="FindBySubjectName">
      </certificatereference></servicecertificate>

      <issuernameregistry type="Microsoft.IdentityModel.Tokens.ConfigurationBasedIssuerNameRegistry, Microsoft.IdentityModel, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
        <trustedissuers>
          <add name="[cert dns/hostname]" thumbprint="[cert thumb print]">
        </add></trustedissuers>
      </issuernameregistry>

      <certificatevalidation certificatevalidationmode="None">

      
    </certificatevalidation></service>
  </microsoft.identitymodel>



Useful links:








Sunday, 15 April 2012

Azure Web roles, configuring IIS - basic auth / ssl example

Windows Azure roles have the minimum configuration which covers most needs, but when you need to ensure that your deployment works with additional features some configuration is required. The best place to do this is when the web role starts. In this example I want to ensure that basic authentication is possible with my azure hosted website against a local NT account; so the basic authentication role module needs to be added to IIS7 when the role starts, I also need to unlock the config sections so that my web.config can configure the authentication (disable anonymous, enable basic authentication and allow SSL). Step 1. Create the file in the startup folder, first off allow powershell: enablepowershell.cmd - containing a single line:
  
powershell -command "set-executionpolicy Unrestricted"&

Step 2. A command file to add basic-auth and "unlock" the IIS config sections (easy one to forget!) that I require here:

configiis.cmd
  
ServerManagerCmd.exe -install web-basic-auth 

%windir%\System32\inetsrv\appcmd.exe unlock config /section:system.webServer/security/access 

%windir%\System32\inetsrv\appcmd.exe unlock config /section:system.webServer/security/authentication/anonymousAuthentication 

%windir%\System32\inetsrv\appcmd.exe unlock config /section:system.webServer/security/authentication/basicAuthentication 


The basic principle can be extended to cover other scenarios. Step 3. Finally, some powershell to add a user "AddUser.ps1", in this example I'll configure the basic authentication such that a local user account provides the credentials, but as such an account doesn't yet exist, I will need to create it :

  $username="test"
  $computer = [ADSI]"WinNT://localhost"
  $user_obj = $computer.Create("user", "$username")
  $user_obj.SetPassword("testpassword1!")
  $user_obj.SetInfo()
  Write-Host "$username created."


Plenty of examples out there about how to make this a bit tighter. Now for tying this in with my azure web role. First off I want to up the OS version in my azure config (.cscfg) to version 2 (2008R2 I believe)

<ServiceConfiguration serviceName="AzureMocks" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="2" osVersion="*" >
.
.
</ServiceConfiguration >


Next I need to add some sections to my service definition file (.csdef) This runs the command files under elevated privs.

    <Startup>
      <Task commandLine="Startup\enablepowershell.cmd" executionContext="elevated" />
      <Task commandLine="Startup\configiis.cmd" executionContext="elevated" />
    </Startup >
    <Runtime executionContext="elevated" />

Lastly in my web role start I call the powershell "AddUser.ps1"

            
            try
            {
                Startup.RunPowershellConfig(@".\startup\AddUser.ps1", "AddUser.ps1.txt");

            }
            catch (Exception e)
            {
                RoleDiagnosticsHelper.WriteExceptionToBlobStorage(e, @"An error occured running the powershell startup script '.\startup\AddUser.ps1'");
                return false;
            }
Where those functions are defined as
  

        public static void RunPowershellConfig(string path, string outPath)
        {
            var startInfo = new ProcessStartInfo
            {
                CreateNoWindow = true,
                WindowStyle = ProcessWindowStyle.Hidden,
                FileName = "powershell.exe",
                Arguments = path,
                RedirectStandardOutput = true,
                UseShellExecute = false,
            };

            var writer = new System.IO.StreamWriter(outPath);
            var process = Process.Start(startInfo);

            process.WaitForExit();

            writer.Write(process.StandardOutput.ReadToEnd());
            writer.Close();
        }

        public static void WriteExceptionToBlobStorage(Exception ex, string additionalInfo)
        {
            if (null == additionalInfo)
                additionalInfo = string.Empty;

            var storageAccount = CloudStorageAccount.Parse(
                RoleEnviroment.GetConfigurationSettingValue(
                  "Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"));

            var container = storageAccount.CreateCloudBlobClient()
              .GetContainerReference("rolestartexceptions");
            container.CreateIfNotExist();

            var blob = container.GetBlobReference(string.Format(
              "role-start-exception-{0}-{1}.log",
               RoleEnvironment.CurrentRoleInstance.Id,
               DateTime.UtcNow.ToLongDateString()));
            
            // tostring should include inner exception if exists
            blob.UploadText(ex + " Additional information " + additionalInfo);
        }


I am careful to ensure that I catch and write any exceptions, a problem on role start can be tricky to find and fix otherwise. See cweyers post here
Of course I should not forget, my website config needs the required settings to actually allow SSL and basic auth now all the above are complete


  <system.webServer>
    <security xdt:Transform="Replace">
      <access sslFlags="Ssl" />
      <authentication>
        <anonymousAuthentication enabled="false"/>
        <basicAuthentication enabled="true"/>
      </authentication>
    </security>
  </system.webServer>


Important note
When running all this locally in the emulator that last bit of config can cause some problems, if you spin the site or service up in the emulator the debugger may refuse to attach (we've reported this as a bug under tools version 1.6 Nov 2011 and I believe it's known. The work around is to comment the offending section and when the emulator role starts, go into IIS and configure through the management console. Can use a config transform to ensure the config really does go in for real as this issue appears to be limited to the emulator. So if you see a weird emulator error complaining about invalid site, the config is a good place to start.

Wednesday, 26 October 2011

WCF Hosting in IIS and serviceHost.Open()

We recently had a valid reason for *looking* to see if we could (when hosting WCF in IIS) override the ServiceHost.Open() method - specifically we were looking at the possibility of implementing a retry for certain exceptions thrown on open.


The most obvious course of action was to look at declaring a custom service host factory, reference this from the "factory" attribute in the .svc markup and implement our own "ServiceHost" implementation derived from ServiceHostBase.


The problem is that "open" is defined on the communication channel class and it's not virtual, we thought about redefinition by hiding (declaring a reimplementation of "open" with the the "new" keyword in our service host class) but it would appear that IIS internally takes a reference to serviceHostBase - and redefining that would be a step too far IMO.


So in short it doesn't look like it's (easily) possible, if at all, I guess not many people have a reason to consider doing this with IIS hosted services.


(We did also take a quick look at what might be possible if we hooked up to some of the events around open, but this "felt" wrong then and still does, even though it *might*  be technically possible).


This was all in context of "on-premise" services connecting to azure via the Azure "ServiceBus", in some circumstances we were getting "AddressAlreadyInUseException" - the discussion was around what it might take to implement  a "retry" if the serviceHost.Open() failed with that exception; subsequently we understand that MS intend to release changes to support load balancing in SB - in other words allow more than one connection per URI, so our issue should be resolved with this change.