Damir Dobric Posts : WCF, AppFabric

Announcing my Session “Connecting Web Systems” at Advanced Developer Conference 2012

Damir Dobric — Sat, 27 Oct 2012 12:45:15 GMT

If you are professional developer or architect who focus Microsoft technologies, then ADC is the conference of great choice. Internationally recognized and legendary experts will present most popular technology topics.
One of best things at this conference I personally like is “NO MARKETING”. All session content is pure experience with pre-filtered sponsoring and marketing noise.

At the upcoming Advanced Developer Conference 2012 in Germany I will give two session. Here is the first look on agenda for “Connecting Web Systems” session. Because this session is not related to some product, it will focus new peaces of technology at the Microsoft’s development platform. This time I decided to talk about technologies which might dramatically change the web soon. However this might also influence the way how we thing about applications and it will definitely change the architecture of future applications. As Integration MVP in last two years I was involved in some more or less hidden and still not well documented technology areas with focus on WebSockets, SignalR and communication in Web in general.
Content presented in this session is pure result of this work. You are welcome.

How to activate WCF on IIS8 and Windows 8?

Damir Dobric — Mon, 24 Sep 2012 15:03:18 GMT

When you start working with WCF on Windows 8 and IIS8, you might run into following problem.

HTTP Error 404.3 - Not Found

The page you are requesting cannot be served because of the extension configuration. If the page is a script, add a handler. If the file should be downloaded, add a MIME map.

To make this working you have enable (install/activate) Advanced .NET 4.5 Features, which are mostly related to WCF:

ServiceBus Message Broker Service is starting and starting…(Cannot Deploy SPS Workflow)

Damir Dobric — Tue, 18 Sep 2012 05:21:00 GMT

When working with Windows Azure Workflow, Workflow Host Manager or Share Point 2013 Workflow you might experience several problems related to Message Broker Service.

Issues related to broker In SharePoint are manifested through workflow deployment issues. For example, if you try to deploy the workflow you might get following error:

Error 1 Error occurred in deployment step 'Activate Features': System.IO.InvalidDataException: A response was returned that did not come from the workflow service. Status code = 503:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Service Unavailable</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Service Unavailable</h2>
<hr><p>HTTP Error 503. The service is unavailable.</p>

If you are rather Service Bus guy, you will figure out that Broker Service does not start. It is all the time in starting status. In both cases, take a look on status of Windows Fabric Host Service. You will probably notice that this service is not running. If so start it, stop the broker and start the broker again.
If you cannot stop the broker service while starting, got to task manager and kill Microsoft.ServiceBus Message Broker process. Then start the broker service.

Is Service Bus Server an ESB product?

Damir Dobric — Wed, 01 Aug 2012 06:23:00 GMT

Service Bus 1.0 (also called Service Bus Server) is a Microsoft messaging product, which many people will compare with Enterprise Service Bus. Currently Microsoft is NOT positioning Service Bus Server as an ESB product. The reason for that is that Service Bus doesn’t support many of features which are supported by typical ESB. For example a common service bus supports features like We don’t have any of the ‘other’ ESB features like: protocol conversion, message transformation, adapters (in term of BizTalk), business monitoring etc.

Service Bus is for now a messaging runtime focused on rich messaging patterns like pub-sub & queues. Moreover it guaranties on-prem and cloud symmetry.

To recap, Service Bus is not an ESB product but rather a messaging platform.

An interesting combination of Service Bus Server and Workflow enables however easy implementation of highly scalable workflow based and long running scenarios, which make place for new innovative solutions.

For more information take a look here.

Installing Workflow 1.0 BETA with Workflow Host

Damir Dobric — Tue, 31 Jul 2012 07:34:36 GMT

First of all download all components from http://www.microsoft.com/en-us/download/details.aspx?id=30369 To install the Service Bus run workflow.3f.3f.3fnew.exe.
Another possibility would be to use Web Platform installer.

After start of installation setup will detect dependent components. Please note that restart will be required assuming that .NET 4.5 has not been previously installed. It is important to understand that Workflow Host has a dependency to Service Bus.

If you previously didn’t install Service Bus you can proceed with installation of Workflow Manager directly. Setup will automatically recognize missing dependencies.

Click I Accept to proceed. Please also note that Workflow used in Workflow 1.0 BETA (the name might be changed future) is built on top of .NET 4.5. This means that this version of Workflow is almost identical with the common WF 4.5. This Workflow BETA 1 contains few more activities which support activation of workflow instances via service bus message. Moreover, current version does not support Messaging activities like ReceiveMessage. This is an important architecture issue you should be aware of!

After installation of .NET 4.5 you will have to restart the machine.

At the end following appears:

After all Workflow will appear in start menu:

Last but not least, do not forget to install Workflow Tools for Visual Studio.

If you don’t install them new activities will not appear in designer and design time APIs cannot be uses (i.e. publishing of workflow).

Installing Service Bus Server 1.0 BETA

Damir Dobric — Tue, 24 Jul 2012 13:35:00 GMT

In July 2012 Microsoft has for the first time published a peace of Windows Azure which is installable locally (On-Premise). It is Service Bus (v1 BETA) which is slightly adapted version of Windows Azure Service Bus. If you are aware of this product this post will help you to install it. If you are new to Service Bus Server (this is how some people call it) it is highly recommended to take a look on this article.

Before you start with installation go to following URL and download all required components from http://www.microsoft.com/en-us/download/details.aspx?id=30376. To install the Service Bus run servicebus.3f.3f.3fnew.exe. Another option to do that is to use Web Platform installer v4. Following picture shows two new components in Platform Installer. This post is related to installation of Service Bus 1.0. If you want to install Workflow 1.0 you will anyhow have to first install Service Bus.

After start of Service Bus Server setup following dialog appears:

Immediately after start the list of missing dependencies will appear (see picture below). Please note that after installation restart will be required! Note that if you previously have installed Windows Fabric component you will have to uninstall it. Note that it is not necessary to go to direct link (shown below). Simply click "I Accept" to proceed. Setup will automatically download and install required components.

After installation of Service management Components 3.0 you will have to restart the machine.

After restart the installation will proceed automatically. At the end the start menu will contain the node “Windows Azure Service Bus” :

How to uninstall Service Bus Server and Workflow Host?

Damir Dobric — Thu, 19 Jul 2012 05:42:00 GMT

This post is related to BETA version of Service Bus Server and Workflow Host which is used in Share Point 2013. It should help you to uninstall previous version of Service Bus and Workflow.

Use the following procedure to remove Workflow and Service Bus from a node in a farm. You must perform these actions on every node from which you want to delete Workflow and Service Bus.

1. Go to stare menu locate AppFabric Server and start configuration wizard. Remove the node from the farm as described in "Leaving a Farm". If you have lot of time you can also use PowerShell.

2. Open the Control Panel windows and uninstall following components:

Microsoft AppFabric

Microsoft Windows Fabric

After these steps ServiceBus and Workflow including Windows Azure Fabric which is running locally are removed. However all databases will remain at the SQL server. If you want to remove them, you can/should first make a backup.

You can use SQL Management Studio to remove the databases that were associated with Workflow and Service Bus.
Following databases are related to SB and WF Host:

a. AppFabricManagementDB

b. SbGatewayDatabase

c. SBContainer01

d. WFInstanceManagementDB

Last but not least. if you have Visual Studio installed at the machine and have previously installed Workflow Client, you will have to remove it too. If you don’t do it the installation of the new Workflow Client and Workflow itself will fail.

Hello Service Bus 1.0, Workflow 1.0 and SharePoint Workflow Host

Damir Dobric — Tue, 17 Jul 2012 04:08:18 GMT

The OnPrem version of Windows Azure Service Bus is probably firs peace of Azure Cloud which can be installed on your laptop. Personally I would call it Enterprise Service Bus, which can and should be used for almost any kind of messaging in enterprise. The good thin on this is that the exactly (more or less) the same version is running in the Windows Azure Cloud. The exactly same means in term of API which developers will used. “More or less” means that there are still few differences.
First and probably less important difference is Relaying feature. This feature is currently supported in the cloud only (at least for now). Queue and Topic messaging is fully supported.

Windows Token

However from the architecture point of view, it is more important that On-Premise version (Service Bus Server) additionally to ACS use Windows Token to access queues and topics. To make this working, SB supports internally some kind of Security Token Service, which is able to issue the ACS compatible token on behalf of Windows Token tight in the first message request.
This is all you should now for now about this STS. In fact it is transparent for developer and you shouldn’t eve know that it exist. This information is interesting to understand that there is no any magic behind changed security system. Please note that Service Bus Server can also create a namespace in SB which uses ACS just like SB in azure.

Service Bus Configuration in PowerShell

The currently available release supports Power Shell Based configuration only. The RTM version will additionally support a Configuration Wizard.
Note the Service Bus Explorer supports Service Bus Server (see link below).

Installation via WEB PI 4

Currently you will have to use WEB Platform Installer v4 http://www.microsoft.com/web/downloads/platform.aspx. to install the bits. The setup contains in fact two products: Windows Azure Workflow and Service Bus. Both products are installable on your machine. Yes, that’s right. Windows Azure Workflow is installable on the local machine. Depending on what you want to install, select SB or/and WF and proceed with installation.

Topology Facts

The SB cluster is supported with High Availability feature similar to caching service. That means HA is supported by 3+ nodes in the cluster.
The underlying database can be SQL Express (no joke), SQL 2008 and even SQL 2012.
Workgroups are supported too.
Up to now, bits can be installed on Windows 7 and Windows 8 client OS family. Both are currently supported for development purposes only.

Workflow 1.0

The workflow version installed here needs much more space and time to be explained. For now it is important that you can run your Workflows in Windows Azure. The same version can also be run on-premise. Unfortunately this version cannot use Windows Server AppFabric. The team has internally decided to build the new host for massive scale which activates Workflow Instances from Service Bus topic. At the moment you will have decide whether to use the Windows Server AppFabric host or this one. The new host has been specifically built for SharePoint 15 which is first Microsoft product which will span its workflows on top of Service Bus on premise.
Note that the same host and workflow will be used by SharePoint Online Version.
With this version SharePoint has finally a serious host for running processes outside of SPS pool. My best wishes.(I was waiting almost 10 years for this). More experienced architects and developers might start to think about SharePoint as a SAP Netweaver. Please just don’t overuse it in this case (as often in the past of SPS).
Authoring of Workflows and SPS13 Workflows is WF 4.5.
NOTE, that not all activities of the common WF 4.5 version are supported. For example messaging activities (i.e. ReceiceMessage) and versioning are not supported. Fortunately there are special activities which can perform activation of workflow by message. In fact this is the only way to start he workflow.
If you want to use custom activities in On-Prem version of workflow, there are few extra steps which you have to manually (for now) perform.

Following sample illustrate how to receive the message from topic.

using System;
using System.Collections.Generic;
using System.Net;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Description;
using Microsoft.ServiceBus.Messaging;

namespace Daenet.ServiceBus
{

    public class Subscriber
    {
        private static string ServiceNamespace;
        private static string Issuer;
        private static string Key;
        private static string TopicName = "demotopic";


        static void Main(string[] args)
        {
            ServicePointManager.ServerCertificateValidationCallback = (a, b, c, d) =>
            {
                return true;
            };

            Uri rootAddressManagement =
            ServiceBusEnvironment.CreatePathBasedServiceUri("https",
             “testspace01”, string.Format("{0}:{1}", "YOURHOST", 4446));

            Uri rootAddressRuntime = ServiceBusEnvironment.CreatePathBasedServiceUri("sb",
               “testspace01”, string.Format("{0}:{1}", "YOURHOST", 9354));

            NamespaceManagerSettings nmSettings = new NamespaceManagerSettings();
            nmSettings.TokenProvider = TokenProvider.CreateWindowsTokenProvider(new List<Uri>() { rootAddressManagement });
            NamespaceManager namespaceManager = new NamespaceManager(rootAddressManagement, nmSettings);

            MessagingFactorySettings mfSettings = new MessagingFactorySettings();
            mfSettings.TokenProvider = TokenProvider.CreateWindowsTokenProvider(new List<Uri>() { rootAddressManagement });
            MessagingFactory factory = MessagingFactory.Create(rootAddressRuntime, mfSettings);

            SubscriptionClient subscriptionClient =
            factory.CreateSubscriptionClient(TopicName, "Subscription1", ReceiveMode.PeekLock);

            while (true)
            {
                BrokeredMessage message = subscriptionClient.Receive(TimeSpan.FromSeconds(1500));
                if (message == null)
                    break;

                try
                {
                    Console.WriteLine(string.Format("Message received: SEQUENCE = {0}, Id = {1},
                    Body = {1}", message.Properties["SEQUENCE"], message.MessageId, message.GetBody<string>()));
                    message.Complete();
                }
                catch
                {
                    message.Abandon();
                }
            }

            Console.WriteLine("\nEnd of receiving, press any key to exit.");
            Console.ReadLine();

            subscriptionClient.Close();
        }
    }
}

It is important in this sample that things like Issuer and Key are not required. The scope used by issuing of the token is like

scope=http%3a%2f%2yourhostname%2ftestspace01%2fdemotopic%2fSubscriptions%2fSubscription1.

The issued token is is private for developers and it will be passed internally to all calls to service bus. On thing of your interest is that this token contains claims which describe users AD membership. Based on this information SB will perform authorization.

Resources:

Service Bus Beta resources:

Service Bus 1.0 Beta (download)

Service Bus for Windows Server (Service Bus 1.0 Beta) MSDN documentation

Service Bus Explorer for Windows Server

Workflow 1.0 Beta resources

Workflow 1.0 Beta (download)

Workflow 1.0 Beta MSDN Documentation

Enjoy

University of Applied Sciences in Frankfurt: - Agenda for 25.June 2012

Damir Dobric — Sun, 24 Jun 2012 21:20:35 GMT

NOTE SESSION LOCATION IS CHANGED:

Next Session: https://maps.google.com/maps?q=Microsoft+Bad+Homburg,+Siemensstra%C3%9Fe,+Bad+Homburg,+Deutschland&hl=en&ie=UTF8&ll=50.216637,8.615749&spn=0.006824,0.012553&sll=49.593567,7.75526&sspn=0.884833,1.60675&oq=microsoft+bad+homburg+siemens&t=h&hq=Microsoft&hnear=Siemensstra%C3%9Fe,+Bad+Homburg+61352+Bad+Homburg,+Hessen,+Germany&z=16

Microsoft GmbH
Siemensstraße 27
61352 Bad Homburg vor der Höhe, Germany
Get directions‎
06172 661-0

Last but not least, at the end of sessions I will show few exam questions!

IIS Service Bus Bridge

Damir Dobric — Sat, 23 Jun 2012 09:49:26 GMT

On of upcoming interesting new features related to Service Bus is support for relaying of anything hosted in IIS. Today we are able to build SOAP and REST services which can be relayed with Service Bus. However this is limited to services only. Another option to relay the TCP traffic is “anycast” feature which is not easy to configure and it is in general not well known. Soon, it will be possible to install the IIS Service Bus bridge (see pictures below) which relays the IIS application.

This means, you can host any kind of web application (not only WCF services as now) even on your laptop and access it via publicly available URL listened at the service bus endpoint. Just imagine applications running on the private IP address and accessible publicly in internet without of doing anything on firewall and infrastructure.

More about that: http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/AZR308

Speedup your Web Application with Windows Azure Web and new “Cache Worker Role”

Damir Dobric — Sun, 17 Jun 2012 18:13:21 GMT

This article describes how to create ASP.NET MVC application as a WebRole and to speed it up by using CacheRole introduced in Windows Azure Platform release 1.7 (also called Spring Release [2012]). The way how the distributed cashe works and how it can be consumed by an application didn’t really change. Because of that I will focus only new features, because the way how they are build up and how they works has been slightly changed in this release.

Create Solution with Cache Role

To build solution I will use Visual Studio 2012 and Windows Azure SDK 1.7. If you didn’t install it already do it as described in my previous post here.
Now create the solution and add one WebRole (for example ASP.NET MVC4) and the CacheWorkerRole (I will explain it a bit latter).

The project SharePointFastDataReaderCache is of type CacheWorkerRole and it represents the virtual machine which will host dedicated instance of distributed cache. This cache will be consumed by project (WebRole) SharePointFastdata reader. Please don’t' be confused with SharePoint name. It is not my focus in this article. I just used an existing solution which does the more or less same thing with SharePoint.

The just created role is in fact WorkerRole which is place holder for Distributed Cache Service. That means: “Do not add any code to that role!!” Before you do deployment select the Worker Cache Role and go to properties (dbl-click on SharePointFastDataReaderCache ). Select enable cache and click on Dedicated Role as shown below.

If you have added Cache as role project, you should select “dedicated role”. In this case Windows Azure will during deployment process allocate Virtual Machine (or more if you have selected more than one instance) and install Distributed Cache Service on it (them). If you don’t want to have dedicated machine for cache then go to properties of you WebRole and select Co-located Role, Windows Azure will in this case install distributed cash service on the same machine where you web role is running. In this case you even don’t need Worker Cache Role as project. But you can have it. It might be that you are doing lot of caching and from architecture point of view this might also be a valid solution. I means you have Worker Cache Role as dedicated cache and you also have co-located cache at the WebRole. Physically you would by deployment allocate two different caches with number of chosen instances for each of them.

If you ever worked with Windows Server AppFabric (WSA) similar version of cache will be installed in the slightly different version of cache. In contract to WSA the new one is called simply CacheService. Remember, WSA cache service is called “AppFabricCachingService”. Following pictures shows task manager of the WinAzure machine which host cache as dedicated role.

I captured pictures by connecting to dedicated worker cache role (if Dedicated Role is used) and by connecting to WebRole (if Co-Located cache has been used).
If the WebRole don’t specify the cache processes and services shown above will not be installed on the machine.
Co-located cache-role would be configured as shown at the next picture.

In the picture above is also shown how to add additional caches to the cache instance. remember, this has not been possible by using Windows Azure Cache in version 1.6.

Note that emulator of cache is also provided. The cache is emulated by the cache service emulator. In the picture below, I captured emulator for two cache instances.

Last, but not least the cache instance provisioned here (v1.7) is different one than cache instance v1.6 known as Windows Azure AppFabric Cache, which is shown at the picture below:

Installing cache Client components

After that you have created the solution you will have to install Windows Azure Caching (currently in preview.) libraries by using NuGet. Note that value “Include Prereleases“ (see picture below) is selected in the combo. Otherwise you will find the NuGet package Windows Azure Caching (almost the same name), which is related to version 1.6. That one will not work in this scenario. Even better to say, this one will work, but you will have to use Windows Azure AppFabric cache, which is not the same as Windows Azure Cache Role, which we will focus in this article. Following picture shows what and how to install by NuGet.

After the package has been successfully installed following references are added to project.

Additionally the package inserts following section in the configuration file. This section looks slightly different that the common caching section.
The identifier specified in dataCacheClient element is the name of the role which hosts physically distributed cache. For example following configuration will setup using of dedicated cache deployed by the Worker Cache Role.

If you rather used the Co-located cache in your WebRole (no Worker Cache Role used) then only identifier has to be changed to the name of the WebRole instead of dedicated cache role as shown below:

In the MVC (WebRole) application I defined two actions PutCacheValue and GetCacheValue. Please note in picture below that output-cache has been disabled. This is very important, because output-caching (which is feature of ASP.NET) would prevent the system to work properly.

The JavaScript functions which puts value in the cache and gets it out look like:

At the end the sample application looks like:

More about Caching: https://www.windowsazure.com/en-us/develop/net/how-to-guides/cache/

About direct and brokered messaging

Damir Dobric — Sun, 17 Jun 2012 18:01:29 GMT

When multiple systems are connected to each other different connecting patterns can be used. Most typical kind of connection is called direct messaging. This is not related to terms like direct TCP connection or request response HTTP messaging. When we are trying to connect systems usually we do it with WCF. That means, we will try to abstract (decouple) the transport from contract. By using this approach we do not think about transport and this is why we do not think TCP direct or HTTP req/resp when we say direct connection. Direct means in this context, one system is sending message to another system by using any kind of protocol without of some intermediary between them. Router in this context is not in a role of an intermediary, because it is a part of infrastructure.
Typical example of such systems is typical WCF client/service scenario. Windows community calls this client/service, but the right term would be rather consumer/service, which is usually used in JAVA community.

Additionally to classical direct messaging (consumer sends messages to service), we also have a relayed messaging as an important pattern introduced by Windows Azure Service Bus.In relayed system, participants use some kind of intermediary usually called relay. However the relay establish always a direct connection between participants in the system. This is why the relaying is direct messaging, although it uses “something” between participants.

Typical example of relayed system is service consumer communication through Windows Azure Service Bus relay bindings.

Another important communication pattern is brokered messaging. This pattern assumes existence of an intermediary system between sender and receiver of the message. In contrast to relay there is one broker between participants, which decuples them. When the broker is used in communication system, all participants are decoupled and we usually do not use terms service and consumer. Instead we use terms like sender and receiver.

Typical example of brokered messaging is Windows Azure Service Bus queue and topic based system

Direct and relayed messaging are subject to backpressure from the consumer. Systems which use direct messaging are limited by speed of consumption of messages by consumer. The consumption of message might directly influence the speed of production of the message. Such systems are usually limited in scale. in contract to systems with brokered messaging. By definition, brokered messaging always uses an intermediate store to consume and durably persist messages. Consumers are responsible to pool messages. In brokered systems producer and consumer can scale independently. Usually communication between participants and broker is transaction safe.

University of Applied Sciences in Frankfurt: - Agenda for 18.June 2012

Damir Dobric — Sat, 16 Jun 2012 18:04:28 GMT

DataCacheFactory dcf = new DataCacheFactory();

DataCache cache= dcf.GetCache(cCacheName);

List<DataCacheTag> tags = new List<DataCacheTag>();

            tags.Add(new DataCacheTag("key1"));
            tags.Add(new DataCacheTag("key2"));
            tags.Add(new DataCacheTag("key3"));

bool isCreated = ucdCache.CreateRegion("RegionName");

IEnumerable<KeyValuePair<string, object>> items;

items = cache.GetObjectsByTag(new DataCacheTag("nosuchtag"), "RegionName");

            items = cache.GetObjectsByTag(tags.First(), "RegionName");
            if (items.Count() > 0)
                cache.ClearRegion("RegionName");

            DataCacheItemVersion ver2 = cache.Add("Tagged", 123, tags, "Company2Customer2");

            cache.Add("Tagged1", 123, tags, "RegionName");
            cache.Put("Tagged1", 123, tags, "RegionName");
            cache.Add("Tagged2", 456, new List<DataCacheTag>() { tags.First() }, "RegionName");
            cache.Add("Tagged3", 789, new List<DataCacheTag>() { tags.Last() }, "RegionName");

cache.GetObjectsByTag(tags.First(), "RegionName").ForEach(i => Console.WriteLine(i));

NOTE: array.ForEach() is provided by daenet.system.dll. It is not contained in .NET framework!

Download Presentation: http://developers.de/media/p/317742.aspx

Microsoft GmbH

Siemensstraße 27

61352 Bad Homburg vor der Höhe, Germany

Get directions‎

06172 661-0

University of Applied Sciences in Frankfurt: - Agenda for 4.June 2012

Damir Dobric — Sun, 03 Jun 2012 16:38:24 GMT

Click here to download presentation: http://developers.de/media/p/304235/download.aspx

WinDays 2012 KeyNote Demo Step Through

Damir Dobric — Sat, 12 May 2012 11:54:46 GMT

Few days ago I showed the Demo at WinDays 2012 keynote under the name “SharePoint Ralay”. The idea of the keynote was to show some important features which never have been shown or documented in the way as of very one common industry scenario. To keep things simple we didn’t go to deep into details of SharePoint and ServiceBus. We wanted to to build a demo which will show how to build few complex and enterprise relevant scenarios, but to be in the same time interesting for most developers in audience and even for journalists who have been attending the key note.
So I decided to target three common industrial issues which daenet has been asked to solve multiple times during 2011:

1. How to get (or put) SharePoint data out of enterprise without changing of infrastructure?
2. How to build dynamic WCF contract?
3. How to build Failover cluster in 48 Seconds?

1. How to get/put data out of SPS without changing of infrastructure?

Most SharePoint Developers live in their collaborative context which mostly does not require such scenarios. But when building SharePoint solutions you will figure out that SharePoint isn’t application platform in term how one solution or enterprise architect expect to be. Today, complex systems are systems which consist of usually distributed and hybrid hybrid applications.Distributed means they at least do not run in one process and hybrid means they can be build on top of different technologies.
So, imagine now one application which is build in .NET 4.5 which should run in SharePoint. To make it more simpler image a JAVA application which runs at different machine on different geographic location. Now put it in SharePoint and let talk to each other. Application Platform is the platform which enables to do such things. Any other application platform like SharePoint, CRM, BizTalk or anything else is a platform of their own definition with huge limitations. Because of that be careful, when you say platform. “One can be a boss in his village, but just some little peace in the global world”. The part of Microsoft which build such platforms is called Business Platform Division.

So, let’s get now some data out of SharePoint in context of distributed application scenario. First of all in the world of distributed thinking, you should never allow external parties get in touch with your platform directly. To illustrate this constraint, think on giving the public access to your database. You will probably never allow somebody to send you DB-commands which will be executed against your DB (SQL Injections etc..). If you do it it is wrong and please think change it.. By following the same rule you shouldn’t let anybody to dig in your SharePoint platform for the same reason. You even should not share the secret, that you platform is SPS.

The easiest way is to provide some kind of intermediary component which will abstract the business scenario over secret platform which is running in the background. The technology which does exactly this job is WCF. We will put WCF service between SPS and external party. At some point of time when you have lot of such services, you can start to talk about Middle-Ware. This is a fine solution, but there is a still one practical problem.

The service contract could look in our case as shown below:

[ServiceContract(Name = "EchoContract", Namespace = "http://samples.microsoft.com/ServiceModel/Relay/")]
public interface ISpsRelayService
{
[OperationContract]
List<MyEntity> Query(QueryEntity query);
}

You should define the QueryEntity which somehow describes what you want. SPS developer would like to to see here probably SPS- “ClientObjects”. Unfortinatelly they have never been designed for such scenarios. They can only be used by explicit helper libraries. Such libraries do not work out of SharePoint and their dependent entities are not serializable.
But never mind, define your entities like your business needs them (see service contract above).

The service implementation (see code below) will read the QueryEntity instance and build appropriate SPS-query (not shown in the demo below).
After all we will call SPS WebService hosted internally in our enterprise at http://yourhost:44445 and finally build response as collection of MyEntity instances.
This service decouples the SPS from external parties and provides very simple and business related contract as usual in the SOA world.
Following code snippet shows how to implement the service which gets documents from some SharePOint library and returns them as collection to the external party. In fact this is nothing very complicated (not yet)..

public List<MyEntitity> Query(QueryEntity query)
{
ClientContext clientContext = new ClientContext("http://yourhost:44445");

            Web oWebsite = clientContext.Web;
            var files = oWebsite.GetFolderByServerRelativeUrl("/Shared%20Documents").Files;
            clientContext.Load(files);
            clientContext.ExecuteQuery();

List<MyEntitity> items = new List<MyEntitity>();

            foreach (var doc in files)
            {
                dynamic o = new MyEntitity();
                o.Name = doc.Name;
                o.Something = DateTime.Now.Ticks;
                o.Version = doc.UIVersionLabel;

items.Add(o);
}

return items;
}

To make it available in the internet this service needs to have a public endpoint accessible by the external party. And exactly this is the change demand in infrastructure which is very, very often an organizational issue or even “No go”. First somebody hast to give you an public IP address, then you need to configure NAT and DNS for it, then Security, port mapping from probably 80 to 4445 etc., etc..
I want to build some kind of VPN between my WCF (talking to SPS) and external caller, but without changing any piece of infrastructure. The next picture shows this scenario. Windows Azure provides a platform service called “Service Bus”. This one provides a feature called relying. Relaying can connect two participants directly thorough firewall. Two participants are connected through Service Bus intermediary by using bidirectional TCP connection, which remains open after once is established.

Following code shows how to start-up the service which will listen on Service-Bus packets and relay SharePoint to external caller.

static void Main(string[] args)
{
ServicePointManager.ServerCertificateValidationCallback = validateServerCert;

           // create the service URI based on the service namespace which you will obtain when creating the
           // service bus namespace.
          Uri address = ServiceBusEnvironment.CreateServiceUri("https", serviceNamespaceDomain, "SpsRelayService/DocumentLibrary");

          // create the credential object for the endpoint
          TransportClientEndpointBehavior sharedSecretServiceBusCredential = new TransportClientEndpointBehavior();
          sharedSecretServiceBusCredential.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuerName, issuerSecret);

// create the service host reading the configuration
ServiceHost host = new ServiceHost(typeof(SpsRelayService), address);

          // create the ServiceRegistrySettings behavior for the endpoint
          IEndpointBehavior serviceRegistrySettings = new ServiceRegistrySettings()
{ DisplayName = "Sharepoint Relay Document Library Service", DiscoveryMode = DiscoveryType.Public };

          foreach (ServiceEndpoint endpoint in host.Description.Endpoints)
          {

               // add the Service Bus credentials to all endpoints specified in configuration
             endpoint.Behaviors.Add(sharedSecretServiceBusCredential);

             // Optionally you can add the URI in the registry.
             endpoint.Behaviors.Add(serviceRegistrySettings);
          }

// Open the service
host.Open();

Console.ForegroundColor = ConsoleColor.Cyan;

          Console.WriteLine("Service address: " + address);
          Console.WriteLine("Press [Enter] to exit");
          Console.ReadLine();

          // close the service
          host.Close();
      }

Similarly, external party will also establish the connection as shown in the next snippet:

static void Main(string[] args)
{
ServicePointManager.ServerCertificateValidationCallback = validateServerCert;

// create the service URI based on the service namespace
Uri address = ServiceBusEnvironment.CreateServiceUri("https", serviceNamespaceDomain, "SpsRelayService/DocumentLibrary");

           // create the credential object for the endpoint
           TransportClientEndpointBehavior sharedSecretServiceBusCredential = new TransportClientEndpointBehavior();
           sharedSecretServiceBusCredential.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuerName, issuerSecret);

// create the channel factory loading the configuration
ChannelFactory<ISpsRelayService> channelFactory = new ChannelFactory<ISpsRelayService>("RelayEndpoint", new EndpointAddress(address));

// apply the Service Bus credentials
channelFactory.Endpoint.Behaviors.Add(/*swtBehavior*/sharedSecretServiceBusCredential);

           // create and open the client channel
           ISpsRelayService channel = channelFactory.CreateChannel();

           ((ICommunicationObject)channel).Open();

           Console.WriteLine("Enter text to echo (or [Enter] to exit):");

           string input = Console.ReadLine();

           List<MyEntity> res = channel.Query(new QueryEntity(){…});

            ((ICommunicationObject)channel).Close();

           channelFactory.Close();
       }

If you ever worked with WCF there is not much more to say. The code shown above contains a bit sugar around standard WCF.Everything else is a matter of channel which is hidden behind RelayBinding. For more information about ports in use take a look here.
Following binding must be configured in both service and client.

After you start service and then client,the client will send the request to service through firewall. The service will get data out of SPS and relay it back to the client.

2. How to build dynamic WCF contract?

In the world of SOA you always should build a clear contract between two boundaries. That means contracts like following one are not well designed:

Message Operation(Message msg);

I have also seen in some projects following variant of the similar contract, which is semantically the same one as previous one:

string Operation(string msg);

By using such contract the client has no way to know how to serialize the message. Even after the service version is changed, there is no way to communicate this to anybody instead of verbal human communication which is time intensive and error prone. Compare it with “add service reference” to create the proxy. The client and service must in such cases implement code which serializes/deserializes respectively the message (string).
In other words, using service contracted this way is called “Anti-Pattern”, which should not be used in the SOA world.

Many developers think they can build SOA solution if they are just able to build WCF based service. This is a mistake. When you use WCF it doesn’t mean at all that you are by default SOAP capable. In our scenario I will use out this fact as my advantage. Connecting external party to our relay service is not SOA focused scenario. It is more about classical EAI and not SOA. Such kind of connecting system is known as interface with point-to-point connection. This is also not the best pattern in the world of SOA (I mean point2point interface). and it should b e avoided when building middle-wares. But this is another story.

In other words, If you are connecting two systems as point2poin it might be valid to apply SOA anti-pattern.This is in context of SOA an anti-pattern, but not an anti-pattern in universe of possible solutions. If you have two parties talking to each other then you can often change the contract between them without of too much organizational difficulties.If you are changing often the contract (in our case MyEntity and MyQuery) it is easier to make it dynamic. In this case you can build it once without need to change it in the future, because it is dynamic one. Following example shows how to use ExpandoObject to build dynamic WCF contract. If you don’t know what this is take a look on Alexandra’s and Rick’s posts.

ServiceContract(Name = "SpsRelayContract", Namespace = "http://daenet.deServiceModel/Relay/")]
public interface ISpsRelayService
{
[OperationContract]
List<ExpandoObject> Query(ExpandoObject expression);
}

The good thing on Expando-contract is that you can do something like this:

dynamic o = new ExpandoObject();
               o.Name = “bla bla”;
               o.Something = DateTime.Now.Ticks;
               o.Method= new SomeDelegate(someMethod);

You can set on this object whatever property you want (more or less). But note in a case of WCF the method instance as property wouldn’t make a sense, because that instance will not exist on some other process and some other machine. However all other properties can be set as you like. And WCF will serialize them automatically for you. In other words, if you have a new property you don’t have to change contract. You just set it like:

o.NewPropery = 17;

The consumer of this value just hast to know that is there (because,. remember, there is not clear contract, Add-Reference or SVCUTIL).

As a recap, note that ExpandoObject provides a dynamic contract. Build your Service on top of this entity and you will not have to change the contract when new properties are added in the business scenario, which service implements. Remember that this is not a SOA capable solution, but it is valid to build Point-To-Point scenarios when one party is connecting to another one.

3. How to build Failover cluster in 48 Seconds?

Last but not least I there is an issue which has been mostly forgotten in almost all our discussion when building relaying scenarios. I every project you will reach a point when the solution has to be deployed in the production. Then you will ask yourself how to build redundancy based on relying? This means, you want to provide a fail-over instance of you WCF relaying service which will be used when the first one is not reachable. The picture shown bellow illustrates this requirement.

To make this working you really do not have to change any peace of infrastructure. Just start the another instance of the service on another machine. That’s all. This will work by using both protocols TCP and HTTP. There are currently few undocumented differences in routing when using different protocols, nut in general both will provide you the stable and reliable fail-over mechanism.

Remark: If you are demoing this scenario with both WCF-instances on one machine, the basicHttpRelayBinding will not work with out-of-the-box configuration, because both instances will try to start listening for contract sharing at the same address. In a case of netTcpRelayBindig this will not happen. So, the latter one can be started multiple timas at one box.

Thanks to Tomislav Tipuric director of WinDays technical track who has visionary recognized the industry need for this scenario.