First we'll start with what is Windows Azure?
Windows Azure is Microsoft's operating system for cloud computing and it is intended to simplify IT management and minimize up-front and ongoing expenses, for building, deploying and managing applications and services through a global network of Microsoft-managed data-centers. It provides both PaaS and IaaS services and supports many different programming languages, tools and frameworks, including both Microsoft-specific and third-party software and systems.
As a BizTalk developer we are familiar with schemas, maps, adapters, pipelines and all the other working parts of BizTalk Server. Now Microsoft has introduced several options for BizTalk in “The Cloud” or BizTalk in Azure. This support for BizTalk in Azure comes in two forms: running BizTalk in virtual machines hosted in Azure (Infrastructure as a Service – IaaS) or using BizTalk Services (Platform as a Services – PaaS).
Running BizTalk in Azure Virtual Machines – IaaS
The first challenge is we have to build development environment for our BizTalk solution. we need to facilitate a SQL Server cluster, BizTalk Servers, and regular web servers. Then try building development, test, and production environments (servers, software's,...) and resources that need to be acquired or at least provisioned. This is where the IaaS offering in Azure can simplify our work by providing virtually unlimited resources and enabling you to very quickly provision the servers we need.
Windows Azure provides a gallery of Virtual Machine (VM) images that can be used to provision a server in an Azure data center. The below picuture shows an example of creating a BizTalk Server 2013 image using the Azure portal.
Azure Virtual Network : Virtual networks enable access to Virtual Machines and Cloud Services in Azure The communication between virtual machines is challenging because of the networking limitations in place for security considerations. So, here Microsoft introduced virtual network concept.
We can place our virtual machines in virtual networks. These virtual networks provide a private IP address space that our virtual machines can share to be able to easily address each other, making connections between Active Directory, SQL, MS DTC, and services built with local network connectivity in mind much easier to manage.
Virtual Network provides a VPN connection between our virtual network in Azure and an on premises network. VPN connections can either be point-to-site or site-to site.
Point-to-site: A single server in our data center has a secure connection to our Azure network. A point-to-site connection is easy to setup as connecting our laptop to a VPN which is finally what we are doing in this scenario. If we just have one or several servers that we need to connect to our Azure network, this is the simplest solution.
Site-to-site: We setup a secure connection between the two networks.we need a supported external VPN device and IPv4 address already configured. We can then run a script to setup a connection between the two VPN networks allowing multiple machines in our environment to be connected to the Azure network without having to install or configure any software on the individual servers.
Azure Express Route
It provides a dedicated secure connection between our network in Azure and our on premises network. It is supported by many different network providers and provides more reliability, security (nothing goes over the public internet), faster speeds and lower latency than virtual networks.
There are several application level options for connecting our BizTalk application in Azure to our on premises resources: Hybrid Connections and Azure Service Bus.
Hybrid connections: Allow us to connect to on premises databases and HTTP resources from Azure websites and mobile services using regular connection strings and URLs. Hybrid connections are a feature of BizTalk Services and provide connectivity similar to point-to-site virtual networks in that one server on our environment is connected to Azure resources. Only a connection to a particular resource on that server will be exposed to Azure instead of entire server.
It depend on a service installed and configured on a server in our data center. This service acts as the gateway between connections from Azure and our local resources on that server or other servers in our environment. One useful feature of Hybrid Connections is that a single defined connection can be shared across multiple websites and mobile services.
Service bus: Provides both message relay and brokered messaging options that can traverse firewalls and network address translators.
With relayed messaging both an on premises resource and cloud resource connect to the Service Bus at a pre-arranged address and messages are sent through the bus.
With brokered messages provide one-way queued messaging with publish and subscribe functionality. This type of communication is similar to how BizTalk internally handles messages on the Message Box.
The Service Bus can be used to connect our BizTalk IaaS application from Azure to on premises resources using either the relay or brokered option. Using the same configuration we can enable partner organizations to interact with our BizTalk solution without having to provide direct access to the BizTalk servers themselves.
Service bus allows us to send messages through an internet relay to which both applications connect enabling secure traversal of firewalls and network address translators. Additionally, Service Bus provides brokered messaging, much like BizTalk itself, using queues and topics. This brokered messaging model supports occasionally connected clients and a publish / subscribe model of communication. Using Service Bus along with our BizTalk deployment in Azure can greatly reduce the complexity of connecting BizTalk to other applications over the internet.
Building solutions on BizTalk Services – PaaS
As per the IaaS model we might have noticed something: we still have to install all the software and manage all the patches on all the machines running in our Azure BizTalk solution. The PaaS model aims to avoid all of that maintenance by providing us with a BizTalk service that we provision and into which we can deploy our solution. The only infrastructure decision we need to make is what features and scale we need to handle your business.
Bridge
A bridge is defined with a message itinerary that describes the end to end processing of a message from the time it is received until it is sent out of the system. The below image shows a simple message processing itinerary that receives a message from a Service Bus Queue, processes the message and then routes the message to one of two destinations based on contextual properties in the message.
The arrows specify the path that a message can follow. It is a visual flow representation that links different components together.
The BizTalk developers are familiar with this type of processing. The Bridge covers the definition of a receive location (adapter and pipeline), routing configuration or subscriptions, and send ports. The “SourceQueue” item defines receiving a message from a Service Bus queue, which is essentially the adapter configuration. The “DestinationQueue” and “FanOutTopic” items define the equivalent of Send Port Adapters and a routing table defines the rules for routing messages as they finish the “XmlBridge” step.
The “XmlBridge” step can be expanded as shown in below picture and includes all of the typical pipeline and transform logic commonly known as Validate, Enrich and Transform. The bridge starts with a stage to define the message types that will be processed. Despite being called an “XML” bridge this type of bridge can handle both XML and flat file documents. The “Decode” and “Encode” stages of the process enable receiving and/or sending flat files using flat file schemas. The “Validate” stage validates documents against a specific XSD schema just as in a BizTalk receive pipeline.
The transform stage can be configured with several maps and determines which map to use at runtime based on the incoming message type. Finally, the “Enrich” stages allow for adding or copying properties. Developers can also write code to execute before or after each stage in the bridge. The simple .NET interface you implement allows us to process the message and manipulate context properties much as we would with a custom pipeline component. After we develop our bridges in Visual Studio along with our maps, schemas, and any custom code we then deploy the package to our BizTalk Service. Once we’ve deployed our bridge we can send messages via HTTP if our bridge is setup for that protocol or the bridge will start monitoring queues, FTP servers and other sources we specified.
EDI
BizTalk Services also support both X12 and EDIFACT EDI processing. Just like in BizTalk Server these interchanges depend on the configuration of partners, profiles and agreements that define the EDI processing and the deployment of EDI schemas for the transaction sets our environment supports. Configuration of the partners and agreements all happens through a web interface but provides most of the same settings we would configure in a BizTalk Server environment. For example, the below picture shows an X12 agreement being defined between two parties. Once the basic details are provided the system allows you to specify the specific schemas being used, authorization data, and other EDI processing settings.
BizTalk Services is a Platform as a Service (PaaS) offering which means there are no virtual machines to manage.
Window Azure BizTalk Services:
Azure BizTalk Services is a simple, powerful and extensible cloud-based integration service. It provides Business-to-Business (B2B), Enterprise Application Integration (EAI) and Hybrid Connections capabilities for delivering cloud and hybrid integration solutions. The service runs in a secure, dedicated, per-tenant environment that you can provision on demand.
Windows Azure users can create applications that run on the cloud. However, given the fact that these applications operate in their own ‘space’ on the cloud but at the same time need to interact with other on-premise or cloud applications, there is a need to bridge the message and transport protocol mismatch between these different applications. Bridging these mismatches is the realm of integration. There can be different forms of integration.
WABS provides rich EAI capabilities in providing config driven design tools to bridge the message and transport protocol mismatch between two disparate systems. To name just a few of WABS EAI capabilities:
- The ability to connect systems following different transport protocols
- The ability to validate the message originating from the source endpoint against a standard schema
- The ability to transform the message as required by destination endpoints
- The ability to enrich the message and extract specified properties from the message. The extracted properties can then be used to route the message to a destination or an intermediary endpoint.
- The ability to track messages.
The WABS B2B solution, which comprises of the BizTalk Services Portal and B2B pipelines, enables customers to add trading partners and configure B2B pipelines that can be deployed to WABS. The trading partners will then be able to send EDI messages using HTTP, AS2, and FTP transports. Once the message is received, it will be processed by the B2B pipeline deployed on the cloud and will be routed to the destination configured in the B2B pipeline. Few of the WABS EDI capabilities are:
- Easily manage and onboard trading partners using the BizTalk Services Portal. With the BizTalk Services Portal, customers will be able to cut down the on-boarding time from weeks to days.
- Leverage Microsoft hosted B2B pipelines as services to exchange B2B documents and run them at scale for customers. This minimizes overhead in managing B2B pipelines and their corresponding scale issues with dedicated servers.
- Ability to track messages.
Each WABS project can contain a single “bridge configuration” file. This file defines the flow of data between source and destination endpoints. Once we have a WABS project, we can add XML schemas, flat-file schemas, and maps. The Schema Editor looks identical to the BizTalk Server Schema Editor and lets we define XML or flat file message structures. While the right-click menu promises the ability to generate and validate file instances. The mapper is very different from the BizTalk Mapper and that’s a good thing. The UI has very good features, but the more important change is in the palette of available “functoids” (components for manipulating data). First, we'll see more sophisticated looping and logical expression handling. This include a ForEach Loop and finally, an If-Then-Else Expression option.
The concept of “lists category” are also entirely new. We can populate, persist, and query lists of data and create powerfully complex mappings between structures.
The “miscellaneous” operations introduce useful funtoids. These functoids let us grab a property from the message’s context (metadata), generate a random ID, and even embed custom C# code into a map.
The bridge configuration tells about what source and destination endpoints are supported. The toolbox for the bridge configuration file shows three different types of bridges: XML One-Way Bridge, XML Request-Reply Bridge, and Pass-Through Bridge.
- XML One-Way bridge: this is a bridge that processes incoming messages and passes them on to the next matching connection.
- XML Request-Reply bridge: this is a bridge that processes incoming messages and their corresponding response message.
- PassThrough bridge: this is a bridge that does not look at the message details and can parse every type of message.
- EDI bridge: unfortunately, this bridge is not available to the Visual Studio designer and is shielded by the TPM portal.
Sources:
We can do synchronous or asynchronous XML messaging, or any flat file transmission. To get data into a bridge, we can use HTTP, FTP, or SFTP. Notice that “HTTP” doesn’t show up in that list as each bridge automatically has a Windows Azure ACS(Access control service)-secured HTTP endpoint associated with it.
Destinations:
We can consume web services, Service Bus Relay endpoints, Service Bus Queues / Topics, Windows Azure Blobs, FTP and SFTP endpoints.
An individual bridge has a number of stages that a message passes through. Double-clicking a bridge reveals steps for identifying, decoding, validating, enriching, encoding, and transforming messages.
Each individual step exposes relevant configuration properties. The “Enrich” stage provides a way to populate data in the outbound message’s metadata (context) properties. Options include pulling values from the source message’s SOAP or HTTP headers, XPath against the source message body, lookup to a Windows Azure SQL database, and more.
When a bridge configuration is completed and ready for deployment, simply right-click the Visual Studio project and choose Deploy and fill in valid credentials for the WABS preview.
BizTalk Services VS Azure BizTalk VM
BizTalk Services provides a true Platform-as-a-Service (PaaS) architecture for building integration solutions in the cloud. With the PaaS model, we focus completely on the application logic and leave all of the infrastructure management to Microsoft, including:
- No need to manage or patch virtual machines
- Microsoft ensures availability
- We control scale on-demand by simply requesting more or less capacity through the Azure management portal
BizTalk Server on Azure Virtual Machines provides an Infrastructure-as-a-Service (IaaS) architecture. We create virtual machines and configure them exactly like our on-premises environment, making it easier to run existing applications in the cloud with no code changes. With IaaS, we are still responsible for configuring the virtual machines, managing the virtual machines (for example, installing software and OS patches), and architecting the application for high availability.
Use BizTalk Services, if we are looking at building new integration solutions that minimize our infrastructure management effort.
Use BizTalk Server on an Azure Virtual Machine, If we are looking to quickly migrate our existing BizTalk solutions or looking for an on-demand environment to develop and test BizTalk Server applications.
BizTalk Adapter Service vs Hybrid Connections
The BizTalk Adapter Service uses the BizTalk Adapter Pack to connect to an on-premise Line of Business (LOB) system. A Hybrid Connection provides an easy and convenient way to connect Azure applications, like Websites and Mobile Services, to an on-premises resource.
Itinerary flow
A message itinerary is a definition of a flow (it was called the flow designer in the bridge). An itinerary does not have any persistence points and executes 100% in memory. (example: If an FTP destination in an itinerary is unavailable, then there are no retries or whatsoever. the consumer will get a 500 exception in that case!)
WABS maps don’t run in BizTalk Server, and vice versa. Also, there’s no concept of long-running workflow (i.e. orchestration), and none of the value-added services that BizTalk Server provides (e.g. Rules Engine, BAM).
