Property Promotion inside Orchestration

Properties get promoted within BizTalk at different places, Example: some of the properties gets promoted by Adapters, Pipelines Components, Messaging Engine, etc.

    In custom pipeline components, developers can use either IBaseMessageContext.Write to write context properties or IBaseMessageContext.Promote to promote context properties. The basic difference between writing and promoting properties are, properties that are written cannot be used for message routing, whereas properties that are promoted can be used for message routing.

Below is the sample for promoting properties within Orchestration is:
    MSG_OUT(ProjectNameSpace.PropertySchema.PropertyName) = "XXX"

If we look at the outgoing message context, the above prompted property will not promote and it would be in the "NotPromoted" state.


Please follow the below instructions to promote your property within the orchestration:

    In BizTalk we have a Correlation concept, basically which routes the response messages to the correct running orchestration instance that initiated the request message. Within Orchestration we use Correlation Set and Correlation type to archive this type of instance routing.

Correlation type is nothing but a set of Properties and Correlation Set is based on Correlation type and its a set of properties with specific values. When we "Initialize a Correlation set" within orchestration, the orchestration instance automatically promotes those properties in Correlation set into the message context.

We can use this mechanism to promote properties and there is no need to do a follow up of the Correlations sets we initialized, and it NOT going to create unnecessary subscriptions.

Modify your configuration file

Add "System.Configuration" reference to your application


 ExeConfigurationFileMap configFileMap = new ExeConfigurationFileMap();      
            configFileMap.ExeConfigFilename = @"{path}";

            Configuration config = ConfigurationManager.OpenMappedExeConfiguration(configFileMap, ConfigurationUserLevel.None);


            Console.WriteLine(config.AppSettings.Settings["YourKey"].Value);
            config.AppSettings.Settings["YourKey"].Value = DateTime.Now.Date.ToString("yyMMdd");
            config.Save();

Copy list to new list, while adding new value to list overwrites previous value in the old list

List<T> Constructor (IEnumerable<T>):
Initializes a new instance of the List<T> class that contains elements copied from the specified collection.

Sample:
            List<string> col1 = new List<string> { "R1", "R2", "R3" };
            List<string> col2 = col1;

            col2.Add("R4");

            Console.WriteLine("Col1 Count is:" + col1.Count);
            Console.WriteLine("Col2 Count is:" + col2.Count);
            Console.ReadLine();

The OutPut is:
Col1 Count is:4
Col2 Count is:4

            List<string> col1 = new List<string> { "R1", "R2", "R3" };
            List<string> col2 =  new List<string>(col1);

            col2.Add("R4");

            Console.WriteLine("Col1 Count is:" + col1.Count);
            Console.WriteLine("Col2 Count is:" + col2.Count);
            Console.ReadLine();

The OutPut is:
Col1 Count is:3
Col2 Count is:4

Custom File Adapter for Pooling Interval

The Pooling Interval property decide how long system should wait to poll the receive location again, if it get no message last time it polls.

Polling intervals in File/FTP adapters not works as expected, This is how it works
1. You drop 4 files at a time.
2. Biztalk File/FTP adapter reads 1 file at a time (according to batch size=1)
until the receive location is empty.
          This means while your first file being picked up, Biztalk FTP/File adapter identifies 3 more files are pending. So FTP/File adapter discards the polling interval and pick one after the other until your receive location is empty.

3.If the receive location is empty , it waits util the next polling interval.

I think the batching works well but it seem like it does not wait for the polling interval.

We want to change the behavior for the Pooling Interval property, it is all about performance. We don't want our CPU to be fully occupied with receiving messages because lots of them arrived is a short period of time, maybe making our system to work really slow, expiring timeout of other ports.

Microsoft released all the most important BizTalk parts with source code, we can locate those in C:\Program Files (x86)\Microsoft BizTalk Server 2010\SDK\Samples and in this directory exist the source code of the BizTalk file adapter, HTTP adapter, and we can find all the base class library here which will be useful to develop a new BizTalk Adapter.

we'll use the source code of the File Adapter (located in AdaptersDevelopment\File Adapter)

In the directory we'll find ,BizTalk Project directory with one BTS sample, Design Time directory is a project to manage adapter configuration, Runtime directory is the project of adapter runtime.

     Runtime project is the most important, open it, in this project you can observe the two principal files DotNetFileReceiverEndpoint.cs that contain the code for receive the stream and DotNetFileTransmitterEndpoint.cs that contain the code to transmit the stream.
In DotNetFileReceiverEndpoint.cs

• public void ControlledEndpointTask (object val)

        {
            if (this.controlledTermination.Enter())
            {
                try
                {
                    lock (this)
                    {
                        this.EndpointTask();                      
                    }
                    GC.Collect();                  
                }
                finally
                {
                    this.controlledTermination.Leave();
                    Thread.Sleep(new TimeSpan(0, 0, 0, ((int)this.properties.PollingInterval)));
                }
            }
        }

•     In PickupFilesAndSubmit(), modify the existing code with the below one.

         FileInfo[] items = di.GetFiles(this.properties.FileMask)
               Add System.Linq; namespace at the using block.
        FileInfo[] items = di.GetFiles(this.properties.FileMask).Take(maxNumberOfFiles).ToArray();
     

• please add the below code, to avoid The process cannot access the file because it is being used by another process error. 

               // If we couldn't lock the file, just move onto the next file
                IBaseMessage msg = CreateMessage(fileName, renamedFileName);
                             
                MemoryStream ms = new MemoryStream();
                msg.BodyPart.Data.CopyTo(ms);
                msg.BodyPart.Data.Position = 0;

                if ( null == msg )
                    continue;

                if ( null == renamedFileName )
                    files.Add(new BatchMessage(msg, fileName, BatchOperationType.Submit));
                else
                    files.Add(new BatchMessage(msg, renamedFileName, BatchOperationType.Submit));
             
                if (ms.CanSeek)
                {
                    ms.Position = 0;
                }
                msg.BodyPart.Data = ms;

                //  keep a running total for the current batch
                bytesInBatch += item.Length;

Now we can rebuild our adapter and have to register it.

To register the adapter we must execute the registry file StaticAdapterManagement.reg, we found some possible issues in this file.

The first is the directories, check all path directory because all are wrong:

• C:\Program Files\Microsoft BizTalk Server 2010\SDK\Samples\AdaptersDevelopment\FileAdapter\Runtime\bin\Debug\Microsoft.BizTalk.SDKSamples.Adapters.DotNetFile.Runtime.dll”

• “OutboundEngineCLSID”=”{024DB758-AAF9

• Is not FileAdapter but File Adapter

The second, check if your sample is under Program Files or Program Files (x86) in case modify the path or create your own different directory project.

The third and the most important is, if your BizTalk run in 64 bit mode you must modify the registry keys because BizTalk server in 64 bit search configuration under Wow6432Node

So, for example, if you want work with BizTalk configuration in 32 bit mode the correct string is

[HKEY_CLASSES_ROOTCLSID{62018D08-281A-415b-A6D3-6172E3762867}]

in 64 bit is

[HKEY_CLASSES_ROOTWow6432NodeCLSID{62018D08-281A-415b-A6D3-6172E3762867}]

In the BizTalk Admin console, right click on the adapters, new and select Static DotNetFile, for the adapter name as you want.

Restart the host instance

Create one receive port, one send with correct filter and test the solution

The result must be this

Here we have set the Number of files in Batch as 2 and Pooling interval as 60 seconds.

Failed to start the BizTalk Host instance. Check the event log on the server for more details.

Have you done any changes to BizTalk config file recently. Can you open the file in any XML editor any check it has any format error. Also check for a custom entires, which may not be vaild. I would have taken backup of the config files before any chances, if you have then compare the changes. If you have done some changes make sure you have done them in the 64bit config files.

System.Xml.XmlDocument is not marked as serializable in BizTalk

XmlDocument is not a serializable type.

     This error usually occurs when an orchestration (which uses any of your assembly that has XmlDocument type field) is going to be dehydrated, the BizTalk engine is going to serialize every variable.

     This won't apply to variables that are of the XmlDocument type in the orchestration as BizTalk knows it's not serializable and won't try. Because this is so widely used that it is treated as a special case. The BizTalk Engine knows how to serialize it so the XLang compiler doesn't raise an error. The serialization process saves the type of the variable and the value of the OuterXml property, which can then later be used to instantiate a matching
System.Xml.XmlDocument when deserializing.


I got this exception when we want to receive a response type as System.Xml.XmlDocument from a C# class. or if we have System.Xml.XmlDocument object declaration at class level.

 Use Default Constructor property causes an instance of the object to be constructed when the orchestration instantiates. If this property is set to False, the variable will need to be instantiated in an Expression shape through the new keyword. Try this, otherwise use below option.
                                      OR

Mark the member as NonSerializable. BizTalk will not attempt to serialize the object and you shouldn't see this error
                                     OR
The solution is very simple: if you're using, in any of your orchestrations, assemblies that have a XmlDocument type field as a response. Then make that class and members are static (No instance, no persistence, no serialization).


Possible error is:
Type 'System.Xml.XmlDocument' in Assembly 'System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' is not marked as serializable.

   

Compensation block

Scope: Scopes are used for three reasons:

1. to configure transaction (long running or atomic)
2. to handle exceptions
3. to trigger compensating logic

A scope shape can have one or more Exception handling blocks and one Compensation Block. The Transaction type you've chosen for your scope will define which of these blocks can be added. An atomic scope can only have a compensation block, and a scope configured with the transaction type none can only have an Exception block. While a long running scope can have both blocks.

For an overview of the possibilities, check the below screenshot.



There could be more than one Exception block, but only one Compensation block added to a Scope.


Compensation block
Compensation is a way to correct or undo logical piece of work that have previously committed. 
       An exception can occur in the orchestration after successfully executing all logic in the scope. In this case, the process might be in a state where it is logically incorrect. So the action already performed in a transnational scope need to be compensated... this can be achieved using a compensation block
If no compensation block is added, the default compensation will be performed. this means calling all compensation blocks of the nested scopes, starting with the most recently completed ones, just as with the default exception handler.

Compensations blocks may be created for both atomic and long running transactions.  Compensation blocks must be explicitly invoked. To invoke the Compensations block, finally drop the "Compensate" shape into the exception handler block (Compensate shape can be placed only in exception or compensation block) and set the "Compensation" property to the corresponding transaction name in the properties window.

A particular scope can be compensated only once.The compiler will prevent more than one path leading to the compensation of a particular scope.

Compensate
      The compensate shape can only be used inside of an exception handler. The explicit use of this compensate shape, allows us to specify the order of compensation execution ourselfs. So this can be used in cases the default order (starting from te most recently completed ones and working its way back) doesn't fit.
1 compensate shape can only trigger 1 compensation block of a transaction. So the order must be defined using multiple compensate shapes, calling the compensation blocks in the desired order.




When an Atomic transaction throws an exception, this will generally be caught at an outer non-atomic scope. Atomic scopes cannot have exception handlers of their own. Compensation can only be invoked if the exception is caught by an outer LongRunning transaction.

For Ex:

Consider the above scenario for basic compensation model

We have 1 LongRunning transaction and 3 nested atomic transactions which insert data in DB with their compensation blocks associated with them,now the 3rd atomic transaction fails to commit then it invoke the compensation block in backward direction .

Execution order:

Insert into Table1(scope Shape)
Insert into Table2(scope Shape)
Insert into Table3(scope Shape) —  not commited
Catch exception
Compensation block 2
Compensation block 1