Get Unique values at document level using xpath

I have used preceding-sibling and following-sibling to get the unique skill values from the below XML message.

XmlDocument doc = new XmlDocument();
doc.Load(@"{file-uri}");

MemoryStream xmlStream = new MemoryStream();
            doc.Save(xmlStream);

            xmlStream.Flush();//Adjust this if you want read your data
            xmlStream.Position = 0;

            //XmlReader reader = XmlReader.Create(doc.OuterXml);
            //XPathDocument document = new XPathDocument(reader);
            XPathDocument document = new XPathDocument(xmlStream);
            XPathNavigator xPathNavigator = document.CreateNavigator();

            xPathNavigator.MoveToRoot(); // Move to the root element.
            if (xPathNavigator.HasChildren)
            {
                xPathNavigator.MoveToFirstChild();
            }
XmlNamespaceManager namespaceManager = new XmlNamespaceManager(new NameTable());
          nsmgr.AddNamespace("ns0:", "http://www.RVRCompany.com/en/core/employee");

XPathNodeIterator nodIterator = xPathNavigator.Select("//ns0:Employee/ns0:Project/ns0:Skills/ns0:Skill/ns0:SkillName[not(../../../../following-sibling::ns0:Employee/ns0:Project/ns0:Skills/ns0:Skill/ns0:SkillName = .) and not(preceding-sibling::ns0:SkillName=.)]",namespaceManager);

Console.WriteLine("Unique values **********************************************");
            while (nodIterator.MoveNext())
            {
                Console.WriteLine(nodIterator.Current.Value);
            }
            Console.ReadLine();

Sample XML:

<ns0:UpdateEmployeesRequest xmlns:ns0="http://www.RVRCompany.com/en/core/employee">
<ns0:EmployeeDetails>
<ns0:Employee>
<ns0:EmpID>1</ns0:EmpID>
<ns0:Project>
<ns0:ProjectName>PROJ1</ns0:ProjectName>
<ns0:Skills>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>.Net</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>BizTalk</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>.Net</ns0:SkillName>
</ns0:Skill>
</ns0:Skills>
</ns0:Project>
</ns0:Employee>
<ns0:Employee>
<ns0:EmpID>2</ns0:EmpID>
<ns0:Project>
<ns0:ProjectName>PROJ2</ns0:ProjectName>
<ns0:Skills>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>.Net</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>BizTalk</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>.Net</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>Sun Micros</ns0:SkillType>
<ns0:SkillName>Java</ns0:SkillName>
</ns0:Skill>
</ns0:Skills>
</ns0:Project>
</ns0:Employee>
<ns0:Employee>
<ns0:EmpID>3</ns0:EmpID>
<ns0:Project>
<ns0:ProjectName>PROJ2</ns0:ProjectName>
<ns0:Skills>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>VB6</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>.Net</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>SQL Server</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>IIS</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>Sun Micros</ns0:SkillType>
<ns0:SkillName>Linux</ns0:SkillName>
</ns0:Skill>
</ns0:Skills>
</ns0:Project>
</ns0:Employee>
<ns0:Employee>
<ns0:EmpID>3</ns0:EmpID>
<ns0:Project>
<ns0:ProjectName>PROJ2</ns0:ProjectName>
<ns0:Skills>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>VB6</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>MS</ns0:SkillType>
<ns0:SkillName>IIS</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>Sun Micros</ns0:SkillType>
<ns0:SkillName>Linux</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>Sun Micros</ns0:SkillType>
<ns0:SkillName>Tomcat</ns0:SkillName>
</ns0:Skill>
<ns0:Skill>
<ns0:SkillType>Sun Micros</ns0:SkillType>
<ns0:SkillName>BizTalk</ns0:SkillName>
</ns0:Skill>
</ns0:Skills>
</ns0:Project>
</ns0:Employee>
</ns0:EmployeeDetails>

</ns0:UpdateEmployeesRequest>

Programmatically Enable/Disable Receive pipeline

We can achieve this functionality using the  "Microsoft.BizTalk.ExplorerOM" component.

Add reference "Microsoft.BizTalk.ExplorerOM" component to your application.

BtsCatalogExplorer bceExplorer = new BtsCatalogExplorer();
//Edit the following connection string to point to the correct database and server
bceExplorer.ConnectionString = "Integrated Security=SSPI;database=BizTalkMgmtDb;server=localhost";

public bool SetReceiveLocationState(string receivePortName, bool stateValue)
{
            bool returnValue = false;
            try
            {
                ReceivePort receivePort = bceExplorer.ReceivePorts[receivePortName];
                receivePort.ReceiveLocations.Cast<ReceiveLocation>().ToList().ForEach(receiveLocation => receiveLocation.Enable = stateValue);

                bceExplorer.SaveChanges();
                returnValue = true;
            }
            catch (Exception e)
            {
                System.Diagnostics.EventLog.WriteEntry("SetReceiveLocationState", e.Message);
                bceExplorer.DiscardChanges();
                returnValue = false;
            }
            return returnValue;
        }

Programmatically (Dynamically) change the properties of a Receive Location

We can update the properties of a File adapter Receive Location using the Microsoft.BizTalk.ExplorerOM (from {BizTalkInstallation}\DeveloperTools) and I refereed useful article from MSDN for my development.

   We have a requirement to programmatically(dynamically) change the File Mask property of the File receive location.
   The following code only deals with a File Mask property that can be changed in the File receive location.
It accepts few things as input, we accept the port name, which receive location you want to change, file mask and it will change the file mask property in the receive location.

Add reference "Microsoft.BizTalk.ExplorerOM" component to your application.

I have written below as per the MSDN article and this trick didn't worked for me.

BtsCatalogExplorer bceExplorer = new BtsCatalogExplorer();
//Edit the following connection string to point to the correct database and server
bceExplorer.ConnectionString = "Integrated Security=SSPI;database=BizTalkMgmtDb;server=localhost";

public bool UpdateReceiveLocationFileMask(string receivePortName, string receiveLocationName, string fileMask)

{

            bool returnValue = false;

            string transportData = string.Empty;

            XmlDocument doc = null;

            XmlNode root = null;

            XmlNode fileMaskNode = null;

            try

            {

                if (!string.IsNullOrEmpty(fileMask))

                {

                    ReceivePort receivePort = bceExplorer.ReceivePorts[receivePortName];

                    ReceiveLocation receiveLocation = receivePort.ReceiveLocations.Cast<ReceiveLocation>().Where(item => item.Name == receiveLocationName).FirstOrDefault();

                    if (receiveLocation != null)

                    {                        

                        transportData = receiveLocation.TransportTypeData;

                        doc = new XmlDocument();

                        doc.LoadXml(transportData);

                        root = doc.DocumentElement;

                        fileMaskNode = root.SelectSingleNode("FileMask");

                        if (fileMaskNode != null)

                        {

                            fileMaskNode.InnerText = fileMask;

                            receiveLocation.TransportTypeData = doc.OuterXml;

                            bceExplorer.SaveChanges();

                            returnValue = true;

                        }

                        bceExplorer.SaveChanges();

                        returnValue = true;

                    }

                }

            }

            catch (Exception e)

            {

                System.Diagnostics.EventLog.WriteEntry("UpdateReceiveLocationFileMask", e.Message);

                bceExplorer.DiscardChanges();

                returnValue = false;

            }

            return returnValue;

} 

The above code didn't worked, i have used Address property of the receive location and this trick worked for me.

public bool UpdateReceiveLocationFileMask(string receivePortName, string receiveLocationName, string fileMask)

{

            bool returnValue = false;

            string transportData = string.Empty;

            XmlDocument doc = null;

            XmlNode root = null;

            XmlNode fileMaskNode = null;

            try

            {

                if (!string.IsNullOrEmpty(fileMask))

                {

                    ReceivePort receivePort = bceExplorer.ReceivePorts[receivePortName];

                    ReceiveLocation receiveLocation = receivePort.ReceiveLocations.Cast<ReceiveLocation>().Where(item => item.Name == receiveLocationName).FirstOrDefault();

                    if (receiveLocation != null)

                    {

                        receiveLocation.Address = Path.Combine(Path.GetDirectoryName(receiveLocation.Address), fileMask);                        

                        bceExplorer.SaveChanges();

                        returnValue = true;

                    }

                }

            }

            catch (Exception e)

            {

                System.Diagnostics.EventLog.WriteEntry("UpdateReceiveLocationFileMask", e.Message);

                bceExplorer.DiscardChanges();

                returnValue = false;

            }

            return returnValue;

}

Property Promotion inside Orchestration

Properties get promoted within BizTalk at different places, Example: some of the properties gets promoted by Adapters, Pipelines Components, Messaging Engine, etc.

    In custom pipeline components, developers can use either IBaseMessageContext.Write to write context properties or IBaseMessageContext.Promote to promote context properties. The basic difference between writing and promoting properties are, properties that are written cannot be used for message routing, whereas properties that are promoted can be used for message routing.

Below is the sample for promoting properties within Orchestration is:
    MSG_OUT(ProjectNameSpace.PropertySchema.PropertyName) = "XXX"

If we look at the outgoing message context, the above prompted property will not promote and it would be in the "NotPromoted" state.


Please follow the below instructions to promote your property within the orchestration:

    In BizTalk we have a Correlation concept, basically which routes the response messages to the correct running orchestration instance that initiated the request message. Within Orchestration we use Correlation Set and Correlation type to archive this type of instance routing.

Correlation type is nothing but a set of Properties and Correlation Set is based on Correlation type and its a set of properties with specific values. When we "Initialize a Correlation set" within orchestration, the orchestration instance automatically promotes those properties in Correlation set into the message context.

We can use this mechanism to promote properties and there is no need to do a follow up of the Correlations sets we initialized, and it NOT going to create unnecessary subscriptions.

Modify your configuration file

Add "System.Configuration" reference to your application


 ExeConfigurationFileMap configFileMap = new ExeConfigurationFileMap();      
            configFileMap.ExeConfigFilename = @"{path}";

            Configuration config = ConfigurationManager.OpenMappedExeConfiguration(configFileMap, ConfigurationUserLevel.None);


            Console.WriteLine(config.AppSettings.Settings["YourKey"].Value);
            config.AppSettings.Settings["YourKey"].Value = DateTime.Now.Date.ToString("yyMMdd");
            config.Save();

Copy list to new list, while adding new value to list overwrites previous value in the old list

List<T> Constructor (IEnumerable<T>):
Initializes a new instance of the List<T> class that contains elements copied from the specified collection.

Sample:
            List<string> col1 = new List<string> { "R1", "R2", "R3" };
            List<string> col2 = col1;

            col2.Add("R4");

            Console.WriteLine("Col1 Count is:" + col1.Count);
            Console.WriteLine("Col2 Count is:" + col2.Count);
            Console.ReadLine();

The OutPut is:
Col1 Count is:4
Col2 Count is:4

            List<string> col1 = new List<string> { "R1", "R2", "R3" };
            List<string> col2 =  new List<string>(col1);

            col2.Add("R4");

            Console.WriteLine("Col1 Count is:" + col1.Count);
            Console.WriteLine("Col2 Count is:" + col2.Count);
            Console.ReadLine();

The OutPut is:
Col1 Count is:3
Col2 Count is:4

Custom File Adapter for Pooling Interval

The Pooling Interval property decide how long system should wait to poll the receive location again, if it get no message last time it polls.

Polling intervals in File/FTP adapters not works as expected, This is how it works
1. You drop 4 files at a time.
2. Biztalk File/FTP adapter reads 1 file at a time (according to batch size=1)
until the receive location is empty.
          This means while your first file being picked up, Biztalk FTP/File adapter identifies 3 more files are pending. So FTP/File adapter discards the polling interval and pick one after the other until your receive location is empty.

3.If the receive location is empty , it waits util the next polling interval.

I think the batching works well but it seem like it does not wait for the polling interval.

We want to change the behavior for the Pooling Interval property, it is all about performance. We don't want our CPU to be fully occupied with receiving messages because lots of them arrived is a short period of time, maybe making our system to work really slow, expiring timeout of other ports.

Microsoft released all the most important BizTalk parts with source code, we can locate those in C:\Program Files (x86)\Microsoft BizTalk Server 2010\SDK\Samples and in this directory exist the source code of the BizTalk file adapter, HTTP adapter, and we can find all the base class library here which will be useful to develop a new BizTalk Adapter.

we'll use the source code of the File Adapter (located in AdaptersDevelopment\File Adapter)

In the directory we'll find ,BizTalk Project directory with one BTS sample, Design Time directory is a project to manage adapter configuration, Runtime directory is the project of adapter runtime.

     Runtime project is the most important, open it, in this project you can observe the two principal files DotNetFileReceiverEndpoint.cs that contain the code for receive the stream and DotNetFileTransmitterEndpoint.cs that contain the code to transmit the stream.
In DotNetFileReceiverEndpoint.cs

• public void ControlledEndpointTask (object val)

        {
            if (this.controlledTermination.Enter())
            {
                try
                {
                    lock (this)
                    {
                        this.EndpointTask();                      
                    }
                    GC.Collect();                  
                }
                finally
                {
                    this.controlledTermination.Leave();
                    Thread.Sleep(new TimeSpan(0, 0, 0, ((int)this.properties.PollingInterval)));
                }
            }
        }

•     In PickupFilesAndSubmit(), modify the existing code with the below one.

         FileInfo[] items = di.GetFiles(this.properties.FileMask)
               Add System.Linq; namespace at the using block.
        FileInfo[] items = di.GetFiles(this.properties.FileMask).Take(maxNumberOfFiles).ToArray();
     

• please add the below code, to avoid The process cannot access the file because it is being used by another process error. 

               // If we couldn't lock the file, just move onto the next file
                IBaseMessage msg = CreateMessage(fileName, renamedFileName);
                             
                MemoryStream ms = new MemoryStream();
                msg.BodyPart.Data.CopyTo(ms);
                msg.BodyPart.Data.Position = 0;

                if ( null == msg )
                    continue;

                if ( null == renamedFileName )
                    files.Add(new BatchMessage(msg, fileName, BatchOperationType.Submit));
                else
                    files.Add(new BatchMessage(msg, renamedFileName, BatchOperationType.Submit));
             
                if (ms.CanSeek)
                {
                    ms.Position = 0;
                }
                msg.BodyPart.Data = ms;

                //  keep a running total for the current batch
                bytesInBatch += item.Length;

Now we can rebuild our adapter and have to register it.

To register the adapter we must execute the registry file StaticAdapterManagement.reg, we found some possible issues in this file.

The first is the directories, check all path directory because all are wrong:

• C:\Program Files\Microsoft BizTalk Server 2010\SDK\Samples\AdaptersDevelopment\FileAdapter\Runtime\bin\Debug\Microsoft.BizTalk.SDKSamples.Adapters.DotNetFile.Runtime.dll”

• “OutboundEngineCLSID”=”{024DB758-AAF9

• Is not FileAdapter but File Adapter

The second, check if your sample is under Program Files or Program Files (x86) in case modify the path or create your own different directory project.

The third and the most important is, if your BizTalk run in 64 bit mode you must modify the registry keys because BizTalk server in 64 bit search configuration under Wow6432Node

So, for example, if you want work with BizTalk configuration in 32 bit mode the correct string is

[HKEY_CLASSES_ROOTCLSID{62018D08-281A-415b-A6D3-6172E3762867}]

in 64 bit is

[HKEY_CLASSES_ROOTWow6432NodeCLSID{62018D08-281A-415b-A6D3-6172E3762867}]

In the BizTalk Admin console, right click on the adapters, new and select Static DotNetFile, for the adapter name as you want.

Restart the host instance

Create one receive port, one send with correct filter and test the solution

The result must be this

Here we have set the Number of files in Batch as 2 and Pooling interval as 60 seconds.