220017 ‘Woalktrough: Creating a Custom Datatlow Block Type Walkthrough: Creating a Custom Dataflow Block Type -NET Framework (current version) Although the TPL Dataflow Library provides several dataflow block types that enable a variety of functionality, you can also create custom block types. This document describes how to create a dataflow block type that implements custom behavior. Prerequisites Read Dataflow before you read this document. VTi The TPL Dataflow Library (System.Threading,Tasks Dataflow namespace) is not distributed with the NET Framework 4.5. To install the System-Threading Tasks Dataflow namespace, open your project in Visual Studio 2012, choose Manage NuGet Packages from the Project menu, and search online for the Microsoft. Tpl.Dataflow package Defining the Sliding Window Dataflow Block Consider a dataflow application that requires that input values be buffered and then output in a sliding window manner. For example, for the input values (0, 1, 2, 3, 4, 5} and a window size of three, a sliding window dataflow block produces the output arrays (0, 1, 2), (1, 2, 3), (2,3, 4), and (3, 4,5). The following sections describe two ways to create a dataflow block type that. implements this custom behavior. The first technique uses the Encapsulate method to combine the functionality of an ISourceBlock object and an ITargetBlock and combines existing functionality to perform custom behavior. Using the Encapsulate Method to Define the Sliding Window Dataflow Block The following example uses the Encapsulate method to create a propagator block from a target and a source. A propagator block enables a source block and a target black to act as a receiver and sender of data, ‘This technique is useful when you require custom dataflow functionality, but you do not require a type that provides additional methods, properties, or fields vB * Creates a IPropagatorBlock object propagates data in a * sliding window fashion. Public Shared Function CreateSlidingWindow(Of T) (ByVal windowSize As Integer) As PropagatorBlock(Of T, T()) * Create a queue to hold messages. Din queue = New Queue(OF T)() * The source part of the propagator holds arrays of size windowSize * and propagates data out to any connected targets. Dim source = New BufferBlock(Of T())() "The target part receives data and adds them to the queue. Dim target = New ActionBlock(OF T)(Sub(item) "Add the item to the queue Remove the oldest item when the queue size exceeds the window size. Post the data in the queue to the source block when the queue size equals the window size. hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspx 18s22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type queue. Enqueue( item) TF queve.Count > windowSize Then queue .Dequeue() End TF If queue.Count = windowSize Then source. Post (queue. ToArray()) End If End Sub) "When the target is set to the completed state, propagate out any * pemaining data and set the source to the completed state. target.Completion.ContinueWi th(Sub() IF queue.Count > @ AndAlso queue.Count < windowSize Then source.Post (queue. Toarray()) End If source.Complete() End Sub) Return a TPropagatorBlock object that encapsulates the target and source blocks. Return DataflowBlock.Encapsulate(target, source) End Function Deriving from IPropagatorBlock to Define the Sliding Window Dataflow Block ‘The following example shows the SLidingWindowBllock class. This lass derives from IPropagatorBlock so that it can act as both a source and a target of data. As in the previous example, the SLidingWindowBlock class is built on existing dataflow block types. However, the SLidingWindowBLock class also implements the methods that are required by the ISourceBlock,ITargetBlock, and DataflowBlock interfaces. These methods all forward work to the predefined dataflow block type members. For example, the Post method defers work to the m_target data member, which is also an HTargetBlock object. ‘This technique is useful when you require custom dataflow functionality, and also require a type that provides additional methods, properties, or fields, For example, the SlidingWindowBlock class also derives from IReceivableSourceBlock so that it can provide the TryReceive and TryReceiveAll methods, The SLidingwindowBlock class also demonstrates extensibility by providing the WindowSize property, which retrieves the number of elements in the sliding window. vB * propagates data in a sliding window fashion. Public Class SlidingWindowBlock(Of T) Implenents TPropagatorBlock(Of T, T()), TReceivableSourceBlock(Of 7()) "The size of the window. Private ReadOnly mwindowSize As Integer * The target part of the block. Private ReadOnly mtarget As TTargetBlock(Of T) "The source part of the block Private ReadOnly m_source As IReceivableSourceslock (Of T()) * Constructs a SlidingWindowBlock object. Public Sub New(ByVal windowSize As Integer) "Create a queue to hold messages. Dim queue = New Queue(of T)() "The source part of the propagator holds arrays of size windowSize ‘and propagates data out to any connected targets. Din source = New BufferBlock(Of T())() The target part receives data and adds then to the queue. Dim target = New ActionBlock(OF T) (Sub(item) Add the item to the queue. Renove the oldest item when the queue size exceeds the window size. " Post the data in the queue to the source block when the queue size "equals the window size. queue. Enqueue( item) If queue.Count > windowSize Then hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspx 28s22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type ‘queue Dequeue() End TF Tf queve.Count = windowSize Then source. Post (queue, ToArray()) End If End Sub) "When the target is set to the completed state, propagate out any " remaining data and set the source to the conpleted state. ‘target .Conpletion..Continuediith(sub() TF queue.Count > @ AndAlso queue.Count < windowSize Then source.Post (queue. ToArray()) End IF source.Complete() End sub) m_windowSize = windowsize m_target End sub Retrieves the size of the window. Public ReadOnly Property WindowSize() As Integer Get Return m_windowSize End Get End Property “wRegion “IReceivableSourceBlock members" * attempts to synchronously receive an item from the source. Public Function TryReceive(Byval filter As Predicate(Of T()), ByRef item() As T) As Boolean Implements IReceivableSourceBlock(Of T()).TryReceive Return m_source.TryReceive(filter, item) End Function * attempts to remove all available elements from the source into a new * array that is returned. Public Function TryReceiveAll ( ByRef itens As IList(OF T())) As Boolean Implements TReceivableSourceBlock(OF T()).TryReceiveAl1 Return m_source. TryReceiveAll (items) End Function "End Region #Region "ISourceBlock menbers Links this dataflow block to the provided target. Public Function LinkTo(Byval target As ITargetBlock(Of T()), ByVal LinkOptions As DataflowLinkoptions) As IDisposable Inplenents 1SourceBlock(Of T()).LinkTo Return m_source.LinkTo(target, linkOptions) End Function Called by a target to reserve a message previously offered by a source * but not yet consumed by this target. Private Function ReserveMessage(SyVal messageHeader As DataflowMessageHeader, ByVal target As ITargetBlock(Of T())) As Boolean Implenents TSourceBlock(Of T()).ReserveMessage Return m_source.ReserveMessage(messageHeader, target) End Function * called by a target to consume a previously offered message from a source. Private Function ConsumeMessage(yVal messageHeader As DataflowMessageHeader, ByVal target As ITargetBlock(Of T()), ByRef messageConsumed As Boolean) As T() Inplements TSourceBlock(Of T()).ConsuneMessage Return m_source.ConsumeMessage(messageHeader, target, messageConsumed) End Function * Called by a target to release a previously reserved message from a source. Private Sub ReleaseReservation(Syval messageHeader As DataflowMessageHeader, ByVal target As TTargetBlock(Of T())) Implements TSourceBlock(Of T()).ReleaseReservation m_source.ReleaseReservation(messageHeader, target) End sub hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspxs22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type end Region Region "ITargetBlock members: * Asynchronously passes a message to the target block, giving the target the * opportunity to consume the message. Private Function OfferMessage(ByVal messageHeader As DataflowMessageHeader, ByVal messageValue As T, ByVal source As ISourceBlock(Of T), ByVal consumeToAccept As Boolean) As DataflowessageStatus Inplenents ITargetBlock (Of 1) .OFferMessage Return m_target.OfferMessage(messageHeader, messageValue, source, consuneToAccept) End Function #End Region #Region "IDataflowBlock members" " Gets a Task that represents the completion of this dataflow block. Public ReadOnly Property Completion() As Task Inplenents IDataflowBlock.Completion Get Return m_source.Conpletion End Get End Property * Signals to this target block that it should not accept any more messages, * nor consune postponed messages. Public Sub Conplete() Inplenents IDataflowBlock.Complete n_target .conplete() End sub Public Sub Fault(ByVal [error] As Exception) Implements IDataflowBlock.Fault m_target.Fault(Lerror]) End sub end Region End Class The Complete Example ‘The following example shows the complete code for this walkthrough. It also demonstrates how to use the both sliding window blocks in a method that writes to the block, reads from it, and prints the results to the console. ve Imports System Imports System.Collections .Generic Imports System.Ling Imports Systen. Threading. Tasks Imports System. Threading. Tasks.Dataflow * Demonstrates how to create a custom dataflow block type. Friend Class Program * Creates a TPropagatorBlock object propagates data in a * sliding window fashion. Public Shared Function CreateSlidingWindow(Of T) (ByVal windowSize As Integer) As PropagatorBlock(Of T, T()) * Create a queue to hold messages. Dim queue = New Queue(Of T)() * The source part of the propagator holds arrays of size windowSize * and propagates data out to any connected targets. Dim source = New BufferBlock(Of T())() "The target part receives data and adds them to the queue Dim target = New ActionBlock(Of T)(Sub(item) "Add the item to the queue " Remove the oldest item when the queue size exceeds the window size. " Post the data in the queue to the source block when the queue size * equals the window size. queue. Enqueue( item) hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspx 48s22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type If queue.Count > windowSize Then queue .Dequeue() End IF TF queue.Count = windowsize Then source. Post (queue. ToArray()) End TF End Sub) "hen the target is set to the completed state, propagate out any " remaining data and set the source to the completed state. target.Completion.ContinueWi th(Sub() TF queue.Count > @ AndAlso queue.Count < windowsSize Then source.Post (queue. ToArray()) Eng If source.Complete() End sub) Return a IPropagatorBlock object that encapsulates the ‘target and source blocks. Return DataflowBlock.Encapsulate(target, source) End Function * propagates data in a sliding window fashion. Public Class SlidinghindowBlock(Of T) Inplenents TPropagatorBlock(Of T, T()), IReceivableSourceBlock(Of 7()) "The size of the window. Private ReadOnly mwindowSize As Integer * The target part of the block. Private ReadOnly mtarget As TTargetBlock(Of T) * The source part of the block. Private ReadOnly msource As TReceivableSourceslock (Of T()) * Constructs a SlidingWindowBlock object. Public Sub New(ByVal windowSize As Integer) * Create a queue to hold messages. Dim queue = New Queue(OF T)() "The source part of the propagator holds arrays of size windowSize “and propagates data out to any connected targets. Dim source = New BufferBlock(Of T())() "The target part receives data and adds then to the queue. Din target = New ActionBlock(OF T)(Sub(iten) "add the item to the queue. * Renove the oldest item when the queue size exceeds the window size. "Post the data in the queue to the source block when the queue size * equals the window size. queue. Enqueue( item) Tf queve.Count > windowSize Then ‘queue. Dequeue() End If If queve.Count = windowSize Then source.Post(queue. ToArray()) End If End Sub) linen the target is set to the completed state, propagate out any remaining data and set the source to the completed state. ‘target .Conpletion.Continuedith(sub() Tf queue.Count > @ AndAlso queue.Count < windowSize Then source.Post (queue. ToArray()) End IF source. Conplete() End Sub) mwindowSize = windowsize m_target m_source End sub * Retrieves the size of the window. Public ReadOnly Property WindowSize() As Integer Get Return m_windowsize hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspxs22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type End Get End Property “wRegion “IReceivableSourceBlock members” attempts to synchronously receive an item from the source, Public Function TryReceive(Byval filter As Predicate(Of T()), ByRef item() As T) As Boolean Implements IReceivableSourceBlock(OF T()).TryReceive Return m_source.TryReceive(filter, iten) End Function * attempts to remove all available elements from the source into a new array that is returned, Public Function TryReceiveAll ( ByRef items As TList(OF T())) As Boolean Inplenents TReceivableSourceBlock(OF T()).TryReceivesl] Return m_source. TryReceiveAll (itens) End Function End Region Region "ISourceBlock members’ * Links this dataflow block to the provided target. Public Function LinkTo(ByVal target As ITargetBlock(Of 1()), ByVal Linkoptions As DataflowLinkoptions) As IDisposable Inplenents 1SourceBlock(Of T()).LinkTo Return m_source.LinkTo(target, linkoptions) End Function * called by a target to reserve a message previously offered by a source * but not yet consumed by this target. Private Function ReserveMessage(yVal messageHeader As DataflowMessageHeader, ByVal target As ITargetBlock(Of T())) As Boolean Implenents TSourceBlock(Of T())-ReserveMessage Return m_source. ReserveNessage(messageHeader, target) End Function * called by a target to consume a previously offered message from a source. Private Function ConsumeMessage(ByVal messageHeader As DataflowMessageHeader, 8yVal target As ITargetBlock(Of T()), ByRef messageConsumed As Boolean) As T() Implements ISourceBlock(OF T()).ConsumeMessage Return m_source.ConsumeMessage(messageHeader, target, messageConsumed) End Function " called by a target to release a previously reserved message from a source. Private Sub ReleaseReservation(Syval messageHeader As DataflowMessageHeader, ByVal target As ITargetBlock(Of T())) Implements ISourceBlock(OF T()).ReleaseReservation m_source.ReleaseReservation(messageHeader, target) End sub #End Region Region "ITargetBlock members’ Asynchronously passes a message to the target block, giving the target the opportunity to consume the message. Private Function OfferMessage(SyVal messageHeader As DataflowMessageHeader, ByVal nessageValue As T, ByVal source As ISourceBlock(Of T), ByVal consumeToAccept As Boolean) As DataflowMessageStatus Inplenents ITargetSlock(Of T).OfferMessage Return m_target.OfferMessage(messageHeader, messageValue, source, consumeToAccept) End Function #End Region Region "IDataflowBlock members” * Gets a Task that represents the completion of this dataflow block. Public ReadOnly Property Completion() As Task Inplenents IDataflowBlock. Completion Get Return m_source.Conpletion End Get End Property * Signals to this target block that it should not accept any more messages, * nor consume postponed messages hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspxs22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type Public Sub Conplete() Implements IDataflowBlock.Complete m_target .Complete() End sub Public Sub Fault(ByVal [error] As Exception) Implements IDataflowBlock.Fault m_target .Fault([error]) End sub #End Region End Class * benonstrates usage of the sliding window block by sending the provided * values to the provided propagator block and printing the output of that block to the console. Private Shared Sub DenonstrateSlidingwindow(Of T)(ByVal slidinghindow As TPropagatorslock(Of T, T()), ByVal values As TEnunerable(OF T)) Create an action block that prints arrays of data to the console. Din windowConna As String = String-Enpty Din printiindow = New ActionBlock(OF T())(Sub (window) Console.write(windowConma) Console.write("(") Din conma As String = String.Enpty For Each item As T In window Console.Write(comma) Console.Write(item) Next item Console.write("}") windowComma 1 End Sub) * Link the printer block to the sliding window block. sLidingWindow. LinkTo(printwindow) * Set the printer block to the completed state when the sliding window * block completes. ‘sLidingWindow.Completion.ContinueWith(Sub() printWindow.complete()) * Print an additional newline to the console when the printer block completes Dim completion = printWindow.Completion.continueWith(sub() Console.WriteLine()) * Post the provided values to the sliding window block and then wait * for the sliding window block to complete. For Each value As T In values sLidingwindow. Post (value) Next value ‘sLidingWindow.complete() Wait for the printer to complete and perform its final action. conpletion.wait() End sub Shared Sub Main(Byval args() As String) Console.Write("Using the DataflowBlockExtensions.Encapsulate method ") Console.WriteLine("(T=int, windowSize=3):") DenonstrateSlidingWindow(CreateSlidingWindow(Of Integer)(3), Enumerable.Range(®, 10)) Console.writeLine() Dim slidingWindow = New SlidinghindowBlock(of char)(4) Console.Write("Using SlidingWindowBlock ") Console.WriteLine("(T=char, windowSize=(8}):", slidingWindow.WindowSize) DenonstrateSlidingwindow(slidingwindow, _ Fron n In Enunerable.Range(65, 10) _ Select ChrW(n)) End sub End Class * output: “sing the DataflowBlockExtensions.Encapsulate method (T=int, windowSiz "{81,2}, (12,3), (23,4), (3,4,5}, (4,546), (5,657), (6,78), (7,8,9} hnipssimsch microsoft comen-usibraryhh228608{ eprint v=vs. 110). aspx 78s22n017 ‘Woalktrough: Creating a Custom Datatlow Block Type “Using SlidingWindowBlock (T=char, windowSize=4) “{A,B,C,D}, {B,C,0,E}, {C,D,E,F}, (D,E,F,G), CEF.GH}, (F,GH,T}, (6H,1,3) Compiling the Code Copy the example code and paste it in a Visual Studio project, or past iin a file that is named S1idingWindowBlock.cs (Slidingwindowlock. vb fr Visual Basic) ané then run the following command in a Visual Studio Command Prompt window. Visual cH «se.exe /r-System.Threading,Tasks.Dataflow.dl SidingWindowBlock.cs Visual Basie vbc.exe :System.Threading.Tasks.Dataflow.dll SlidingWindowBlock.vb Next Steps See Also Dataflow © 2017 Microsoft hnpssimsch microsoft coment aryhh22860e{deprirarv=vs. 110). aspx 88