TPL Dataflow–An [Extremely Short] Introduction

by TheJet 16. April 2012 00:00

In the process of preparing my talk for the 2012 Fox Valley Day of .NET, I decided that the best way to get a reasonable talk laid out is to write a blog series about it Smile.  Maybe that means I’m weird, I don’t know, but it does mean that I’ll actually have some new entries for the blog after far, far too long.  I need to strike that balance between ‘long ass boring’ and ‘you are done already?’, hopefully this will be a good way to get there.  I’ll go into a lot more detail about the project and resulting solution here than I ever could in a talk, so I’ll apologize in advance if you’re attending and see something here you were hoping I’d elaborate on in my session.  If you see these, and have specific questions, I’ll be glad to answer them here or in person.

So, obviously, I’m not going to outline the history of multi-threaded programming, or even the complete history of multi-threaded programming in the context of the .NET Framework. I will start out trying to set the context for the world we’re currently faced with, and lay out what I hope to accomplish in this series. Future articles will delve more deeply into specific topics around Dataflow, and we’ll see our simple app hopefully mature into a full-fledged test-bed for Dataflow experimentation.

 

Where are we?

The 4.0 .NET Framework introduced a new concept for dealing with multi-threaded programming, the Task Parallel Library [or TPL from here on out].  The TPL sought to pull out the ‘good’ bits of parallel programming, from both best practices to burnt fingers, and roll them into a nice, easy-to-use framework.  For the most part, at least in my experience, Microsoft succeeded in this goal.  Certainly it wasn’t perfect, and there were bits missing [many of which appear in the .NET 4.5 preview], but it was an excellent ‘1.0’.

Those who used the TPL built up various solutions for retrieving the data to process, bundling it up, spinning up tasks to do the processing and generally ‘managing’ the flow of data through your application to get your job done.  Stephen Toub explained this approach very well in his 2011 //BUILD session, talking about the ‘data first’ nature of the existing TPL and some of the limitations of the approach.  TPL Dataflow [TDF] really turns things on their head a bit, switching to a ‘computation first’ model, whereby you setup the computation and then ‘wait’ for data to flow into the network for processing.

To be sure, everything you can do with TDF you can do with TPL or TPL+Rx or TPL+(your library of choice), and many of you have already built similar solutions to the framework.  What’s exciting about TDF is that a lot of the ‘stuff’ that needed to be managed as part of the ‘overhead’ of a project is now handled ‘out of the box’, and we can move from concept to solution quickly and spend more time on the rules/scalability/robustness/documentation side of the equation than we ever had time for previously.

 

What is TPL Dataflow [TDF] Anyway [in 100 words or less]?

TPL Dataflow is a set of Interfaces (IDataflowBlock, ITargetBlock, ISourceBlock and IPropagateBlock) and some implementing classes that can be used to build data processing networks.  The classes are located in the System.Threading.Tasks.Dataflow namespace, and are part of .NET Framework 4.5 [currently in beta].  The interfaces/classes are designed from the ground up to support multi-threaded, asynchronous programming and can be used to solve many classes of problems that were difficult to do with the TPL natively.

Sidebar: These are .NET 4.5 classes, so you WILL NOT be using them in new ‘Windows 8/RT Metro’ applications as they are not part of the surface area exposed by the new Windows Runtime.  You may be able to use them ‘wrapped’ in a WinRT component, but I haven’t explored that option yet, and I’m not sure that you’d want to in any case.  You WILL be using these classes on the ‘backend’ that you write to service all these ‘consumption’ Metro apps that you are building.

 

So Where am I Going With This?

The long term goal is a series of articles featuring these little guys (OK, so yes, they are currently big):

TPLemming

I’m sure you’re as pleased to meet them as they are to meet you. Smile  I hope to use these TPLemmings to help demonstrate how to use the various classes that TDF offers, and have a little fun at the same time!  Unfortunately you won’t be able to do any more than meet them today, but I do hope you’ll come back and say ‘Hi’.  After all they’ve got stuff to show you!

 

Conclusion

This article really serves as an introduction to the series, and gives you an idea of what TPL Dataflow is.  Future articles will cover the interfaces and classes supplied by TDF and go into detail on what types of problems they can solve.  And because I’d be remiss if I didn’t include _some_ ‘code’ in the article, I’ll close with this little snippet.

Save the image above to your HDD, and then create a new [empty] WPF application.  Include the PNG file in an ‘Assets’ folder in the project, and then paste the following XAML into MainWindow.xaml:

        <Image x:Name="lemming" Source="Assets/TPLemming.png" Stretch="None" HorizontalAlignment="Center" RenderTransformOrigin="0.5,0.5">
            <Image.Resources>
            <Storyboard x:Key="Walk" AutoReverse="True" RepeatBehavior="Forever">
                    <DoubleAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.RenderTransform).(TranslateTransform.X)" Storyboard.TargetName="lemming">
                        <DiscreteDoubleKeyFrame KeyTime="0:0:0.5" Value="-55" />
                        <DiscreteDoubleKeyFrame KeyTime="0:0:1" Value="61" />
                    </DoubleAnimationUsingKeyFrames>
                    <PointAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.Clip).(PathGeometry.Figures)[0].(PathFigure.StartPoint)" Storyboard.TargetName="lemming">
                    <PointKeyFrameCollection>
                        <DiscretePointKeyFrame KeyTime="0:0:0.5" Value="122,0" />
                        <DiscretePointKeyFrame KeyTime="0:0:1" Value="0,0" />
                    </PointKeyFrameCollection>
                </PointAnimationUsingKeyFrames>
                <PointAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.Clip).(PathGeometry.Figures)[0].(PathFigure.Segments)[0].(LineSegment.Point)" Storyboard.TargetName="lemming">
                    <PointKeyFrameCollection>
                        <DiscretePointKeyFrame KeyTime="0:0:0.5" Value="244,0" />
                        <DiscretePointKeyFrame KeyTime="0:0:1" Value="122,0" />
                    </PointKeyFrameCollection>
                </PointAnimationUsingKeyFrames>
                <PointAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.Clip).(PathGeometry.Figures)[0].(PathFigure.Segments)[1].(LineSegment.Point)" Storyboard.TargetName="lemming">
                    <PointKeyFrameCollection>
                        <DiscretePointKeyFrame KeyTime="0:0:0.5" Value="244,244" />
                        <DiscretePointKeyFrame KeyTime="0:0:1" Value="122,244" />
                    </PointKeyFrameCollection>
                </PointAnimationUsingKeyFrames>
                <PointAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.Clip).(PathGeometry.Figures)[0].(PathFigure.Segments)[2].(LineSegment.Point)" Storyboard.TargetName="lemming">
                    <PointKeyFrameCollection>
                        <DiscretePointKeyFrame KeyTime="0:0:0.5" Value="122,244" />
                        <DiscretePointKeyFrame KeyTime="0:0:1" Value="0,244" />
                    </PointKeyFrameCollection>
                </PointAnimationUsingKeyFrames>
            </Storyboard>
            </Image.Resources>
            <Image.Triggers>
                <EventTrigger RoutedEvent="FrameworkElement.Loaded">
                    <BeginStoryboard Storyboard="{StaticResource Walk}" />
                </EventTrigger>
            </Image.Triggers>

            <Image.RenderTransform>
                 <TranslateTransform X="61"/>
            </Image.RenderTransform>
            <Image.Clip>
                <PathGeometry>
                    <PathFigure IsClosed="True" StartPoint="0,0">
                        <LineSegment Point="122,0" />
                        <LineSegment Point="122,244" />
                        <LineSegment Point="0,244" />
                    </PathFigure>
                </PathGeometry>
            </Image.Clip>
        </Image>

 

Enjoy!

Binding Flagged Enumerations

by TheJet 13. October 2011 06:23

Introduction

Recently a tweet brought my attention to this blog post by @san0x, regarding the difficulties of binding flagged enumerations to UI fields in a meaningful and easy-to-use fashion.  The post was largely dealing with ways to work around the limitation of the IValueConverter, and its ‘context agnostic’ processing of values during the conversion process.  The IValueConverter does not know the context of the value being converted, only the value itself and an [unbindable] conversion parameter.

Scenario

I’ll avoid going into great detail about the scenario here, as it’s explained in detail in the linked post above, but the main focus is binding a series of checkboxes to a single flagged enumeration value.  The classes I’ll use in my example are adapted from the original post, I’ve included them here as a convenience to the reader.

    public class Species : ViewModelBase
    {
        /// 
        /// The  property's name.
        /// 
        public const string NamePropertyName = "Name";

        private string _name = String.Empty;

        /// 
        /// The species name
        /// 
        public string Name
        {
            get
            {
                return _name;
            }

            set
            {
                if (_name == value)
                {
                    return;
                }

                var oldValue = _name;
                _name = value;

                // Update bindings, no broadcast
                RaisePropertyChanged(NamePropertyName);
            }
        }

        /// 
        /// The  property's name.
        /// 
        public const string FlagsPropertyName = "Flags";

        private SpeciesFlag _flags = SpeciesFlag.None;
        
        /// 
        /// The properties of the species, as a flagged enumeration
        /// 
        public SpeciesFlag Flags
        {
            get
            {
                return _flags;
            }

            set
            {
                if (_flags == value)
                {
                    return;
                }

                var oldValue = _flags;
                _flags = value;

                // Update bindings, no broadcast
                RaisePropertyChanged(FlagsPropertyName);
            }
        }
    }

 

    [Flags]
    public enum SpeciesFlag : byte
    {
        None = 0,
        BreathesAir = 0x1,
        BreathesWater = 0x2,
        Photosynthesizes = 0x4,
        Eats = 0x8
    }

Note: My examples use the excellent MVVM Light Toolkit by @LBugnion, but are NOT intended to be examples of best practices when using the toolkit

 

Potential Solutions

Many developers [myself included], when presented with these types of problems, take a similar route to @san0x, attempting to coerce the value converter into something that is context-aware in any number of ways.  My personal investigations led me to first investigate using WPF MultiBinding [a little used, but occasionally very handy feature].  The promise of the MultiBinding was that I could pass in a number of bound values, and get back a suitable result [true/false].  As things usually go [for me at least], this worked fabulously for the first 50% of the problem, binding from the source –> the CheckBox.IsSelected property worked just as expected with a binding very similar to this:

<CheckBox.IsChecked>
    <MultiBinding Mode="TwoWay" 
                   Converter="{StaticResource FlaggedEnumToBoolMultiConverter}" ConverterParameter="BreathesAir">
      <Binding Path="Species" />
      <Binding Path="Species.Flags" />
    </MultiBinding>
</CheckBox.IsChecked>

 

This allowed me to reflect the flag value on the CheckBox, and changes to the value were automatically reflected on the UI without a lot of additional work.  The issue, of course, was how to update the underlying object when the CheckBox itself is used to change the value.  This is much harder, because the IMultiValueConverter::ConvertBack method has two problems:

  1. Only a single [boolean] value, and [object] parameter are passed to the ConvertBack method.
  2. ConvertBack must pass back an array of values, one for each Binding contained in the MultiBinding.

So… we’re stuck… there’s no way to convert from a single boolean back into a Species and its flags while only effecting the single ‘bit’ or ‘bits’ we care about.  We simply do not have enough information available to us as the time of conversion.  Sure, there are workarounds, and one such workaround is listed in @san0x’s follow-up post [using a standard IValueConverter], but in the end, those workarounds all ‘smell’ like hacks to me.  They all involve keeping the context of the binding around in some fashion during conversion.

Reframing the Problem

My suggestion on Twitter was to utilize the ViewModel to expose the individual flags as bindable properties to the View.  This successfully reframes the problem, but does suffer from the requirement that you update your ViewModel every time your enumeration changes [the View needs to be updated with any solution, since we’re using individual CheckBox(s)].  This ‘smells’ better to me simply because it does what ViewModel’s are supposed to do, abstract away the underlying model when necessary to make data binding feasible.  It also doesn’t resort to any of the same hacks required to get binding to work properly, and context is retained because it’s handled by the bound view model.

Proposed Solution

So with the reframing of the question in mind, I moved on to whether or not it would be possible to simplify the process of binding to that flagged enumeration.  I wanted to minimize coding effort, and have the solution work for any general flagged enumeration.  Back in the .NET 1.x/2.x days, I had worked on a number of projects that leveraged the ICustomTypeDescriptor and it’s siblings to provide customized design and run-time behavior for objects and properties.  Silverlight 5 and .NET 4.5 bring the ICustomTypeProvider to bear on the problem, and could be used in those environments.  For this post, since we’re running WPF and .NET 4.0, I’m going to utilize DynamicObject to provide for runtime binding of the flagged enumeration values.

The FlaggedEnumViewModel Class

I’ll start by introducing the class that enables runtime binding and implements the bits necessary for the bindings to resolve properly:

    public class FlaggedEnumViewModel<TEnum>: DynamicObject, INotifyPropertyChanged
    {
        private Func<TEnum> _getter;
        private Action<TEnum> _setter;

        public FlaggedEnumViewModel(Func<TEnum> getter, Action<TEnum> setter)
        {
            if (!typeof(TEnum).IsEnum) 
                throw new ArgumentException("This type must only be used with enumerations.");
            
            _getter = getter;
            _setter = setter;
        }

        public override DynamicMetaObject GetMetaObject(System.Linq.Expressions.Expression parameter)
        {
            return base.GetMetaObject(parameter);
        }
        public override IEnumerable<string> GetDynamicMemberNames()
        {
            return Enum.GetNames(typeof(TEnum)).Select(i => "Has" + i);
        }

        public override bool TryGetMember(GetMemberBinder binder, out object result)
        {
            if (!binder.Name.ToLower().StartsWith("has") || binder.Name.Length < 4)
            {
                result = null;
                return false;
            }

            long checkedFlag = Convert.ToInt64(Enum.Parse(typeof(TEnum), binder.Name.Substring(3), true));
            long currentValue = Convert.ToInt64(_getter());

            
            result = (checkedFlag & currentValue) != 0;
            return true;
        }

        public override bool TrySetMember(SetMemberBinder binder, object value)
        {
            if (!binder.Name.ToLower().StartsWith("has") || binder.Name.Length < 4)
            {
                return false;
            }

            long checkedFlag = Convert.ToInt64(Enum.Parse(typeof(TEnum), binder.Name.Substring(3), true));
            long currentValue = Convert.ToInt64(_getter());

            if ((bool)value == true)
            {
                currentValue |= checkedFlag;
            }
            else
            {
                currentValue &= ~checkedFlag;
            }

            
            _setter((TEnum)Enum.ToObject(typeof(TEnum), currentValue));

            OnPropertyChanged(binder.Name);

            return true;
        }

        protected void OnPropertyChanged(string propertyName)
        {
            if (PropertyChanged != null)
            {
                PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
            }
        }

        public event PropertyChangedEventHandler PropertyChanged;
    }

 

In the above code, I’ll call attention to three bits:

  1. The constructor takes a ‘getter’ and ‘setter’ argument that wraps access to the underlying flagged enum store.
  2. The TryGetMember method handles converting from the flagged value to a boolean
  3. The TrySetMember method handles converting from the boolean back to the flagged value

 

The View/ViewModel

Utilizing the class is fairly straightforward, in my ViewModel, I’ve exposed a property [WrappedFlags] of the FlaggedEnumViewModel type that I use for binding on the UI, and the ‘Species’ property notifies that both properties change whenever the Species property changes.  This is sufficient to enable TwoWay binding on the View.

    public class MainViewModel : ViewModelBase
    {
        public string Welcome
        {
            get
            {
                return "Sample Flagged Enum Editor";
            }
        }

        /// <summary>
        /// Initializes a new instance of the MainViewModel class.
        /// </summary>
        public MainViewModel()
        {
            if (IsInDesignMode)
            {
                // Code runs in Blend --> create design time data.
            }
            else
            {
                // Code runs "for real"
            }

            LoadRandomSpecies = new RelayCommand(() => AssignRandomSpecies());
            WrappedFlags = new FlaggedEnumViewModel<SpeciesFlag>(
                                   () => Species.Flags, 
                                   (v) => Species.Flags = v);
        }

        /// <summary>
        /// The <see cref="Species" /> property's name.
        /// </summary>
        public const string SpeciesPropertyName = "Species";

        private Species _species = new Species() { Name = "Sample Species" };

        /// <summary>
        /// The Species we're working with
        /// </summary>
        public Species Species
        {
            get
            {
                return _species;
            }

            set
            {
                if (_species == value)
                {
                    return;
                }

                var oldValue = _species;
                _species = value;

                // Update bindings, no broadcast
                RaisePropertyChanged(SpeciesPropertyName);
                RaisePropertyChanged("WrappedFlags");
            }
        }

        public FlaggedEnumViewModel<SpeciesFlag> WrappedFlags { get; private set; }

        public RelayCommand LoadRandomSpecies { get; private set; }

        protected void AssignRandomSpecies()
        {
            Random rnd = new Random();
            byte value = (byte)rnd.Next(0, 16);

            Species = new Species()
            {
                Name = "Random Species " + value.ToString(),
                Flags = (SpeciesFlag)value
            };
        }
    }

 

Lastly, the view code is pretty basic, I’ll just call out one of the CheckBox elements to show how the binding was done:

    <CheckBox Content="Breathes Air" 
               IsChecked="{Binding Path=WrappedFlags.HasBreathesAir, Mode=TwoWay}" />

 

Conclusion

Transitioning from a converter-based approach to enhancing the ViewModel makes the problem much easier to implement and troubleshoot, it also avoids many of the pitfalls of introducing state to your I[Multi]ValueConverter instances.  Utilizing a DynamicObject class derivative enables you to write much less code and get the same overall benefit at runtime.  In a future post, I hope to find time to walk through enhancing the design-time experience to support setting up bindings in the VS2010 XAML designer.  I look forward to hearing your comments, or seeing alternative solutions to the problem.

Thanks for reading!

Sample Code: WPFBindingFlaggedEnum-20111013.zip (19 kb)

Who am I?

Me

Husband/Father, Solutions Architect [MCPD/MCSD], .NET Developer, Tech Enthusiast

 

Blogging primarily on Silverlight/WPF development, with an occasional foray into all things .NET, including Windows Phone 7, RIA Services, ASP.NET, etc.  And yes, some day, I'll even come up with a non-canned blog theme :)

 

Twitter: @virtualolympus

LinkedIn: Ben Gavin