Filtering pipes

A common Pipes pattern is the filtering of objects. This section discusses the basics of filtering and then explains the interfaces and their implementations.

Basic Filtering

A filter-based pipe will consume objects and either emit them or not. If they are not emitted, then they are considered filtered by the pipe. A useful interface to implement that describes this behavior is IFilterPipe<S> extending IPipe<S, S>. A basic filter is defined below. This pipe is called RandomFilterPipe and it comes with the Pipes distribution. RandomFilterPipe will only allow a consumed object to be emitted if a biased coin toss lands on “heads.” At the extremes, if bias is 0.0 then no incoming objects are emitted and if bias is 1.0, then every incoming object is emitted.

public class RandomFilterPipe<S> : AbstractPipe<S, S>, IFilterPipe<S>
{

    private static readonly Random _Random = new Random();
    private        readonly Double _Bias;

    public RandomFilterPipe(Double myBias)
    {
        _Bias = myBias;
    }

    public override Boolean MoveNext()
    {

        if (_InternalEnumerator == null)
            return false;

        while (_InternalEnumerator.MoveNext())
        {

            if (_Bias >= _Random.NextDouble())
            {
                _CurrentElement = _InternalEnumerator.Current;
                return true;
            }

        }

        return false;

    }

}

The MoveNext() method structure above is a common pattern in filter-based pipes. In short, a while(_InternalEnumerator.MoveNext()) { ... } is evaluated until an incoming source object meets the criteria and is returned (i.e. emitted). If there are no more source objects and the criteria is not met, then the routine will return false. Beware of using recursion to accomplish a similar behavior. As the amount of data grows that is streaming through a process graph, recursion-based methods can easily yield a StackOverflowException.

Creating a Pipe home Comparison Based Filtering