How to speed up comparisons?

[jzabroski]

1. I am getting an error that says two objects are not the same type - it would be useful when comparing deeply nested object graphs to know what types and property names are being compared.
2. Hitting Pause in the debugger reveals some somewhat surprising hotspots
3. What if there was a C# Source Generator for comparisons?

In my example case, I am talking to a FinTech gRPC API and trying to validate that post-save security response I get back from the vendor is the same as the response I get from fetching the latest version. For some reason, I had to add ComparisonOptions.AllowCompareDifferentObjects even though the top-level objects are the same gRPC object. I'm assuming that somewhere in my sub-graph, I have a polymorphic collection that contains a different sub-type, .e.g.

message Value {
  oneof oneOfValue {
    string stringValue = 1;
    bool boolValue = 2;
    int32 intValue = 3;
    int64 longValue = 4;
    double doubleValue = 5;
    CustomTypedValue customTypedValue = 6;
  }
}

Below is my example code - it has been running for at least 8 minutes just to compare one relatively tiny object graph, on .NET Framework 4.8.2. - Maybe the issue here is with slow reflection on an older framework, but I was not expecting it to be this slow.

var comparisonResult = AnyDiff.AnyDiff.Diff(savedSecurity, latestSecurity, ComparisonOptions.CompareProperties | ComparisonOptions.CompareCollections | ComparisonOptions.AllowCompareDifferentObjects);

[jzabroski]

As an update, it returned after about 20 minutes.

The ComparisonResult object was somewhat shocking.

var comparisonResult = AnyDiff.AnyDiff.Diff(savedSecurity, latestSecurity, ComparisonOptions.CompareProperties | ComparisonOptions.CompareCollections | ComparisonOptions.AllowCompareDifferentObjects);
var difference = comparisonResult.FirstOrDefault(x => x.Delta != null);
Console.WriteLine($"Index: {difference.ArrayIndex}"); // when array elements differ in value
  Console.WriteLine($"Delta: {difference.Delta}"); // when numbers, Dates, Timespans, strings differ in value
  Console.WriteLine($"Left: {difference.LeftValue}"); // the left value being compared
  Console.WriteLine($"Right: {difference.RightValue}"); // the right value being compared
  Console.WriteLine($"Property name: {difference.Property}"); // the name of the field/property
  Console.WriteLine($"Property type: {difference.PropertyType}"); // the type of the field/property

outputs

Index:
Delta: +get_RemotingCache
-GetType

Left: GetType
Right: get_RemotingCache
Property name: MethodName
Property type: System.String

Is this a problem with using AnyDiff with gRPC object trees?

[jzabroski]

I think this problem can probably be replicated via generating the gRPC class from the gRPC Value proto message, which gets you:

  public sealed partial class Value : pb::IMessage<Value>
  #if !GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE
      , pb::IBufferMessage
  #endif
  {
    private static readonly pb::MessageParser<Value> _parser = new pb::MessageParser<Value>(() => new Value());
    private pb::UnknownFieldSet _unknownFields;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public static pb::MessageParser<Value> Parser { get { return _parser; } }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public static pbr::MessageDescriptor Descriptor {
      get { return global::Ft.CommonReflection.Descriptor.MessageTypes[5]; }
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    pbr::MessageDescriptor pb::IMessage.Descriptor {
      get { return Descriptor; }
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public Value() {
      OnConstruction();
    }

    partial void OnConstruction();

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public Value(Value other) : this() {
      switch (other.OneOfValueCase) {
        case OneOfValueOneofCase.StringValue:
          StringValue = other.StringValue;
          break;
        case OneOfValueOneofCase.BoolValue:
          BoolValue = other.BoolValue;
          break;
        case OneOfValueOneofCase.IntValue:
          IntValue = other.IntValue;
          break;
        case OneOfValueOneofCase.LongValue:
          LongValue = other.LongValue;
          break;
        case OneOfValueOneofCase.DoubleValue:
          DoubleValue = other.DoubleValue;
          break;
        case OneOfValueOneofCase.CustomTypedValue:
          CustomTypedValue = other.CustomTypedValue.Clone();
          break;
      }

      _unknownFields = pb::UnknownFieldSet.Clone(other._unknownFields);
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public Value Clone() {
      return new Value(this);
    }

    /// <summary>Field number for the "stringValue" field.</summary>
    public const int StringValueFieldNumber = 1;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public string StringValue {
      get { return oneOfValueCase_ == OneOfValueOneofCase.StringValue ? (string) oneOfValue_ : ""; }
      set {
        oneOfValue_ = pb::ProtoPreconditions.CheckNotNull(value, "value");
        oneOfValueCase_ = OneOfValueOneofCase.StringValue;
      }
    }

    /// <summary>Field number for the "boolValue" field.</summary>
    public const int BoolValueFieldNumber = 2;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public bool BoolValue {
      get { return oneOfValueCase_ == OneOfValueOneofCase.BoolValue ? (bool) oneOfValue_ : false; }
      set {
        oneOfValue_ = value;
        oneOfValueCase_ = OneOfValueOneofCase.BoolValue;
      }
    }

    /// <summary>Field number for the "intValue" field.</summary>
    public const int IntValueFieldNumber = 3;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public int IntValue {
      get { return oneOfValueCase_ == OneOfValueOneofCase.IntValue ? (int) oneOfValue_ : 0; }
      set {
        oneOfValue_ = value;
        oneOfValueCase_ = OneOfValueOneofCase.IntValue;
      }
    }

    /// <summary>Field number for the "longValue" field.</summary>
    public const int LongValueFieldNumber = 4;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public long LongValue {
      get { return oneOfValueCase_ == OneOfValueOneofCase.LongValue ? (long) oneOfValue_ : 0L; }
      set {
        oneOfValue_ = value;
        oneOfValueCase_ = OneOfValueOneofCase.LongValue;
      }
    }

    /// <summary>Field number for the "doubleValue" field.</summary>
    public const int DoubleValueFieldNumber = 5;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public double DoubleValue {
      get { return oneOfValueCase_ == OneOfValueOneofCase.DoubleValue ? (double) oneOfValue_ : 0D; }
      set {
        oneOfValue_ = value;
        oneOfValueCase_ = OneOfValueOneofCase.DoubleValue;
      }
    }

    /// <summary>Field number for the "customTypedValue" field.</summary>
    public const int CustomTypedValueFieldNumber = 6;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public global::Ft.CustomTypedValue CustomTypedValue {
      get { return oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue ? (global::Ft.CustomTypedValue) oneOfValue_ : null; }
      set {
        oneOfValue_ = value;
        oneOfValueCase_ = value == null ? OneOfValueOneofCase.None : OneOfValueOneofCase.CustomTypedValue;
      }
    }

    private object oneOfValue_;
    /// <summary>Enum of possible cases for the "oneOfValue" oneof.</summary>
    public enum OneOfValueOneofCase {
      None = 0,
      StringValue = 1,
      BoolValue = 2,
      IntValue = 3,
      LongValue = 4,
      DoubleValue = 5,
      CustomTypedValue = 6,
    }
    private OneOfValueOneofCase oneOfValueCase_ = OneOfValueOneofCase.None;
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public OneOfValueOneofCase OneOfValueCase {
      get { return oneOfValueCase_; }
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public void ClearOneOfValue() {
      oneOfValueCase_ = OneOfValueOneofCase.None;
      oneOfValue_ = null;
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public override bool Equals(object other) {
      return Equals(other as Value);
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public bool Equals(Value other) {
      if (ReferenceEquals(other, null)) {
        return false;
      }
      if (ReferenceEquals(other, this)) {
        return true;
      }
      if (StringValue != other.StringValue) return false;
      if (BoolValue != other.BoolValue) return false;
      if (IntValue != other.IntValue) return false;
      if (LongValue != other.LongValue) return false;
      if (!pbc::ProtobufEqualityComparers.BitwiseDoubleEqualityComparer.Equals(DoubleValue, other.DoubleValue)) return false;
      if (!object.Equals(CustomTypedValue, other.CustomTypedValue)) return false;
      if (OneOfValueCase != other.OneOfValueCase) return false;
      return Equals(_unknownFields, other._unknownFields);
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public override int GetHashCode() {
      int hash = 1;
      if (oneOfValueCase_ == OneOfValueOneofCase.StringValue) hash ^= StringValue.GetHashCode();
      if (oneOfValueCase_ == OneOfValueOneofCase.BoolValue) hash ^= BoolValue.GetHashCode();
      if (oneOfValueCase_ == OneOfValueOneofCase.IntValue) hash ^= IntValue.GetHashCode();
      if (oneOfValueCase_ == OneOfValueOneofCase.LongValue) hash ^= LongValue.GetHashCode();
      if (oneOfValueCase_ == OneOfValueOneofCase.DoubleValue) hash ^= pbc::ProtobufEqualityComparers.BitwiseDoubleEqualityComparer.GetHashCode(DoubleValue);
      if (oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue) hash ^= CustomTypedValue.GetHashCode();
      hash ^= (int) oneOfValueCase_;
      if (_unknownFields != null) {
        hash ^= _unknownFields.GetHashCode();
      }
      return hash;
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public override string ToString() {
      return pb::JsonFormatter.ToDiagnosticString(this);
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public void WriteTo(pb::CodedOutputStream output) {
    #if !GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE
      output.WriteRawMessage(this);
    #else
      if (oneOfValueCase_ == OneOfValueOneofCase.StringValue) {
        output.WriteRawTag(10);
        output.WriteString(StringValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.BoolValue) {
        output.WriteRawTag(16);
        output.WriteBool(BoolValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.IntValue) {
        output.WriteRawTag(24);
        output.WriteInt32(IntValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.LongValue) {
        output.WriteRawTag(32);
        output.WriteInt64(LongValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.DoubleValue) {
        output.WriteRawTag(41);
        output.WriteDouble(DoubleValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue) {
        output.WriteRawTag(50);
        output.WriteMessage(CustomTypedValue);
      }
      if (_unknownFields != null) {
        _unknownFields.WriteTo(output);
      }
    #endif
    }

    #if !GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    void pb::IBufferMessage.InternalWriteTo(ref pb::WriteContext output) {
      if (oneOfValueCase_ == OneOfValueOneofCase.StringValue) {
        output.WriteRawTag(10);
        output.WriteString(StringValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.BoolValue) {
        output.WriteRawTag(16);
        output.WriteBool(BoolValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.IntValue) {
        output.WriteRawTag(24);
        output.WriteInt32(IntValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.LongValue) {
        output.WriteRawTag(32);
        output.WriteInt64(LongValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.DoubleValue) {
        output.WriteRawTag(41);
        output.WriteDouble(DoubleValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue) {
        output.WriteRawTag(50);
        output.WriteMessage(CustomTypedValue);
      }
      if (_unknownFields != null) {
        _unknownFields.WriteTo(ref output);
      }
    }
    #endif

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public int CalculateSize() {
      int size = 0;
      if (oneOfValueCase_ == OneOfValueOneofCase.StringValue) {
        size += 1 + pb::CodedOutputStream.ComputeStringSize(StringValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.BoolValue) {
        size += 1 + 1;
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.IntValue) {
        size += 1 + pb::CodedOutputStream.ComputeInt32Size(IntValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.LongValue) {
        size += 1 + pb::CodedOutputStream.ComputeInt64Size(LongValue);
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.DoubleValue) {
        size += 1 + 8;
      }
      if (oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue) {
        size += 1 + pb::CodedOutputStream.ComputeMessageSize(CustomTypedValue);
      }
      if (_unknownFields != null) {
        size += _unknownFields.CalculateSize();
      }
      return size;
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public void MergeFrom(Value other) {
      if (other == null) {
        return;
      }
      switch (other.OneOfValueCase) {
        case OneOfValueOneofCase.StringValue:
          StringValue = other.StringValue;
          break;
        case OneOfValueOneofCase.BoolValue:
          BoolValue = other.BoolValue;
          break;
        case OneOfValueOneofCase.IntValue:
          IntValue = other.IntValue;
          break;
        case OneOfValueOneofCase.LongValue:
          LongValue = other.LongValue;
          break;
        case OneOfValueOneofCase.DoubleValue:
          DoubleValue = other.DoubleValue;
          break;
        case OneOfValueOneofCase.CustomTypedValue:
          if (CustomTypedValue == null) {
            CustomTypedValue = new global::Ft.CustomTypedValue();
          }
          CustomTypedValue.MergeFrom(other.CustomTypedValue);
          break;
      }

      _unknownFields = pb::UnknownFieldSet.MergeFrom(_unknownFields, other._unknownFields);
    }

    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    public void MergeFrom(pb::CodedInputStream input) {
    #if !GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE
      input.ReadRawMessage(this);
    #else
      uint tag;
      while ((tag = input.ReadTag()) != 0) {
        switch(tag) {
          default:
            _unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, input);
            break;
          case 10: {
            StringValue = input.ReadString();
            break;
          }
          case 16: {
            BoolValue = input.ReadBool();
            break;
          }
          case 24: {
            IntValue = input.ReadInt32();
            break;
          }
          case 32: {
            LongValue = input.ReadInt64();
            break;
          }
          case 41: {
            DoubleValue = input.ReadDouble();
            break;
          }
          case 50: {
            global::Ft.CustomTypedValue subBuilder = new global::Ft.CustomTypedValue();
            if (oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue) {
              subBuilder.MergeFrom(CustomTypedValue);
            }
            input.ReadMessage(subBuilder);
            CustomTypedValue = subBuilder;
            break;
          }
        }
      }
    #endif
    }

    #if !GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE
    [global::System.Diagnostics.DebuggerNonUserCodeAttribute]
    [global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
    void pb::IBufferMessage.InternalMergeFrom(ref pb::ParseContext input) {
      uint tag;
      while ((tag = input.ReadTag()) != 0) {
        switch(tag) {
          default:
            _unknownFields = pb::UnknownFieldSet.MergeFieldFrom(_unknownFields, ref input);
            break;
          case 10: {
            StringValue = input.ReadString();
            break;
          }
          case 16: {
            BoolValue = input.ReadBool();
            break;
          }
          case 24: {
            IntValue = input.ReadInt32();
            break;
          }
          case 32: {
            LongValue = input.ReadInt64();
            break;
          }
          case 41: {
            DoubleValue = input.ReadDouble();
            break;
          }
          case 50: {
            global::Ft.CustomTypedValue subBuilder = new global::Ft.CustomTypedValue();
            if (oneOfValueCase_ == OneOfValueOneofCase.CustomTypedValue) {
              subBuilder.MergeFrom(CustomTypedValue);
            }
            input.ReadMessage(subBuilder);
            CustomTypedValue = subBuilder;
            break;
          }
        }
      }
    }
    #endif

  }

[replaysMike]

Very strange it would be that slow, sounds like something is wrong with recursion detection on the object - or something really wrong is happening. It's all based on reflection but should not be that slow at all.

I'll see if I can dig into this in a day or two - I'm currently away from the computer for a few days