Anonymous methods for TField

By: John Kaster

Abstract: Discussion of a new TField descendant called TValueField, that supports anonymous methods for getting and setting data values

    Introduction

As I've mentioned in some previous articles (DataSnap and ISAPI, and DataSnap user connections) and on our discussion forums, we are working on a new implementation of the QualityCentral server, which is being built with DataSnap/REST.

One of the back-end improvements we've made is switching from the 1998-era keyword indexing system I wrote in Delphi (originally for CodeCentral) to the Lucene-based search engine we now have on the majority of the EDN websites.

The first step in supporting the new indexing engine was creating the native Delphi TSearchIndex component to wrap up calls to our Lucene web service. As we migrate our back-end systems to DataSnap/REST, we'll be relying on this component for interaction with Lucene.

We've created our own list of custom search index fields that may or may not apply to a given web application. For example, the EDN article site supports content in multiple human languages, while the default language for CodeCentral and QualityCentral is English. Therefore, when creating a generic search index component, I wanted to support the concept of default values for fields.

Once I came up with the idea for how to do it, some of the new features in Delphi made the implementation quite simple and easy to implement and use.

    New Delphi features

Over the last few releases of Delphi, the "TField.As*" methods have been extended to support automated conversion from one type to another, making TField objects even more convenient to use in business logic code. It's easier to get data for any field assigned or retrieved from a variety of data types, where a conversion from one type to another can be supported by the object.

The Delphi compiler has also seen significant improvements, particularly in the area of generics and anonymous methods. The existence of anonymous methods provides a surprisingly simple and extremely convenient new way to have custom logic invoked when assigning or retrieving a value from a simple TField object.

There are many ways to apply anonymous methods in a data-binding architecture, and I'm sure you'll see some functionality in future Delphi releases that provides functionality similar to what will be discussed here.

    TValueField building blocks

The source code for TValueField is available on SourceForge in ValField.pas, if you'd like to have the complete source available for the ensuing discussion of its features.

The conventional way to add custom logic when getting or setting the value of a field is using the OnCalcFields event. This is still a handy option when using the Delphi designer and components you drag and drop onto a data or web module. For our server logic, we rely on DbxUtils, which allows us to do all our database operations with one or two lines of code, so adding OnCalcField events to dynamically created datasets was not a reasonable solution.

Creating a new TField descendant that would allow us to assign custom value calculation logic was a very convenient way for us to solve the problem, once anonymous methods were available. We could have created new GetValue and SetValue descendant events, but as I think you will see, using anonymous methods instead makes the solution so much more flexible.

    Anonymous method signatures

Here are the anonymous method signatures used in TValueField.

{$REGION 'Anonymous function for getting the value of a TValueField object'}
/// <summary>Anonymous function for getting the value of a <c>TValueField</c>
/// object</summary>
/// <param name="Sender">The instance of the object making the call</param>
{$ENDREGION}
TGetValueField = reference to function(Sender: TObject): Variant;

{$REGION 'Anonymous method for setting the value of a TValueField object'}
/// <summary>Anonymous method for setting the value of a <c>TValueField</c>
/// object</summary>
/// <param name="Sender">The instance of the object making the call</param>
/// <param name="Value">Value to assign</param>
{$ENDREGION}
TSetValueField = reference to procedure(Sender: TObject; Value: Variant);

Note: I use the excellent RAD Studio IDE plug-in called Documentation Insight by DevJet to create the source code documentation for EDN.

When these anonymous functions are called from TValueField methods, Sender is self. From this self-referencing call, the dataset can be retrieved, so other fields can also be accessed from the anonymous method.

    New or overridden properties

Only two new properties are required, but the DataSet property must be reintroduced because there is no GetDataSet method in TField that can be overridden. TField directly accesses the private FDataSet field as the getter for the DataSet property.

/// <summary>Reintroduces the DataSet property so GetDataSet is used to
/// access it</summary>
property DataSet read GetDataSet write SetDataSet stored False;

/// <summary>Property accessor for the anonymous getter method for the
/// value</summary>
property GetValueFunc: TGetValueField read FGetValueFunc write FGetValueFunc;

/// <summary>Property accessor for the anonymous setter method</summary>
property SetValueProc: TSetValueField read FSetValueProc write FSetValueProc;

    Default field values

For our search indexing component, this is what I wanted to support:

  1. Pass in a dataset of one or more records to a general-purpose routine for indexing with the search engine
  2. Create persistent field references before the iteration loop, so the retrieval of field values while iterating through the dataset was optimal
  3. Use TFields to get the value of all fields that needed to be indexed, without having to resort to field-specific if … then … else blocks inside the dataset iteration loop
  4. Provide a default value for any field that happened to be missing from the dataset passed in to the indexing method

There are several overloaded DefaultField class functions for TValueField:

class function DefaultField(ADataSet: TDataSet; AName: string; AOwner:
  TComponent = nil): TField; overload;

class function DefaultField(ADataSet: TDataSet; AName: string;
  ADefault: Variant; AOwner: TComponent = nil): TField; overload;

class function DefaultField(ADataSet: TDataSet; AName: string;
  AGetValue: TGetValueField; ASetValue: TSetValueField = nil;
  AOwner: TComponent = nil): TField; overload;

class function DefaultField(ADataSet: TDataSet; AName: string;
  ADefault: Variant; AGetValue: TGetValueField;
  ASetValue: TSetValueField = nil;
  AOwner: TComponent = nil): TField; overload;

Here's the implementation of the core DefaultField function the other overloads all end up calling:

class function TValueField.DefaultField(ADataSet: TDataSet; AName: string;
  ADefault: Variant; AGetValue: TGetValueField; ASetValue: TSetValueField;
  AOwner: TComponent): TField;
begin
  if Length(Trim(AName)) > 0 then
    Result := ADataSet.FindField(AName)
  else
    Result := nil;
  if not Assigned(Result) then
  begin
    Result := TValueField.Create(AOwner);
    with Result as TValueField do
    begin
      Name := AName;
      Value := ADefault;
      GetValueFunc := AGetValue;
      SetValueProc := ASetValue;
      DataSet := ADataSet;
    end;
  end;
end;

As you can see from the code above, DefaultField returns the TField object with the requested name if it is found in the provided dataset. If the field name is not found in the dataset, a TValueField is returned, with its Owner, DataSet, Name, Value, getter and setter anonymous methods all assigned. If Owner is nil, the client must take care of freeing the returned object. Otherwise, the TValueField will be freed when its owner is freed.

    Default field values in dataset iteration

The general-purpose routine in TSearchIndex uses the TValueField.DefaultField overloads to set default values for several of the optional fields. While this routine has a lot of code, that's mainly because of all the field assignments that need to be made. It's quite simple to set up and tear down the field references.

function TSearchService.InternalIndexContent(AAppID: string; ADataSet: TDataSet;
  AIdField: string; AFirstNameField: string; ALastNameField: string;
  ATitleField : string; ASummaryField: string; ABodyField: string;
  APubDateField: string; ALangCodeField: string; ACommentsField: string;
  AProductsField: string; AVersionsField: string; ATagsField: string;
  ACategoriesField: string; AExtraDataField: string; AContentTypeField: string;
  AWorkaroundField: string; AArchiveField: string;
  AIndexAsHtml: boolean = false; ACommentFunc: TGetValueField = nil;
  AArchiveFunc: TGetValueField = nil; AIndexIndividualFiles: boolean = false
  ): TSearchResults;
var
  res: TSearchResult;
  current: TSearchResults;
  ContentId,
  FirstName,
  LastName,
  Title,
  Summary,
  Body,
  PubDate,
  LangCode,
  Comments,
  Products,
  Versions,
  Tags,
  Categories,
  ExtraData,
  ContentType,
  Workaround,
  FldArchive: TField;
  Archive: TArchive;
  Errors,
  id: string;

  procedure FreeValueField(var AField: TField);
  begin
    if AField is TValueField then
      FreeAndNil(AField);
  end;

  function Required(const ADesc, AFieldName: string): TField;
  begin
    Result := ADataSet.FindField(AFieldName);
    if not Assigned(Result) then
      Errors := Errors + Format(StrRequiredFieldMissing,
        [ADesc, AFieldName]);
  end;

begin
  id := '';
  if StrIsEmpty(AAppId) then
    Errors := StrAppIdRequired
  else
    Errors := '';
  try
    try
      // Fields that are required
      ContentId := Required(StrContentID, AIdField);
      Title := Required(StrTitle, ATitleField);
      Body :=  Required(StrBody, ABodyField);
      if StrIsFull(Errors) then
        raise Exception.CreateFmt(StrInternalIndexContentErrors,
          [Errors]);

      // Fields that can be defaulted
      FirstName := TValueField.DefaultField(ADataSet, AFirstNameField);
      LastName := TValueField.DefaultField(ADataSet, ALastNameField);
      Summary := TValueField.DefaultField(ADataSet, ASummaryField);
      PubDate := TValueField.DefaultField(ADataSet, APubDateField, MinDateTime);
      LangCode := TValueField.DefaultField(ADataSet, ALangCodeField, 'en');
      Comments := TValueField.DefaultField(ADataSet, ACommentsField,
        ACommentFunc);
      Products := TValueField.DefaultField(ADataSet, AProductsField);
      Versions := TValueField.DefaultField(ADataSet, AVersionsField);
      Tags :=  TValueField.DefaultField(ADataSet, ATagsField);
      Categories := TValueField.DefaultField(ADataSet, ACategoriesField);
      ExtraData := TValueField.DefaultField(ADataSet, AExtraDataField);
      ContentType := TValueField.DefaultField(ADataSet, AContentTypeField);
      Workaround := TValueField.DefaultField(ADataSet, AWorkaroundField);
      FldArchive := TValueField.DefaultField(ADataSet, AArchiveField,
        AArchiveFunc);
      while not ADataSet.EOF do
      begin
        id := ContentId.Value;
        Archive := OpenArchive(FldArchive);
        try
          Current := InternalIndexContent(AAppID,
            id,
            FirstName.Value,
            LastName.Value,
            Title.Value,
            Summary.Value,
            Body.Value,
            PubDate.Value,
            LangCode.Value,
            Comments.Value,
            Products.Value,
            Versions.Value,
            Tags.Value,
            Categories.Value,
            ExtraData.Value,
            ContentType.Value,
            WorkAround.Value,
            Archive,
            AIndexAsHtml,
            AIndexIndividualFiles);
          for res in Current do
            if not res.Result then
              AddResult(Result, res); // only accumulate errors
        finally
          FreeAndNil(Archive);
        end;
        ADataSet.Next;
      end;
    finally
      FreeValueField(ContentId);
      FreeValueField(FirstName);
      FreeValueField(LastName);
      FreeValueField(Title);
      FreeValueField(Summary);
      FreeValueField(Body);
      FreeValueField(PubDate);
      FreeValueField(LangCode);
      FreeValueField(Comments);
      FreeValueField(Products);
      FreeValueField(Versions);
      FreeValueField(Tags);
      FreeValueField(Categories);
      FreeValueField(ExtraData);
      FreeValueField(ContentType);
      FreeValueField(Workaround);
    end
  except
    on E: Exception do
    begin
      raise Exception.CreateFmt(StrIdError, [id, E.Message]);
    end;
  end;
end;

Note: StrIsFull and StrIsEmpty are utility functions that indicate whether the string has only whitespace or not.

The TQCSearch class that descends from TSearchIndex uses an anonymous method to collect all comments on a given report and index them as one value by calling a function in the anonymous method. While the internal indexing routine above has a fair bit of tedious code in it, wrapping it up for any dataset to be indexed can be done in a single statement:

function TQCSearch.Index(const ADataSet: TDataSet): TSearchResults;
begin
  IndexContent(FAppId, ADataSet, SDefectNo, SFirstName, SLastName,
    SShortDescription, SDescription, SSteps, SDateReported, SLangCode,
    SComments, SProject, SVersion, '', SDataType, '', '', SWorkaround, SArchive,
      function(Sender: TObject): variant
      var
        f: TValueField;
      begin
        f := Sender as TValueField;
        Result := QCCommentText(f.DataSet.FieldByName(SDefectNo).AsInteger);
      end
      );
end;

Just to be complete, here's the code for QCCommentText, which uses DbxUtils functions to retrieve all comments for a QC report.

function TQCSearch.QCCommentText(AID: integer): string;
var
  FldFirstName,
  FldLastName,
  FldTitle: TField;
  ds: TDataSet;
begin
  Result := '';
  ds := GetDataSet(Connection, SqlQCComments, RIDParams(AID));
  try
    FldFirstName := ds.FieldByName(SFirstName);
    FldLastName := ds.FieldByName(SLastName);
    FldTitle := ds.FieldByName(SComment);
    while not ds.Eof do
    begin
      Result := Result
        + FullName(FldFirstname.asWideString, fldLastName.AsWideString) + #13
        + FldTitle.AsWideString + #13;
      ds.Next;
    end;
  finally
    FreeAndNil(ds);
  end;
end;

    Conclusion

Although the use case described here is for a specific need, there are many ways to use anonymous methods for a "value" getter/setter on a DataSet (or no DataSet at all). There are certainly many ways to enhance what is currently provided, so you should check on Sourceforge occasionally for updates – or contribute your own!

As I mentioned previously, you may see similar support for anonymous methods and other data-binding enhancements in future releases of Delphi. For now, feel free to use TValueField. Please let us know how you are using it, and how you'd like the data-binding architecture of Delphi to be enhanced in the future.

Server Response from: ETNASC04