Delphi Labs: DataSnap XE - Callbacks

By: Pawel Glowacki

Abstract: "Delphi Labs" DataSnap XE "Callbacks" demo shows the most simple use of callbacks. Both client and server are Delphi VCL Forms applications. This tutorial covers broadcasting to a channel and notifying a specific callback.

    Introduction

The objective of this tutorial is to create the simplest possible DataSnap Delphi client and server applications that use callbacks for communication.

In this lab exercise, we are going to use Delphi XE to build a simple callbacks demo system consisting of server and client applications. The server application will serve as a communication hub for multiple client applications running in the network. It is a more realistic scenario as compared to sending notifications directly from server application user interface to clients. In most scenarios, a server application will not have any user interface, so callbacks are a great mechanism for clients to communicate with each other.

    Message Exchange Patterns

The most common message exchange pattern in client/server applications is “request-response”. One application (“a client”) is sending a message (“a request) to another application running in the network (“a server”) and the server sends back a message (“a response").

Hide image
Click to see full-sized image

In many real world applications, it would also be useful to have the opposite situation, where it is a server application that sends a message (“a notification”) to a client application. A server application may want to inform a client that something interesting has happened on the server. This is called “a callback” – a situation when server “calls back” the client.

expand view>>

Hide image
Click to see full-sized image

Imagine a chat application where multiple client applications connected to the server can communicate with each other. One client sends a message to the server and then the server forwards this message to one or more connected client applications.

expand view>>

Hide image
Click to see full-sized image

The possibility for the server to asynchronously send a notification to one or more clients is very useful in many scenarios.

expand view>>

    DataSnap Callbacks and Channels

In order to use callbacks in DataSnap applications, you need to define a custom callback class that is inherited from the abstract “TDBXCallback” class and override one of its virtual, abstract “Execute” methods which are called by the server and executed on the client. The “TDBXCallback” class is defined in the “DBXJSON” unit as follows (some members striped out for readability):

unit DBXJSON;

interface

// …

type
  TDBXCallback = class abstract
  public
    function Execute(const Arg: TJSONValue): TJSONValue; overload; virtual; abstract;
    function Execute(Arg: TObject): TObject; overload; virtual; abstract;
    // …
  end;

In the previous version of DataSnap that came with RAD Studio 2010 it was only possible to use so-called “lightweight” callbacks. A callback instance was passed to a long running server method as a parameter from a client, so the server could call its “Execute” method within the duration of a method call – for example to notify the client about the progress of a long running operation.

In RAD Studio XE, the latest version available, so called “heavyweight” callbacks have been introduced. They can be used throughout the whole lifetime of a client application and not only during a server method call. This opens a lot of new possibilities for building different types of applications. In the remaining part of this tutorial we are going to focus on “heavyweight” callbacks and for simplicity we are going to refer to them as just “callbacks”.

In DataSnap architecture callbacks are associated with “channels”. In general there could be multiple client applications connected to the server and each of these clients can contain zero or more callbacks. The server can “broadcast” to the channel, so all callbacks on every client that are registered with a specific channel are going to receive this notification. It is also possible to notify a specific callback using its unique identifier used during registering the callback with the server. In this way it is possible to achieve peer-to-peer communication model.

We are going to try both approaches: broadcasting to a channel and notifying a specific callback.

The server application calls “Execute” method on the client callback asynchronously. This is a very important point to realize. Every Delphi VCL Forms application has its main thread of execution and in case of multithreaded applications any calls from other threads that manipulates graphical user interface of the applications need to be synchronized. This is exactly the situation with using callbacks. The callback “Execute” method is called on a different thread then the main thread of the VCL application. There are different ways of synchronizing calls, but probably the easiest option is to use “TThread.Queue” class method, which asynchronously executes a block of code within the main thread.

    Implement the Callback Server

Our server application is going to be super simple. The callback functionality is built into the “DSServer” component, which is the central point of every DataSnap server application. In this demo we do not even need to create any server methods, because we are only going to communicate between client applications using callbacks.

The first step is to create a new DataSnap server application using “DataSnap Server” wizard.

Select “File -> New -> Other” and from the “New Items” dialog double-click on the “DataSnap Server” icon in the “Delphi Projects -> DataSnap Server” category.

Hide image
Click to see full-sized image

In the first tab keep the default DataSnap “Project type” which is “VCL Forms Application”.

expand view>>

Hide image
Click to see full-sized image

On the second tab we keep “TCP/IP” as the communication protocol and we can uncheck the option for generating “server methods class”, because it is not needed for this simple callbacks demo. If you leave the default option to generate server methods, it is not a problem. We are just not going to use them.

expand view>>

Hide image
Click to see full-sized image

On the third screen we keep the default value “211” for the TCP/IP Port. It is always a good idea to click on the “Test Port” to make sure that it is available.

expand view>>

Because we have unchecked the option to generate a server class earlier in the wizard, we are not presented with the screen to select a base class for our server method class.

Click on “Finish” and the wizard should create a new project with just two units: main form and server container. There is no server methods unit this time.

Click on “File -> Save All”.

Create a new directory for all files in this lab – for example “C:\DataSnapLabs\SimpleCallbacks”.

Save main application form as “FormServerUnit” and keep the default name for the server container unit – typically “ServerContainerUnit1”.

Save project as “SimpleCallbacksServer”.

Select the main form in the Object Inspector and change its “Name” property to “FormServer” and its “Caption” property to “Delphi Labs: DataSnap XE - Simple Callbacks – Server”.

Hide image
ServerForm

Open the server container unit and verify that there are only two components there: “DSServer1” and “DSTCPServerTransport1” components.

Hide image
ServerContainer

That’s it. Our server is ready and we do not need to implement anything special on the server, because the support for callbacks is built into the “DSServer1” component. We also have a transport component so that external clients can communicate with the “DSServer1” instance in the server.

“Save All”, “Run without Debugging” and minimize the server application. It should be running for the rest of this tutorial.

    Create the Client Application

Now it is the time to create a client. Just right click on the “Project Group” node in the “Project Manager” window and select “Add New Project”.

Hide image

From the “New Items” dialog select “VCL Forms Application” from “Delphi Projects” category.

Hide image

Click “OK”. A new project will be added to the existing project group.

Click on “File -> Save All”.

Locate the folder where the server project has been saved and save there the main form unit of the client application as “FormClientUnit”, the new project as “SimpleCallbacksClient” and the project group as “SimpleCallbacksGrp”.

    Implementing a callback

The next step is to define a new callback class derived from “TDBXCallback” and implement its “Execute” method. This method will be called asynchronously by the server to notify the client.

Add “DBXJSON” unit to the “uses” clause of “FormClientUnit”, because this is where “TDBXCallback” class is defined.

Define “TMyCallback” class and override its virtual abstract “Execute” method. There are two variants of the “Execute” method you could override. One that takes and returns a “TObject” and the second that takes and returns “TJSONValue”. I’m going to use the second option, because at the end both methods use JSON as the underlying format for sending messages.

At this stage the client unit source code looks like this:

unit FormClientUnit;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, DBXJSON;

type
  TMyCallback = class(TDBXCallback)
  public
    function Execute(const Arg: TJSONValue): TJSONValue; override;
  end;

  TFormClient = class(TForm)
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  FormClient: TFormClient;

implementation

{$R *.dfm}

{ TMyCallback }

function TMyCallback.Execute(const Arg: TJSONValue): TJSONValue;
begin
  // ...
end;

end.

So what should happen when our callback’s “Execute” method is called? This is really up to the programmer and depends on the application logic. To keep this example simple, we are going to add a memo component to the client form and when the callback “Execute” method is called we are going to add a text line to the memo with the contents of the “Arg” parameter converted to a text string.

Let’s define a public method on the form class called “LogMsg” that will take a string parameter with a message to display in the memo. We are also going to add a timestamp.

Drop “TMemo” component on the client form. Change its name in the Object Inspector to “MemoLog”.

Add to “TFormClient” class a public procedure “LogMsg(const s: string)” and implement it in the following way:

procedure TFormClient.LogMsg(const s: string);
begin
  MemoLog.Lines.Add(DateTimeToStr(Now) + ': ' + s);
end;

Now is the tricky part. We need to make a thread-safe call to the “TFormClient.LogMsg” procedure from our “TMyCallback.Execute” method.

Let’s define the thread-safe version of our “LogMsg” method, so it could be called from a different thread.

procedure TFormClient.QueueLogMsg(const s: string);
begin
  TThread.Queue(nil,
    procedure
    begin
      LogMsg(s)
    end
  );
end;

The syntax for using anonymous methods may seem to be exotic at first, but think about it like treating code as data. You just pass a block code as the second parameter to “TThread.Queue” method. This method is a “class” method of the “TThread” class, so we do not need to instantiate “TThread” object in order to be able to call it.

Now we can call the thread-safe version of our “LogMsg” method directly from the “TMyCallback.Execute” method.

function TMyCallback.Execute(const Arg: TJSONValue): TJSONValue;
begin
  FormClient.QueueLogMsg(Arg.ToString);
  Result := TJSONTrue.Create;
end;

We can return anything from our “Execute” method, as long as we do return something, so we just return JSON “true” value.

Now we need to register our callback with the server, so it is informed about what to “call back”.

There is a special class designed for managing client callbacks called “TDSClientCallbackChannelManager” and it is defined in the “DSHTTPCommon” unit.

Drop a “TDSClientCallbackChannelManager” component on the form and set its properties in the Object Inspector.

We need to select a name for a channel on the server that we want to associate our callback with. Let’s call our channel “DelphiLabsChannel”.

We also need to specify “CommunicationProtocol”, “DSHostname” and “DSPort” properties.

Hide image
CallbackChannelMgrProperties

The next thing we are going to do is to clear the “ManagerId” property, because we are going to generate this value at runtime.

This is a very important thing to do. We want every client application instance to be treated by the server differently. The “ManagerId” value is used at the server to identify clients, so this value has to be different for every client instance.

We are going to use “TDSClientCallbackChannelManager.RegisterCallback” method to register our callback instance with the server. This method takes two parameters: the name of the callback the uniquely identifies it on the server and the reference to the callback instance, in our case this will be “FMyCallback”.

If you look into the constructor of the “TDSClientCallbackChannelManager” class you will see that the value for “ManagerId” is generated by a call to “TDSTunnelSession.GenerateSessionId” method that returns a random string made of three numbers. We are going to use this functionality to generate a unique name for our callback instance.

Add “FCallbackName: string” private field to the form class and add code to initialize it in the form’s “OnCreate” event. You will also need to add “DSService” unit to the “uses” clause, because this is where the “TDSTunnelSession” class is defined.

We also need to add code to initialize “DSClientCallbackChannelManager1.ManagerId” property.

uses DSService; // for “TDSTunnelSession”

// … 

procedure TFormClient.FormCreate(Sender: TObject);
begin
  DSClientCallbackChannelManager1.ManagerId := 
    TDSTunnelSession.GenerateSessionId;

  FMyCallbackName := 
    TDSTunnelSession.GenerateSessionId;

  DSClientCallbackChannelManager1.RegisterCallback(
    FMyCallbackName,
    TMyCallback.Create
  );
end;

The callback reference that we pass to the “RegisterCallback” method is owned by the “DSClientCallbackChannelManager1”, so we do not need to keep this reference.

At this point we are ready to receive callbacks. The next step is to implement a functionality to broadcast to a channel. All callbacks registered with a specified channel are going to be notified by the server.

    Broadcasting to the Channel

Note that we could now create a completely different client application for broadcasting to the channel, and our client application, as it is implemented now, would be able to receive the notifications.

To keep things simple, we are going to add broadcasting to channel functionality to our demo client application, and later we are going to run two instances of the same client application to see if we can send messages from one client to another.

The first thing to do on the client is to add a “TSQLConnection” component to the form in order to be able to connect to the server. Probably the easiest way to do it is with “IDE Insight”. Just press F6 and start typing “TSQLConnection” to search for it and then select to add to the form.

Set the “Driver” property of “SQLConnection1” component on the form to “DataSnap”.

Set “LoginPrompt” property to “False”.

Set the “Connected” property to “True” to verify that the client is able to connect to the server.

In a typical scenario, at this stage we would need to generate DataSnap client proxy code in order to be able to call server methods. In this case, however, this step is not necessary, since there are no custom server methods on the server! The Delphi DataSnap proxy generator uses “TDSAdminClient” class as a base class for client proxy classes. This class already contains quite a lot of functionality that can be used on its own, including broadcasting to channels and notifying callbacks. We are going to use “TDSAdminClient” class directly as the way to interact with the server.

We need to extend our client application user interface a bit to support broadcasting to a channel.

Add “TButton” component to the form. Set its “Name” property to “ButtonBroadcast” and its “Caption” property to “Broadcast to Channel”.

Add a “TEdit” component. Set its “Name” property to “EditMsg” and optionally enter some default message into it.

You can also add a label next to the message edit, to indicate that this is the place to enter messages.

Double-click on the button and add the following code to be able to broadcast to messages to a channel. Note that we could pass arbitrary complex data encoded as JSON, so it could be something more complex than just a string.

uses DSProxy; // <- for “TDSAdminClient”

// …

procedure TFormClient.ButtonBroadcastClick(Sender: TObject);
var AClient: TDSAdminClient;
begin
  AClient := TDSAdminClient.Create(SQLConnection1.DBXConnection);
  try
    AClient.BroadcastToChannel(
      DSClientCallbackChannelManager1.ChannelName,
      TJSONString.Create(EditMsg.Text)
    );
  finally
    AClient.Free;
  end;
end;

Now if you run the client application and press on the broadcast button you should see your message entered into the edit received by the callback and displayed in the memo.

Run the second instance of the client application and you should be able to see that messages sent from one application are received by all applications!

Hide image
Click to see full-sized image

That is cool! We can now broadcast arbitrary data that can be encoded in JSON to multiple applications running in the network.

expand view>>

What about pure peer-to-peer communication? Maybe I do not want to send message to all callbacks in the channel?

It would be much better if a given client could send message to one and only one callback instance on a different client.

This is also possible, but and we need to extend our client application to support notifying specific callbacks.

    Notifying Callbacks

Stop both clients, but keep the server running.

The “TDSAdminClient” class also contains “NotifyCallback” method that could be used to achieve peer-to-peer communication model. This method has the following signature:

function TDSAdminClient.NotifyCallback(ChannelName: string; ClientId: string; CallbackId: string; Msg: TJSONValue; out Response: TJSONValue): Boolean;

The “ChannelName” parameter specifies the name of the communication channel the destination client callback is associated with. “ClientId” and “CallbackId” are values that were passed to “RegisterCallback” method of the “DSClientCallbackChannelManager1” at the destination client instance. They were both generated randomly. “Msg” is the JSON value that contains information that we want to send to the destination callback and “Response” is an “out” parameter and contains JSON value with encoded response.

There is also “TDSAdminClient.NotifyObject” that takes similar parameters, but instead of using “TJSONValue” for input and output parameters, it is using a TObject-descendant that is automatically serialized and deserialized from its JSON representation.

The process of notifying individual callbacks is going to be a little bit manual that will involve copying and pasting “ClientId” and “CallbackId” values from one running instance to another.

Let’s add to our client application four additional “TEdit” components, four “TLabel” components and a “TButton”.

Change “Caption” property of the button to “Notify Callback” and rename edits to: “EditLocalClientId”, “EditLocalCallbackId”, “EditDestinationClientId”, “EditDestinationCallbacksId”.

In the “OnCreate” event of the client form add code to initialize edits:

  
EditLocalClientId.Text := DSClientCallbackChannelManager1.ManagerId;
EditLocalCallbackId.Text := FMyCallbackName;
EditDestinationClientId.Text := '';
EditDestinationCallbackId.Text := '';

Double-click on the “Notify Callback” button and enter the following code to notify remote callback:

procedure TFormClient.ButtonNotifyClick(Sender: TObject);
var AClient: TDSAdminClient; aResponse: TJSONValue;
begin
  AClient := TDSAdminClient.Create(SQLConnection1.DBXConnection);
  try
    AClient.NotifyCallback(
      DSClientCallbackChannelManager1.ChannelName,
      EditDestinationClientId.Text,
      EditDestinationCallbackId.Text,
      TJSONString.Create(EditMsg.Text),
      aResponse
    );
  finally
    AClient.Free;
  end;
end;

Now start two or more client application instances and copy “ClientId” and “CallbackId” from a client that you would like to receive notifications to “destination” edits of the client you want to send notification.

Hide image
Click to see full-sized image

That’s it! We have implemented peer-to-peer communication between Delphi DataSnap client applications!

expand view>>

    The Bigger Picture

The idea behind this Delphi Labs demo was to make it as simple as possible.

It is also possible to use callbacks with the HTTP protocol in addition to TCP/IP. Similarly in this demo we have used DataSnap DBX architecture, but the callbacks are also available with DataSnap REST.

RAD Studio XE comes with a very interesting demo project that demonstrates all these possibilities.

You can open this demo directly from inside the IDE using new Subversion integration.

Select “File -> Open from Version Control…” and enter the following URL in the “URL or Repository…” text box:

https://radstudiodemos.svn.sourceforge.net/svnroot/radstudiodemos/branches/RadStudio_XE/Delphi/DataSnap/CallbackChannels

In the “Destination” text box enter a folder that you want to download the demo to. In my case I have created a local “C:\DataSnapLabs\CallbackChannelsDemo” folder.

Hide image
checkout

Just click on OK and be patient. You will see initially empty window with “Updating…” title. After a while it will show the names of all files checked out from the “radstudiodemos” public repository on SourceForge.

Hide image
select project to open

There are three projects there. “ChannelsServerProject” is the main server application. “DBXClientChannels” and “RESTClientChannels” are two client applications. One based on DataSnap DBX architecture and one based on the new DataSnap REST architecture introduced in RAD Studio XE.

Keep the server project selected and click OK to open it in the IDE.

Hide image
Click to see full-sized image

Click “OK” to close the “Updating” window. At this stage only the server project is opened in the IDE.

expand view>>

Now we need to add both client projects to a project group, so we have all three demo projects available inside the IDE.

Right click on the “Project Group” node in the “Project Manager” window, select “Add Existing Project” and choose “DBXClientChannels” project.

Right click again on the “Project Group”, select “Add Existing Project” and this time choose “RESTClientChannels” project.

Select “File -> Save All” or just click on the “Save All” icon.

Give the project group a name. I have chosen for “CallbackChannelsDemo”.

At this stage my “Project Manager” looks like this:

Hide image
CallbackChannelsGrp

I will leave you here. There is plenty to explore in this demo…

    Summary

In this Delphi Lab we have used Delphi XE for building a system consisting of server and client native Win32 applications communicating with each other using TCP/IP protocol and using callbacks.

Callbacks represent a very useful alternative to a traditional request/response message exchange pattern in distributed applications.

With callbacks the server application is able to send asynchronous notifications to one or more registered callback instances inside connected client applications.

The full source code for this article is available from Embarcadero Code Central http://cc.embarcadero.com/item/28288

The video version of steps described in this article can be found on YouTube. There are three parts of the video demonstration:

More information about Delphi can be found on the Delphi home page http://www.embarcadero.com/products/delphi

Server Response from: ETNASC03