Building, debugging, and deploying a DataSnap/REST ISAPI dll

By: John Kaster

Abstract: Steps and suggestions for deploying a DataSnap/REST server as an ISAPI DLL

    Introduction

RAD Studio XE includes some new features for DataSnap that I have eagerly anticipated and been requesting for years. The XE release provides an implementation that is already proving to be very useful and productive. For EDN, we have already started creating new DataSnap/REST based services for our "next generation" of services because of the strength of these new features.

This article discusses the steps taken to build a DataSnap/REST ISAPI dll that is deployed to IIS 6 on Windows Server 2003.

Although the DataSnap wizard can easily make stand-alone DataSnap servers, ISAPI is a valuable option to support. A DataSnap ISAPI server:

  • Can share ports with an existing web server
  • Can be managed with other IIS applications
  • Gets SSL support "for free" with the built-in SSL support of IIS

    Painless proxies

The new DataSnap client proxy generation system included in XE has generators for Delphi, C++, Delphi Prism, PHP and JavaScript. The client proxy generator addresses issues I've long had with consuming any of the REST-based services available on the web. The proxy generator takes the pain of communicating with and consuming REST services away completely by translating the DataSnap methods and even complex types into "native" implementations for the target client language.

    Develop and debug the server

In my opinion, the easiest way to develop and debug a DataSnap server is by using the DataSnap server wizard to create a stand-alone VCL forms application.

    Create the DataSnap/REST VCL form server

From File | New | Other select Delphi Projects, then the DataSnap Server node. Select DataSnap REST Application and click OK.

Hide image
Click to see full-sized image

Note (highlighted with the red box in the screenshot above) the description available for each item in the object gallery when you either select or hover over it with the mouse.

Click OK to start the DataSnap REST application wizard.

Hide image
Click to see full-sized image

The stand-alone VCL application option is the most convenient option for interactively testing and debugging your application, because you can start and stop the DataSnap server at any time, and open a browser page directly from the main form to test your methods with the Javascript-based server explorer. (More on this later.) This is the option we'll use for now.

If you want to do automated testing of your DataSnap server, the stand-alone console application is a good option, because you can start and stop the application easily from an automation system.

When you're ready to deploy your DataSnap server to a production environment running IIS, you can use the ISAPI dynamic link library option. You could also use this if you're familiar with debugging ISAPI dlls or have utilities for debugging ISAPI dlls. We'll use this option later, when we're getting ready to deploy to our production server.

Click Next to go to the next step.

Hide image
Click to see full-sized image

The wizard defaults to using port 8080. You can click Test Port to verify the port is available, or play the DataSnap version of "I feel lucky!" by clicking the Find Open Port button and a port will be assigned to your application.

Click Next to go to the next step.

Hide image
Click to see full-sized image

By default, Server Methods Class and Sample Methods are selected. Leave them selected. The sample methods are trivial to remove later, and are very convenient for testing the basic invocation support of your DataSnap server. Unless you want to implement your authentication methods right away, leave those options deselected for now. Authentication and Authorization is very easy to add later on. You can read Mat Delong's blog post for more information on Auth/Auth.

Click Next to go to the next step.

Hide image
Click to see full-sized image

Each option in the radio group shown in the above screen shot provides additive features to the option listed above it.

The TComponent ancestor type is the lightest-weight ancestor for your DataSnap server. If you don't want a design surface for non-visual components (many people like using the IDE design-time support for database components, etc.) you can select this option.

For this server, I'm selecting TDataModule as my ancestor, because I have a design surface available should I choose to use it.

If you want to provide compatibility with the calling interface for an existing DataSnap or MIDAS server, you can select TDSServerModule, which will also implement IAppServer in addition to the same design surface you'd have with TDataModule.

Click Next to go to the next step.

Hide image
Click to see full-sized image

Choose the project location you want, and click Finish to complete the wizard. You should see something similar to the following in the IDE:

Hide image
Click to see full-sized image

    Unit1

Unit1 contains your server form and the events for starting and stopping your DataSnap server, and opening the browser to the DataSnap server home page.

Note: The Open Browser button will automatically start the DataSnap server if it's not started already.

Hide image
Click to see full-sized image

    Unit2

Unit2 contains all the DataSnap-specific logic for your stand-alone (Indy based) web server. In the structure pane, you can see that it registers support for your templates folder, and mime type support for css, javascript, and png images. It also provides the REST and JSON support your server will use.

    ServerMethods1

In ServerMethods1, you don't have to worry about the web server or marshalling the RPC (remote procedure calls) supported by the DataSnap server, because that's handled in the other units. You can focus instead on your custom logic.

Here's the initial class generated by the wizard. The public or published methods and classes contained between {$METHODINFO ON} and {$METHODINFO OFF} can be marshaled with DataSnap.

unit ServerMethodsUnit1;

interface

uses
  SysUtils, Classes, DSServer;

type
{$METHODINFO ON}
  TServerMethods1 = class(TDataModule)
  private
    { Private declarations }
  public
    { Public declarations }
    function EchoString(Value: string): string;
    function ReverseString(Value: string): string;
  end;
{$METHODINFO OFF}

implementation

{$R *.dfm}

uses StrUtils;

function TServerMethods1.EchoString(Value: string): string;
begin
  Result := Value;
end;

function TServerMethods1.ReverseString(Value: string): string;
begin
  Result := StrUtils.ReverseString(Value);
end;

end.

As you can see from the code, EchoString and ReverseString are already implemented and ready to use. So let's test the application.

    Test the DataSnap server

Testing the application is pretty simple. We just run it with or without debugging support.

Hide image

Click the Open Browser button. This will open the default browser to the DataSnap server home page. The first time you run your DataSnap server by clicking Start or Open Browser, you may see a Windows Security Alert. Hide image

You can click Allow Access to finish loading your DataSnap server application.

The browser page will look something like this:

Hide image
Click to see full-sized image

You can click ReverseString to immediately verify your DataSnap server is running, and it will update the text box to say "C B A" instead.

A more effective way of testing is to use the JavaScript-based server function invoker (I told you we'd get back to this later!) to examine the available methods on your DataSnap server.

The server function invoker option is automatically available when you run your DataSnap/REST server on localhost. The following function enables it:

function TWebModule2.AllowServerFunctionInvoker: Boolean;
begin
  Result := (Request.RemoteAddr = '127.0.0.1') or
    (Request.RemoteAddr = '0:0:0:0:0:0:0:1') or (Request.RemoteAddr = '::1');
end;

Clicking the Server Functions link on the home page opens the server function invoker in a new browser window.

Hide image
Click to see full-sized image

As the plus sign indicates, you can expand both the DSAdmin (DataSnap admin) interface and the TServerMethods1 interface. DSAdmin has many methods available to it, which will have to be covered elsewhere. Let's expand TServerMethods1 and invoke ReverseString.

Hide image
Click to see full-sized image

Now that we've used the automatic testing interface available when you're running the DataSnap server on localhost, we can start adding our "business" logic.

    Add your business logic

Beneath the declaration for ReverseString, we'll add the following declaration:

function Add(x,y: integer): integer;

The editor will create the implementation stub for us if we use Class completion by pressing Ctrl+Shift+C, and our cursor will be positioned in the body of the method.

function TServerMethods1.Add(x, y: integer): integer;
begin

end;

Make the code return the addition of x and y like so:

function TServerMethods1.Add(x, y: integer): integer;
begin
  Result := x + y;
end;

Stop and re-run the server (with debugging) to rebuild it, and click the Start or Open Browser button.

After stopping and restarting the server, if you try to reuse your existing server invoker web page, you'll see an error like the following:

Hide image
Click to see full-sized image

You can refresh the page to establish a session with the new DataSnap server instance. Your new method will also be available after the refresh or reload.

We'll set a breakpoint on our new Add method in the IDE, and execute the call to Add from the invoker web page.

Hide image
Click to see full-sized image

The IDE will stop on the breakpoint, and you can see that x and y have the values specified (5 and 10) on the browser page.

We continue running the DataSnap server, and see the result in the browser:

Hide image
Click to see full-sized image

    Use CreateConnection from DBXUtils.pas

If you're going to use dbExpress for your ISAPI dll, you may want to use CreateConnection to open your database connections.

In the RAD Studio demos project on Sourceforge, there is a branch for RAD Studio XE. Underneath that branch in the Delphi/Database/dbExpress/Utils folder, you will find a set of dbExpress utility functions heavily used by the EDN team in the unit called DBXUtils.

CreateConnection is a utility function that creates and opens a TSQLConnection from a simple call. You pass in the name of the connection, and CreateConnection first looks in the directory containing your dll or exe for an .ini file. If the ini file is found, a section name matching your passed in parameter is used for the definitions of the database connection. It's nearly identical to what might be in your dbxconnections.ini file, but it's a convenient local connection definition that makes it a bit easier to deploy your application to a production server.

Here's an example local application INI file for your dbExpress connection. Database, UserName, and Password are required. Driver defaults to INTERBASE. Any extra properties can be set in the [*Extra] section for the database name:

[Employee]
Database=localhost:d:\ibdata\employee.ib
UserName=sysdba
Password=masterkey
Driver=INTERBASE

[EmployeeExtra]
Dialect=3

Here's the relevant code:

///  <summary>
///  Create and open a <c>TSQLConnection</c> with the specified connection name
///  trying to open it a) first by trying application.ini then b) by
///  reading dbxconnections.ini if a) fails
///  </summary>
///  <param name="AName">Name of connection to open</param>
///  <param name="AIniFile">Optional. Name of the Ini file.
///  Defaults to the application name + '.ini'
///  <remarks>
///  The object returned will need to be freed in the caller
///  </remarks>
function CreateConnection(const AName: string;
  AIniFile: string = ''): TSQLConnection;
var
  error1: string;
begin
  try
    if FileExists(ModuleIniFile) then
      Result := CreateConnectionIni(AName)
    else
    begin
      Result := nil;
      error1 := Format(StrIniWasNotFound, [ModuleIniFile]);
    end;
  except
    on E: Exception do
    begin
      error1 := E.Message;
      Result := nil;
    end;
  end;
  if not Assigned(Result) then
    try
      Result := CreateConnectionName(AName);
    except
      on E: Exception do
      begin
        raise Exception.CreateFmt(StrCouldNotOpenConnection,
          [AName, error1, E.Message] );
      end;
    end;
end; 

If this ini file isn't found or CreateConnectionIni raises an exception, the routine attempts to open the connection using the default behavior of referring to dbxconnections.ini for the connection definition (via the call to CreateConnectionName).

It is also easy to manage the lifetime of your database connection with each instance of your server methods module. For example, here's the interface:

  Private
    FConnection: TSQLConnection;
    function GetConnection: TSQLConnection;
  public
    property Connection: TSQLConnection read GetConnection;
    destructor Destroy; override;

And here's the implementation:

function TRoleCall.GetConnection: TSQLConnection;
begin
  if not Assigned(FConnection) then
    FConnection := CreateConnection(SConnectionName);
  if FConnection.ConnectionState <> csStateOpen then
    FConnection.Open;
  Result := FConnection;
end;
destructor TRoleCall.Destroy;
begin
  if Assigned(FConnection) then
  begin
    FConnection.Close;
    FreeAndNil(FConnection);
  end;
  inherited;
end;

CreateConnection becomes particularly valuable when using an ISAPI dll, because the dbExpress run-time looks for the dbxconnections.ini file in the host environment's install directory (where IIS is installed) rather than in the directory where you deployed your DataSnap dll. This behavior should change in a later version to look in your module's local folder first (using logic similar to the DBXUtils ModuleIniFile routine) for locating your application .ini file.

    Convert the server to ISAPI

When the DataSnap server works as desired, both Unit2 (or whatever its final name is) and your server methods unit in an ISAPI version of the DataSnap server. The following steps describe how to do this using the DataSnap server wizard again.

    Create new DataSnap/REST ISAPI dll server

Start the DataSnap REST application wizard as described above and run it again.

Hide image
Click to see full-sized image

Select ISAPI dynamic link library and click Next all the way through to the last step. We're simply going to discard all the files it creates except for the project files, so it doesn't matter which options you select.

Hide image
Click to see full-sized image

Select a location "near" your other project so it's easy to copy the project files over, and click Finish to complete the wizard.

Save all files, naming the project something easy to recognize. Use File | Close all and close the project.

    Copy the ISAPI project files to your VCL project folder

In this example, the new project is called DSIsapi.dll and the the DataSnap VCL form project is Project8.exe.

From your file manager, copy DSIsapi.dpr and DSIsapi.dproj to the same folder as your DataSnap VCL form application.

Reopen your DataSnap VCL project. Right click on ProjectGroup1 in the project manager, and select Add Existing Project.

Hide image

Select your ISAPI project to open.

If you didn't copy the project .RES file over, you'll get a warning that it was recreated. That's ok.

Hide image

After that, the project will appear in the project group.

Hide image

    Remove generated unit refs from IIS project

You can remove the references to ServerMethodsUnit2 and Unit3 by selecting both of the files in the project manager and selecting Remove from Project.

Hide image

    Add the VCL project units to IIS project

Select ServerMethodsUnit1 and Unit2 from the VCL project and drag and drop them onto DSIsapi.DLL. This will bring up a dialog.

Hide image

Answer Yes and the files will be part of the project.

    Use MidasLib to embed MIDAS.DLL into your application

If you're going to be using datasets in your DataSnap application, I recommend using the MidasLib unit in your project source code. This simplifies the deployment of your application because you don't have to deploy or register MIDAS.DLL.

Go to the project manager, right click on the project, and select View Source.

Hide image
Click to see full-sized image

(You can also click on the project in the project manager and press Ctrl+V, which is an overridden keystroke for the project manager that views the source of the selected project.)

This will display your project source code. Just add MidasLib to the uses clause, save and rebuild the project.

library DSIsapi;

{$R *.dres}

uses
  ActiveX,
  ComObj,
  WebBroker,
  ISAPIApp,
  ISAPIThreadPool,
  DBXCommon,
  DSService,
  MidasLib,
  Unit2 in 'Unit2.pas' {WebModule2: TWebModule},
  ServerMethodsUnit1 in 'ServerMethodsUnit1.pas' {ServerMethods1: TDataModule};

    Test the ISAPI version with IIS 7.5

Compile the DLL. Now that it exists, we're ready to configure IIS to run your DataSnap server.

Many of the steps covered here are only required one time, but they are necessary. Even if they're set, it's good to have them as review points for troubleshooting.

    Make sure ISAPI is enabled in IIS

For IIS 7.5, you can start the IIS manager, select the machine name, and enter "isapi" in the filter box. This will display the ISAPI and CGI Restrictions feature.

Hide image
Click to see full-sized image

Double click on the feature name or click Open Feature in the Actions list. Select Edit Feature Settings from the Actions list. Hide image
Click to see full-sized image

Check Allow unspecified ISAPI modules and click OK.

    Configure the IIS virtual directory

Right mouse click on Default Web Site and select Add Virtual Directory.

Hide image
Click to see full-sized image

Enter the Alias (a simple name with no special characters) and put in the physical path to the DLL.

Hide image

Click OK, and right mouse click on the virtual directory. Select Convert to application.

Hide image

The following dialog is displayed.

Hide image

The default values should be fine, so click OK.

    Check your application pool user account

Security behaviors have changed from IIS 6 to IIS 7 and up. More information about IIS 7.5's account implementation is available on the learn.iis.net web site.

Account preferences can be verified the IIS application pools using the IIS 7.5 management interface.

Select Application Pools, then select DefaultAppPool and click Advanced Settings in the Actions list.

Hide image
Click to see full-sized image

On the Advanced Settings dialog, select the Process Model Identity and click the ellipses.

Hide image
Click to see full-sized image

Select ApplicationPoolIdentity in the built-in account and click OK.

Note: if you're running operating system is 64-bit, also set Enable 32-bit Applications to True under the (General) tab so your 32-bit dll will run.

Hide image

Click OK to close the Advanced Settings dialog also.

Now that we've ensured the appropriate account is used for the application pool, we can configure the appropriate rights for the DataSnap project's folder.

    Add IIS user "write" rights to the project folder

There are two ways to modify the rights for the appropriate account on your computer. The ability to write files is required because of the built-in logic your DataSnap REST server has for automatically updating the JavaScript proxy file for your custom logic. The following routine shows how it works.

procedure TWebModule2.WebFileDispatcher1BeforeDispatch(Sender: TObject;
  const AFileName: string; Request: TWebRequest; Response: TWebResponse;
  var Handled: Boolean);
var
  D1, D2: TDateTime;
begin
  Handled := False;
  if SameFileName(ExtractFileName(AFileName), 'serverfunctions.js') then
    if FileAge(AFileName, D1) and FileAge(WebApplicationFileName, D2) and (D1 < D2) then
    begin
      DSProxyGenerator1.TargetDirectory := ExtractFilePath(AFileName);
      DSProxyGenerator1.TargetUnitName := ExtractFileName(AFileName);
      DSProxyGenerator1.Write;
    end;
end;

Thanks to this routine, you can be assured that the server functions javascript interface is always current with the logic of your application. Since you are in control of the source code, you can modify this behavior if you want – but this routine requires write access to the directory containing your DLL, which is why we're about to modify rights.

If you want to avoid granting write access, you can modify your build and deploy process (particularly with the FinalBuilder bundled with RAD Studio XE) to generate your Javascript file during the build process and use an $IFDEF for production versus debug code so the server only generates the Javascript file locally when running your application in the debug environment.

Note: As Bob Swart pointed out, the js folder is the only folder in your project that needs write access. Bob has some other good feedback on his forum post as well.

Within IIS 7.5's management UI

With IIS 7.5, you can go directly to your virtual folder and click the Edit Permissions action to display the properties dialog for your folder. If you're using a version of IIS manager that has the Edit Permissions action, it's probably more convenient to use that for configuring rights to your virtual directory.

However, since deploying to IIS 6 will also be covered, the steps for setting access rights will be done with Windows Explorer.

With Windows Explorer

You can also set security options from the project folder in Windows explorer. The screenshot below is from Windows 7. The Windows Server 2003 dialog looks a little different, but the steps are basically the same.

Go to your project folder, select the js folder from it, right mouse click, and select Properties to bring up the Properties dialog.

Hide image
Click to see full-sized image

Select the Security tab and click Edit, which brings up the permissions dialog.

Hide image

Click Add to display the dialog for adding an account. Put in "IIS AppPool\DefaultAppPool" and click Check Names.

Hide image

It should change to say DefaultAppPool.

Hide image

Click OK to close the dialog.

Select DefaultAppPool on the permissions dialog, and check Allow for the Write permission.

Hide image

Check Allow for the Write permission, and click OK.

Click OK on the properties dialog to accept the permission changes and close the dialog.

    Configure Authentication for your virtual directory

The virtual directory also needs to be configured to use the application pool identity for its authentication.

Select your virtual directory in IIS manager, select Authentication, and either double click on it or click Open Feature in the Actions list.

Hide image
Click to see full-sized image

Select Anonymous Authentication and click the Edit action.

Hide image

Select Application pool identity and click OK.

    Local debugging using the IDE

Open the browser to the URL for your DataSnap server, and it should be working now. If you've missed a step, you'll see something like this:

Hide image
Click to see full-sized image

If you have it right, you'll see something like this:

Hide image
Click to see full-sized image

Attach to process for debugging

We can use Attach to Process in the IDE to debug this DLL now. Set a breakpoint on the line of code you want to debug (the calculation of Add is used in the screenshot below), and select Run | Attach to Process to hook into w3wp.exe, which is the IIS process.

Hide image
Click to see full-sized image

This displays the the attach to process dialog. Check the include system processes checkbox and locate w3wp.exe.

Hide image

Click Attach, and your IDE will switch to debug mode and (by default) pause at the CPU view. Hit run to continue.

Note: if you don't see w3wp.exe, it may have shut itself down after being idle. Refresh the browser page for your DataSnap application, click the Refresh button on this dialog, and it should be visible again. As you can see from the path of w3wp.exe above, I'm using 64-bit IIS.

Go to the server functions page with the browser and call Add with the values you want to add.

Hide image
Click to see full-sized image

The IDE will stop on your breakpoint.

Hide image

Click run to continue and return the result to your Javascript caller.

Hide image

    Updating your dll

IIS "holds on" to the DLL once it's been invoked, so if changes are required, IIS will need to be told to unload the DLL. Otherwise, rebuilding the dll will fail because the file is already in use.

There are a few options to update the DLL.

  • Use IIS manager to recycle the application pool used by your DataSnap server. This will unload your DLL and it can be rebuilt.

Hide image
Click to see full-sized image

  • Configure the application pool to quickly recycle by clicking on the Recycling action shown above.

Hide image
Click to see full-sized image

  • Use the command-line to stop and start the World Wide Web publishing service
d:\>net stop w3svc
The World Wide Web Publishing Service service is stopping.
The World Wide Web Publishing Service service was stopped successfully.
d:\>net start w3svc
The World Wide Web Publishing Service service is starting.
The World Wide Web Publishing Service service was started successfully.
  • Use the services dialog to restart the World Wide Web publishing service

Hide image

When you're done debugging the application, you're ready to deploy to the production environment!

    Deploy to IIS 6

Our production servers are still primarily Windows Server 2003 with IIS 6, so this section covers what's involved in configuring the DataSnap ISAPI dll IIS 6.

    Create the application directory

We use a standard folder (\apps) to host all our applications. I'll create d:\apps\datasnap on the server, and copy the dll file, and the css, images, js, and templates directories there.

Hide image

    Add IIS user "write" rights

For IIS 6, the account ISUR_<MACHINENAME> should be used for the DataSnap application. For example, if the machine is named MYSERVER, the user would be IUSR_MYSERVER. The steps are nearly identical to those outlined above.

    Allow ISAPI modules

Bring up IIS manager 6, select Web Service Extensions, then All Unknown ISAPI Extensions, and click Allow.

Hide image
Click to see full-sized image

Say Yes to the warning about the security risk.

    Configure the IIS virtual directory

Select the appropriate web site, right mouse click and select New | Virtual Directory.

Hide image

This displays the Virtual Directory wizard.

Hide image

Click Next to go to the next step.

Hide image

Provide the directory alias and click Next.

Hide image

Put in the path for the directory and click Next.

Hide image

Check Run scripts and Execute and click Next.

Hide image

Click Finish to complete the wizard.

    Put the dbExpress DLLs in the system path

If you're using dbExpress for your DataSnap application, you'll need to put whatever dbExpress DLLs you use into your system path. For example, InterBase requires dbxint.dll to be available in the system path.

    Test with local host

Open the browser page on your local machine and make sure the DataSnap server works like it did when you were testing it.

    Launch the app!

When everything is working as expected, you're done!

Of course, now that the DataSnap server is configured, you can use the techniques covered here to quickly deploy updates to it.

I look forward to seeing the DataSnap implementations people come up with.

Enjoy!

Server Response from: ETNASC04