Abstract: Using Ruby to automatically compile Delphi projects, resource scripts, install scripts, ...

Introduction

Compiling a Delphi project is dead simple. Hit Ctrl-F9 and 0.9 seconds later you're done.

The problems arise when you start looking at all the other stuff that needs to be done:

Run unit tests
Update version resources
Compile help files
Build setup applications
Zip the setup application
Upload the zip to a FTP site
Upload the documentation to a website

The list goes on. Each step by itself is easy enough, but in combination it's tedious and prone to error. An automated build system promises to reduce the misery. What follows in this article is a description of how I use Ruby to automate this build process.

In Code Central you'll find a zip file with a Ruby script called "buildTools.rb" that provides the core functionality of the build system. Also included is a Delphi configuration file with all the options documented, a sample Delphi project, and "build.rb" file which is the build script for the sample project.

Getting Started

Where To Get Ruby

The official Ruby home page is www.ruby-lang.org . The "One-Click Installer" is all it takes to get started with Ruby. For assistance with Ruby check out the comp.lang.ruby newsgroup. Dave Thomas and Andy Hunt have kindly made available a CHM version of the 1st edition of their book "Programming Ruby" which is located in the ruby\doc directory. This book is one of the better references to Ruby. You can also order the 2nd edition from their website.

Installing Ruby

The first question is, "Where do you install it?" By default Ruby installs into C:\ruby but I prefer to install it into a subdirectory of my projects folder (P:\builds\ruby). This allows me to easily place it under version control. I'll talk more about the directory structure a couple sections down.

Configuring Ruby

I keep my Ruby scripts in P:\build rather than in ruby\lib, consequently the path list that Ruby uses to search for files needs to be updated. Appending "-IP:\build" to the RUBYOPT environmental variable (without the quotes) solves the problem.

Team Development

The Ruby install automatically configured the PATH and the PATHEXT environment variables for us and also set up file associations for .RB and .RBW. Rather than force every programmer on the team to go through the download and setup process, just create a DOS command script that performs these same actions and place it under version control in the P:\build directory. They can then simply check out P:\build and run the script. This should do the trick nicely:

            @echo off

            :: check if the ruby\bin path already exists in the PATH var
            set rubyPath=%CD%\ruby\bin
            for %%x in ("%path:;=" "%") do (
                if /I %%x == "%rubyPath%" (
                    echo ruby\bin already exists in PATH - no change required.
                    goto skipSettingPath
                )
            )
            setx PATH %PATH%;"%rubyPath%"
            echo Added ruby\bin directory to PATH variable.
            :skipSettingPath

            :: check if the Ruby file extension .rb already exists in PATHEXT
            for %%x in ("%pathext:;=" "%") do (
                if /I %%x == ".RB" (
                    echo RB already exists in PATHEXT    - no change required.
                    goto skipSettingRB
                )
            )
            setx PATHEXT %PATHEXT;.rb
            echo Added .RB to PATHEXT variable.
            :skipSettingRB

            :: check if the Ruby file extension .rbw already exists in PATHEXT
            for %%x in ("%pathext:;=" "%") do (
                if /I %%x == ".RBW" (
                    echo RBW already exists in PATHEXT   - no change required.
                    goto skipSettingRBW
                )
            )
            setx PATHEXT %PATHEXT;.rbw
            echo Added .RBW to PATHEXT variable.
            :skipSettingRBW

            :: create file types
            ftype rbFile=%rubyPath%\ruby.exe "%%1" %%*   > nul:
            ftype rbwFile=%rubyPath%\rubyw.exe "%%1" %%* > nul:

            :: setup file associations. Do NOT put spaces before ">"
            assoc .rb=rbFile>nul:
            assoc .rbw=rbwFile>nul:

            echo File associations configured.

            :: append current directory to RUBYOPT
            set option=-I%CD%
            for %%x in ("%rubyopt: =" "%") do (
                if /I %%x == "%option%" (
                    echo %option% already exists in RUBYOPT - no change required.
                    goto skipSettingRubyOpt
                )
            )
            setx RUBYOPT "%RUBYOPT% %option%"
            echo Added %option% to RUBYOPT variable.
            :skipSettingRubyOpt
            
            :: the build scripts rely on PROJECTS pointing to the root of your
            :: projects directory, which is probably the parent directory of the
            :: current directory
            set originalDir=%CD%
            cd ..
            setx PROJECTS %CD%
            cd %originalDir%

            echo Done.

Directory Structure and File Paths

I've dedicated my P: drive to my projects, and I'm using P:\build to store all of the tools required for building my projects. This includes five different versions of Delphi, two different versions of the .Net framework, HelpMaker , InnoSetup and of course, Ruby itself.

Throughout the rest of the article I'm going to assume that you're using the same directory structure as shown below, but you can easily adjust it to your environment by modifying the path constants at the top of buildTools.rb. This script also requires the PROJECTS environment variable to point to P:.

Hide image

Note that Ruby prefers forward slashes in filenames while Windows prefers backslashes. Both will often - but not always - accept the other. Since most of the functions in buildTools.rb call Windows applications, it's best to use backslashes. Paths can be absolute but it's better to express them relative to the directory that the build script is located in. By convention the build script is called build.rb.

Compiling Delphi

The buildTools.rb script contains the core functions for compiling Delphi (and other) projects. Among other things it defines a function called "compileDelphi" which allows us to compile a DPR or DPK file. For example, it might be called from our build.rb script like this:

        compileDelphi(Delphi10,"sample.dpr","Standard.cfg",'')

The first parameter specifies the Delphi version we wish to use. "Delphi10" is a constant defined in buildTools.rb which defines the path to the directory containing the Delphi command line compiler and libraries.

The second parameter is the project or package to compile. Compiling an entire package group at once is not supported. The path to the project is relative to the current directory (by default this is the directory that contains the build script).

The third parameter is the configuration file. I always create a separate *.cfg file for use solely with the build script. This ensures that the IDE doesn't modify the config file without our knowledge and thus helps create a more stable build system. I use the same config file whether I'm building a debug version or a release version, which leads use to the fourth parameter.

The fourth parameter is optional, but allows you to specify additional compiler options which will override those in the config file. For example, the following two lines will first build a test release, and then build a production release:

        compileDelphi(Delphi10,"sample.dpr","Standard.cfg",
            "-VN -DUSE_JEDI_JCL;FullDebugMode;NoMessageBoxes")
        compileDelphi(Delphi10,"sample.dpr","Standard.cfg")

The test release has debugging information included (-VN) and defines the constants USE_JEDI_JCL, FullDebugMode and NoMessageBoxes. These constants are used by DUnit and FastMM. There's a similar "compileDelphiNet" function for compiling Delphi.Net applications.

Setting up the config file for the very first time can be a daunting task. Who can remember what -$W- or -$YD means? To make it a bit easier I use a config file that contains comments about each option. The one shown below works with Delphi 7 and might need some minor adjustments to work with other versions. You'll notice that \build\Delphi7\lib is specified via a -U switch. This allows the command line compiler to find the standard Delphi libraries. You'll also need to add references to any other libraries that you're using.

        ;==============================================
        ;Directory paths
        ;==============================================
        ;-U... unit directories
        ;-O... object directores
        ;-I... include directories
        ;-R... resource directories

        -U\build\Delphi7\lib

        ;.dcu output directory
        ;-N0.\dcu

        ;.bpl (LE) & .dcp (LN) output directory
        ;-LE
        ;-LN

        ;exe/dll output directory
        ;-E.\bin

        ;use package
        ;-LU(package)

        ;==============================================
        ;Conditional symbols    (-Dblip)
        ;==============================================
        ;-DTEST_MODE

        ;==============================================
        ;Likely to change
        ;==============================================
        -cg             ; application type (g=GUI, c=console)
        ;-VN            ; include Debug information in EXE
        ;-VR            ; generate remote debug info (*.RSM)

        ;==============================================
        ;Warnings
        ;==============================================
        -w-UNSAFE_TYPE
        -w-UNSAFE_CODE
        -w-UNSAFE_CAST
        -w+SYMBOL_DEPRECATED
        -w+SYMBOL_LIBRARY
        -w-SYMBOL_PLATFORM
        -w+UNIT_LIBRARY
        -w+UNIT_PLATFORM
        -w+UNIT_DEPRECATED
        -w+HRESULT_COMPAT
        -w+HIDING_MEMBER
        -w+HIDDEN_VIRTUAL
        -w+GARBAGE
        -w+BOUNDS_ERROR
        -w+ZERO_NIL_COMPAT
        -w+STRING_CONST_TRUNCED
        -w+FOR_LOOP_VAR_VARPAR
        -w+TYPED_CONST_VARPAR
        -w+ASG_TO_TYPED_CONST
        -w+CASE_LABEL_RANGE
        -w+FOR_VARIABLE
        -w+CONSTRUCTING_ABSTRACT
        -w+COMPARISON_FALSE
        -w+COMPARISON_TRUE
        -w+COMPARING_SIGNED_UNSIGNED
        -w+COMBINING_SIGNED_UNSIGNED
        -w+UNSUPPORTED_CONSTRUCT
        -w+FILE_OPEN
        -w+FILE_OPEN_UNITSRC
        -w+BAD_GLOBAL_SYMBOL
        -w+DUPLICATE_CTOR_DTOR
        -w+INVALID_DIRECTIVE
        -w+PACKAGE_NO_LINK
        -w+PACKAGED_THREADVAR
        -w+IMPLICIT_IMPORT
        -w+HPPEMIT_IGNORED
        -w+NO_RETVAL
        -w+USE_BEFORE_DEF
        -w+FOR_LOOP_VAR_UNDEF
        -w+UNIT_NAME_MISMATCH
        -w+NO_CFG_FILE_FOUND
        -w+IMPLICIT_VARIANTS
        -w+UNICODE_TO_LOCALE
        -w+LOCALE_TO_UNICODE
        -w+IMAGEBASE_MULTIPLE
        -w+SUSPICIOUS_TYPECAST
        -w+PRIVATE_PROPACCESSOR
        -w+MESSAGE_DIRECTIVE

        ;==============================================
        ;Historical options
        ;==============================================
        -$H+            ; if on 'string' means a long string (AnsiString), if
                        ;   off, it means a short string (ShortString)
        -$P+            ; 'openstring' checking
        -$V+            ; var string checking - if on, then the argument's
                        ;   length must match parameter's length

        ; unit aliases:
        -AWinTypes=Windows;WinProcs=Windows;DbiTypes=BDE;DbiProcs=BDE;DbiErrs=BDE;


        ;==============================================
        ;Unlikely to change
        ;==============================================
        -$A8             ; align record fields on 8 byte boundary
        -$B-             ; off means disable full boolean evaluation
        -$C+             ; evaluate assertions
        -$D+             ; include line number info in DCUs (has no effect on
                         ;   final executable)
        -$G+             ; use imported data references - allows packages to
                         ;   reference variables in other packages
        -$I+             ; i/o checks, if on an exception will be raised on an
                         ;   i/o error, if off, the IOResult variable will be set
        -$J-             ; if on, typed constants can be modified
        -$L+             ; local symbol info - if enabled, allows the debugger
                         ;   to inspect the values of variables declared in the
                         ; implementation section of the unit, as well as the
                         ;   variables local to a method/function/procedure.
                         ; (no effect on executable size)
        -$M-             ; typeInfo - adds RTTI info for published methods and
                         ;   properties
        -$O+             ; optimization
        -$Q+             ; overflow checks
        -$R+             ; range checks
        -$T-             ; type @ operator
        -$U-             ; if on, protects against divide flaw found in early
                         ;   Pentiums
        -$W-             ; stack frames
        -$X+             ; extended syntax (enables functions as procedures,
                         ;   'result' variable, assigning strings to PChar)
        -$YD             ; generates info used for code navigation by the
                         ;   Project Manager, Code Editor, and Code Explorer
                         ;   $Y- = disabled
                         ;   $YD = include info on where identifiers are declared,
                         ;         and where routines are implemented
                         ;   $Y+ = include same info as $YD, but also include info
                         ;         on where the identifiers are referenced
        -$Z1             ; minimum size of an enumeration - either 1, 2, or 4
        -H+              ; hints on
        -W+              ; warnings on
        -$M16384,1048576 ; min & max stack size
        -K$00400000      ; set image base address
        -GD              ; generate map file (D = detailed, P = publics only,
                         ;   S = segments only)
        -B               ; build rather than make
        -Q               ; quiet compile

        ;==============================================
        ;Purpose unknown, probably not significant
        ;==============================================
        -$E-
        -$F-
        -$K-
        -$N+
        -$S-

        ;==============================================
        ;I'm unlikely to use these
        ;==============================================
        ;-M                              ; make
        ;-F(offset)                      ; Find error
        ;-J                              ; Generate .obj file
        ;-JPHNE                          ; Generate C++ .obj file, .hpp file, in
                                         ;   namespace, export all
        ;-NH(path)                       ; unit .hpp output directory
        ;-NO(path)                       ; unit .obj output directory
        ;-NB(path)                       ; unit .bpi output directory
        ;-NS(namespaces)                 ; Namespace search path
        ;-P                              ; look for 8.3 file names also
        ;-Z                              ; Output 'never build' DCPs
        ;--codepage:(cp)                 ; specify source file encoding
        ;--default-namespace:(namespace) ; set namespace
        ;--depends                       ; output unit dependency information
        ;--doc                           ; output XML documentation
        ;--drc                           ; output resource string .drc file
        ;--no-config                     ; do not load default DCC32.CFG file

The buildTools.rb script expects to find the Delphi files in P:\builds\Delphi{version}. At a minimum you need a "P:\builds\Delphi{version}\bin" subdirectory that contains:

File D5 D6 D7 D8 D9 D10

brcc32.exe Y Y Y N ? Y

dcc32.exe Y Y Y N ? Y

dccil.exe N N N Y ? Y

lnkdfm50.dll Y N N N ? N

lnkdfm60.dll N Y N N ? N

lnkdfm70.dll N N Y N ? N

lnkdfm100.exe N N N N ? Y

rlink32.dll Y Y Y Y ? Y

rw32core.dll Y Y Y Y ? Y

Borland.Vcl.LnkDfm.dll N N N Y ? N

Borland.Delphi.Compiler.ResCvt.dll N N N N ? Y

I don't own Delphi 2005 (i.e.: D9), but I expect that the files it requires are similar to Delphi 8.

The P:\build\Delphi{version}\lib directory should contain all of the standard Delphi files found in the Delphi lib directory (typically C:\Program Files\Borland\Delphi{version}\lib).

Other Tools

InnoSetup

As mentioned earlier, I use JR Software's InnoSetup to build setups. Calling the InnoSetup command line compiler is very straightforward:

            compileInnoSetup("setup.iss","")

The second parameter is optional but can be used to pass options to the iscc.exe program. Currently iscc supports just three parameters:

            /Opath      Output files to the specified path
            /Ffilename  Specifies the filename for the setup program
            /Q          quiet compile

While these parameters are useful, I need greater flexibility than what these provide. This is where the filterFile function comes into play. This is a simple Ruby function that takes the name of a source and destination file and an array of substitutions to apply to the file. For example:

            substitutions = [
                ['{password}', "secret"       ],
                ['{version}',  "1.2.3.892"    ],
                ['{encrypt}',  "Yes"          ],
                ['{fileDir}',  "P:\\myapp"    ]
            ]
            filterFile("setup\\setup.iss","setup\\tempSetup.iss",substitutions)
            compileInno("setup\\tempSetup.iss")

The substitutions variable is an array of arrays with two elements each. The filterFile function will search for all occurrences of the first element and replace it with the second element. The setup script looks like this:

            [Setup]
            LicenseFile         ={fileDir}\license.txt
            Encryption          ={encrypt}
            Password            ={password}
            VersionInfoVersion  ={version}

            [Files]
            Source: {fileDir}\license.txt; DestDir: {app}; Flags: ignoreversion
            ...

There's nothing magical about using braces to enclose the search strings - you can use any character or none at all. The "{app}" string in the above setup script is a standard InnoSetup variable.

HelpMaker

VizAcc's HelpMaker supports compiling help files from the command line so it's very easy to integrate it into our build process:

            compileHelp(   "help\\myapp.sh5")    # standard CHM help file
            compileWebHelp("help\\myapp.sh5")    # generates browser-based help

You can also use "compileHtmlHelp" to invoke the standard Microsoft Help compiler.

I like to include the version number of the software in the help file, but HelpMaker doesn't provide any way of doing this automatically. The filterFile function comes to the rescue again:

            compileHelp("help\\gastown.sh5")
            filterFile("help\\_tmphhp\\introduction.htm", '',
                [['{version}', "1.3.2.892"]] )
            compileHtmlHelp("help\\_tmphhp\\gastown.hhp")

This script uses compileHelp to launch HelpMaker. HelpMaker converts its *.SH5 file to a set of HTML files, and then it invokes the Microsoft Help Compiler to compile them into a CHM file. HelpMaker normally deletes the HTML files afterwards but HelpMaker has an option to disable this. This allows us to use the filterFile function to insert the current version in a HTML file, and then call the Microsoft Help Compiler ourselves to recreate the *.CHM file. It's a roundabout way of doing it, but it works well. Since we left the name of the destination file blank in the call to filterFile it will modify the source file itself rather than create a new file.

Resources

I prefer to manage version information in a separate resource file (*.RC) rather than let the Delphi IDE handle it. Resource files can include other information such as images, SQL scripts, etc. The compileResource function invokes the Borland BRCC.EXE application to compile these scripts. You can specify the file and product version as well as any other substitutions.

            compileResourceScript(
              'main.rc',                      # the resource file
              '1.3.2.59',                     # the file version
              '3.1',                          # the product version
              [['{buildTime}',Time.now]]      # any other substitutions
            )

Unit Tests

You'll notice that there are functions in buildTest.rb for unit testing. Since these use a slightly customized version of DUnit and a simple home-grown testing framework for .Net, I doubt that these will be useful to you in their current form, but they should give you a starting point for creating your own functions.

Utility Functions

There are a variety of utility functions in buildTools.rb, mainly focused on file management. Some are just convenient wrappers around standard Ruby functions. Peruse the file to learn more.

Real World Example

Here's a real-life example of a build script that I use for compiling one of my standard utility packages. This script compiles a resource script, compiles the test project, runs the unit tests, and then compiles the production package -- all in just five statements.

        require "buildTools"

        # Compile the project used for unit testing
        compileResource('main\utStringsTest.rc')
        compileDelphi(Delphi10,'utilsTest.dpr', 'Standard.cfg',
            '-DUSE_JEDI_JCL;FullDebugMode;NoMessageBoxes;LOG_TEST'
        )

        # run the test project. This raises a Ruby exception if a test fails
        dunitTest('..\bin\utilsTest.exe')

        # compile the utility package
        compileDelphi(Delphi10,'utils.dpk', 'Standard.cfg')

Summary

This provides a very good start on a build system. Although I haven't discussed it here I also have a "autobuild.rb" script which scans the entire projects directory looking for "build.rb" scripts, executes them one by one, and logs results to file and email. It can also invoke VMWare to perform the builds in a clean environment.

Building projects need not be an onerous task. With a bit of effort up front you can develop a system that guarantees a smooth, low-effort, predictable build process.