[All]
InterBase 6 Certification and Multi-Platform Test Automation
By: Conference Speaker
Abstract: This paper discusses the structure and ideology of InterBase QA and how our open source community can participate in development testing using TCS, the InterBase Test Control System.
 |
This is the technical paper from a talk given at the 12th Annual Borland Developer's Conference
By Aaron Ruddick – Borland Software Corp. |
Aaron Ruddick has worked on both the JBuilder and InterBase projects. Currently, he is a quality assurance engineer for the InterBase product group. One of his latest projects is refining our cross-platform testing and performance tools. Aarons hobbies are fishing and programming.
Note: No portion of this paper may be distributed or reproduced by any means, or in any form, without the author's and/or Borland's prior written permission.
Introduction
The InterBase QA strategy has been developed and passed on for over 15
years. Today, with our open source community and various InterBase
projects, we are now able to include the entire community in our process.
In this session, I will show you not only how our system currently works,
but how you, as our InterBase partners, can participate in the process.
Overview of InterBase Development Cycle
We break our development cycle into five phases:
Phase_0
The purpose of Phase 0 is to evaluate product opportunities in a structured
manner.
It explains the product concept and assess the concept's fit with the company
product line strategies.
Phase_1
Phase 1 is the fundamental building block in a product development process.
The product is fully defined
and the program is planned. It defines WHAT the product is.
Phase_2
Phase 2 focuses on refining and finalizing the definition of the product.
This requires the completion of
high-level and detailed designs and functional planning for Development,
QA, Publications, Product Marketing,
Marketing Communications, and Support.
Phase_3
All implementation work is completed, with the exception of documentation
refinements, and bug fixing.
At this time Product Marketing and Marketing Communications are finalizing
the product introduction plans.
The product is tested to verify it performs in accordance with the Product
Specification (Phase 1).
Phase_4
Phase 4 validates the products readiness for release. At this time Media
Control produces the pre-master
and master media, for final testing. RTM marks the initiation of associated
launch, Marketing
Communications, and Support activities.
How the community participates in our Development Cycle
Our community is involved in every phase of development:
Phase_0
At the start of every development cycle we filter through hundreds of customer
feature requests and bug reports.
Currently, bugs and feature requests can be entered by going to: http://www.borland.com/devsupport/
Phase_1
This is where we finalize our plan. Your proposed features, that
have made it through our Bug Counsel and
and Core Team, get written up in our "Product Definition".
Phase_2
In phase 2 we will take the "Product Definition" and create both Design,
Functional, and Testing Specifications.
In addition to QA's testing specs. is our Beta testing plan. This
is where you, as InterBase community, will get
to see the product as it progresses. Field testers have the advantage
of using our new cutting edge technologies
before the rest of the world. Without your help QA would never find
the hidden bugs that only "real world"
programs and users will find.
Phase_3
Testing, testing, and more testing. In phase 3, QA and Field testers
do their best to break the product. In addition to
our Test Control System (TCS), we do rigorous manual and load testing.
With the help of our field testers we find,
and R&D fixes, every bug we can get our hands on. This process
can be a long cycle because every time R&D fixes
a new bug QA must fully retest the entire product. And, as all developers
know, no product is without bugs, so this
QA cycle usually goes on until our director chains us in our offices and
runs away to put the cd to market.
Phase_4
In truth, no product leaves Borland without the signature of every Core
Team member. This final phase is our
"Development Hand-off". The final cd is made and our developers are
let out of their chains for one last round of testing.
In the frenzy of final testing, every engineer we have tries his very best
to, once again, break the product. This is
a dangerous time for our team. At any moment we could be maimed by
flying InterBase cd's and Microwave pizza box
explosions. However, after our director again intervenes, we take
the final cd, thoroughly tested and fought over,
and ship it off (RTM).
TCS for field testers
What is tcs...
TCS is InterBase's Test Control System. TCS started some time before
1988 and has evolved for over 12 years.
TCS is a test automation system that can be compiled and ran on virtually
any operating system that can run shell scripts
and InterBase. All tests, and their results, are stored as BLOBs
in InterBase. When ever a test is run, TCS will compare
the current result with the expected result and tell you if it passed or
failed.
How do I get tcs...
0. Download and install cvs: http://www.cvshome.org/dev/codewindow.html
1. Set your system path to the InterBase/bin and your /cvs directory.
2. cvs -d:pserver:anonymous@cvs.ib-tests.sourceforge.net:/cvsroot/ib-tests
login
3. cvs -z3 -d:pserver:anonymous@cvs.ib-tests.sourceforge.net:/cvsroot/ib-tests
co [modulename]
4. To make your binary: Go to the /tcs/tcs directory and
follow the directions in the README file.
How do I use tcs...
When you pull down the "tcs" module, you will have a /doc directory.
Inside are six documents:
0. PORTABLE_TESTS: Tips on Portable test writing in TCS
1. QA_NON_C_LANGUAGES: Guidelines for Writing Portable TCS
Language Tests
2. QA_TCS_QUICK_GUIDE: TCS quick setup guide and TCS notes
3. QA_WRITING_TCS_TESTS: Conventions for test writing in TCS
4. TCS_Q_AND_A: Only answers two questions:
a. Q: How does "Test Versioning" in TCS work?
b. Q: So, which init record version do we use for a given TCS test?
5. VECTOR_RUN_DOC: Explains the VECTORED TCS run method
The 6 documents above do not provide a complete manual for TCS. Currently
the only real way to
learn our system is to work with it. I have provided a brief tutorial
to get you started.
Summary
Now that our QA team consists of not only IB employees but the entire IB
community, we are open and ready for your input. Any ideas or thoughts
on the structure of the QA cycle are very welcome. We are ready and
willing to add your ideas to our model. You are always welcome to
email me your on ideas or post to our opensource newsgroups.
Connect with Us