UNIX network programming: a practical approach
By James Buchanan
If you've ever tried to write a network
application of kind and have been discouraged, do not despair: Here I present
the first of a series of articles focusing on practical network
programming -- specifically for UNIX.
That's not to say your new knowledge will be useless
on other operating systems. In fact the core networking API is pretty much the same
across all platforms. You will see them in any operating system with a
C library of networking functions: socket, listen, accept, bind, send,
and recv. Sometimes recv is #defined as read, and sometimes send is #defined
as write -- the details can differ but the overall approach is the same.
You don't have to use C. You can
use Perl if you want, or Java, or any other language of sufficient power
to write network programs. I will use C in these articles, partly because I've been using C for a long time, partly because
C is the language of UNIX, and partly because most programmers
know something about C. So I assume that you have a basic working knowledge
of the C language: how to write functions, how to handle files, how to
use the control structures, what a struct is, when to use the dot and
hyphen-arrow operators, what a pointer is and what include files are. You
should also have a conceptual appreciation of the Internet and some of
its common lingo: IP address, server, client, and port. Basic
stuff.
Computer science authors go to great pains to tell you how many of their previous books you
will need to read before you can write network programs, and how you need
to know every last detail of the networking protocols as a prerequisite.
What a load of garbage! A basic conceptual understanding of common
network ideas and a knowledge of a suitable programming language is all
you need to get a leg up on some serious, useful network programming.
Nobody writes useless toy applications
that talk over networks, or stupid "Hello World" network applications, when
they need to get a job done. If you're working for a Web development
company you will probably be called upon to write a program that gives
a Website shopping cart functionality, a chat system, message boards pages,
an online auction system, user/password login functionality, or any number
of other e-commerce facilities. The network programming in these articles will focus on practical applications that are
useful and have a leaning toward e-commerce. Maybe you'll go on to write
some great e-commerce software and earn a nice living at it.
Here is a brief rundown of the series I have planned. I
won't go into too much detail here because if I know myself, and I do,
I will change my mind about a few things down the track! But I do promise
that we will cover these topics:
- The most common networking functions you will need to use.
- Client-server design.
- Improving server performance in high-request situations.
- The use of fork() in servers and the family of exec... calls.
- POSIX pthreads -- writing a multithreaded server and why threads are better than
fork().
- Changing IPv4 programs to work with IPv6.
- Non-blocking I/O and other advanced networking I/O.
- Signal handling and writing server control programs as the interface to starting, stopping, and restarting servers.
- Using configuration files for servers.
We will also dissect heaps of real,
practical code, including an echo client and server which I will use to
illustrate common networking functions. Then we'll move on to more useful programs: a chat server and client, a
multi-tier Website auction system, an e-mail client and server, and finally we'll have some fun and implement a real,
working Web server and browser. We'll also dabble in some object-oriented
Java network programming.
So without further ado, let's go!
THE FLOW OF THINGS
A graphic would be in order here. No
doubt you know what happens (or can guess correctly) in a simple client-server
model, but let's have a look anyway. This diagram illustrates what happens in the
echo client and server:
CORE NETWORKING FUNCTIONS AND AN IMPORTANT DATA STRUCTURE
You will use a handful of common networking functions in virtually all network client-server programs you write. They are:
- socket
- bind
- listen
- connect
- accept
- recv
- send
You will also use a socket address
structure, a sockaddr_in. It is an IPv4 socket address structure, also known as an "Internet
socket address structure." It contains -- suprise suprise -- the socket's address.
It also holds the port number, the family of socket, the length (or size)
of the structure itself, and an unused array of 8 characters. It's used
directly in C programs, and indirectly in high level languages. sockaddr_in is a C
struct. More about how to use it in the next article in this series. Now on to the functions.
I would like you to have copies of
the echo client and server source code with you as you read through these
introductory articles, The source files will help put things in place and assist in
a conceptual understanding of the socket address structure and the
network functions. This is the level of understanding we want; after all,
we could waste a good few years getting a deep understanding of the guts
of the networking APIs, hacking networking protocols, and looking at kernel source
code -- and in the process never get any useful network programming done.
Or we can be happy with our conceptual, practical understanding and write
some killer network apps. OK let's rock! Get the source for the echo client
and server here.
The socket function gets a socket descriptor for you, which is like a handle and similar
to a file descriptor. It is the first function you will call in the
networking part of your programs. A socket is an endpoint of communication
through which applications can talk to each other.
The socket function is prototyped like this:
#include <sys/socket.h>
int socket(int family, int type, int protocol);
The header file shown here is of
course where it is prototyped. On success, socket returns a small nonnegative socket descriptor. When it fails,
it returns -1.The protocol family parameter is
normally the constant AF_INET (IPv4). For IPv6 we'd use AF_INET6. For UNIX
domain protocols it is AF_LOCAL (also known as. AF_UNIX), for routing sockets
AF_ROUTE,
and for key sockets AF_KEY. We will be using only AF_INET and AF_INET6
in this series of articles.
The protocol type parameter is normally SOCK_STREAM. You might also
encounter SOCK_DGRAM (datagram socket) and SOCK_RAW (raw
socket), but we won't be using these.
The protocol parameter is almost
always set to 0 (zero, not the capital letter O), except for raw sockets,
which as mentioned we won't be using.
The bind function is prototyped as follows:
#include <sys/socket.h>
int bind(int socket_fd, const struct sockaddr *myaddress, socklen_t address_length);
On success, bind returns 0. When it fails, it returns -1.
The bind function exists to assign a local protocol address to a socket, which is
a combination of an IPv4 or v6 address and a port number. It is dependent
upon the protocol used. Let's not worry about the boring details -- it's time to move
on.
The first parameter is the socket descriptor that the call to the socket function would have returned, if
it succeeded. The second parameter is a pointer to a protocol-specific
address structure, our friend the struct sockaddr_in for IPv4. (We'll talk
more about initializing our address structure a bit later. As it happens,
we can specify to bind just an address, just a port number, or neither, passing this job to the
kernel. Since TCP has well known port numbers, we'll be specifying the
port.) The third and last parameter is the size of the address structure,
which we give by passing sizeof(whatever-its-name-is).
This is the prototype of the listen function:
#include <sys/socket.h>
int listen(int socket_fd, int backlog);
The listen function returns 0 (zero) on success or -1 on error.The first parameter is our socket descriptor.
listen is normally called after socket
and bind but must be called before accept. The function causes the program to put its ear on the network, so to speak,
and listen for incoming requests from clients that wish to establish a
connection. Clients can and do call listen as well, as we shall see in a
later article.
The second parameter -- backlog -- is a funny one. If you ever get interested in the guts of networking
and study it in detail, you will see that documentation is often a little
vague about backlog. For our purposes, and
for most people's purposes, it means the number of connections that the
kernel will queue up. This definition is more than adequate. A simple server, or a useless one like a daytime server, may have a backlog
of just 5. A busy HTTP Web server serving up millions of Web pages a day
will have a larger backlog. Some Unix and Linux kernels
#define it as 5, while actually supporting many more connections. So we will use
a constant, LISTENQ, which is 1024.
The connect function is prototyped as:
#include <sys/socket.h>
int connect(int socket_fd, const struct sockaddr *server_address, socklen_t address_length);
connect is used by a TCP client to establish a connection with a TCP server. It
returns 0 if OK or -1 on error. The first parameter, socket_fd, is
the socket descriptor that was returned by the call to socket in the client. The second argument is a pointer to an address structure
containing information on the server address that the client wants to connect
to. Because it is constant and so cannot be modified in the function, we
must specify the IP address of the server and the port it is listening on -- the kernel cannot help by supplying this for us! (As opposed
to bind,
where we need not specify anything, if we were crazy or had no idea what
we were doing). The third and final argument is where we give sizeof(the-server-address-structure).
The accept function is prototyped as:
#include <sys/socket.h>
int accept(int socket_fd, struct sockaddr *client_address, socklen_t address_length);
accept accepts incoming client requests as its name suggests. It returns a nonnegative
client socket descriptor if OK, or -1 on error.The first parameter is the server's
socket descriptor, the second is a pointer to an address structure containing details about the client, and the third is the size of
the client address structure, again using sizeof(whatever-we-call-it-in-our-code).
recv is prototyped as:
#include <sys/socket.h>
ssize_t recv(int socket_fd, void *buffer, size_t buffer_size, int flags);
It returns the number of bytes actually
read if OK, or -1 on error. The first argument is the socket descriptor,
which will the the descriptor returned by the accept function (the client's socket, so we can tell
recv
where we will be receiving data from). The second is a void pointer to
a buffer, where we will stick all the data when we read it in. The third
argument is the size of the buffer, theoretically so the function doesn't
overwrite the bounds of the buffer. The fourth allows us to specify some
flags which we can OR together. We will specify 0 as the flags
for now, since we aren't interested in them at the moment. The possible
flags are: MSG_DONTROUTE, MSG_DONTWAIT, MSG_OOB, MSG_PEEK and MSG_WAITALL.
We'll talk about them later in this series.
send is prototyped as:
#include <sys/socket.h>
ssize_t send(int socket_fd, const void *buffer, size_t buffer_size, int flags);
send returns the number of bytes actually sent, or -1 on error. The first argument
is the socket descriptor, which is the client's socket so send knows how
to get the data we want to send, to the client. The second is a constant
pointer to a void buffer area which is where the data is stored. The third argument is the size of the buffer. The fourth
is the flags, just like recv. We'll ignore the flags and use 0 for now.
Finally, in the source we use the bzero function. (You could also use
memset).
bzero sets the specified number of bytes to zero. It is not essential, but we
do it anyway. I think it's good programming practice not to take short
cuts, especially when you'd only save a line or two of typing anyway.
COMPILING AND RUNNING THE ECHO SERVER AND CLIENT
I will use the dollar sign ($) to represent the shell prompt.
$gcc -O -o echoserver1 echoserver1.c
$gcc -O -o echoclient1 echoclient1.c
Open a shell, cd to the directory where the programs are and run the server:
$./echoserver1
The shell will appear to freeze and
the program will say nothing, if all went well and none of the socket functions
failed.
Now open another shell on the same
desktop area, cd to the directory where the programs are and run the client:
$./echoclient1
If you did just that the program will terminate with a message:
Usage: ./echoclient1 <IP Address> <Message>
Run the program again (I'm assuming your local loopback address):
$./echoclient1 127.0.0.1 "Hello network world"
Now examine the screens of the client and server. The client will have to be run again for each message you want
to send to the server.
WRAPPING UP
In writing a network client, the core networking API functions you will use, in order, are
socket, connect, send/receive, and close.
In writing a network server, the core networking API functions you will use, in order, are
socket, bind, listen, (start loop),
accept, (optional fork or pthread create), send/receive, close.
Actually, when an application exits
or returns to the operating system the kernel will close any file descriptors that may have been left open,
including socket descriptors.
The use of close is
therefore optional. It is largely a matter of personal programming style.
A PREVIEW OF THE NEXT INSTALLMENT
The next article in this series will
examine conversion functions such as inet_pton, inet_ntop, ntohs, ntohl, htonl,
and htons,
plus other functions like gethostbyname, gethostname, getservbyname
and getservbyport,
among others. We'll also practice with some string-handling functions and write
a library of reusable network code that we will use throughout the remainder
of this series.
FINALLY...
If you are a beginner in network programming
please e-mail me and tell me
anything that could be improved, areas you found difficult, if any, and
suggestions you have for helping me to improve these articles.
Thanks!
Connect with Us