I started here...

-> yum info bind
Loading "installonlyn" plugin
Loading "priorities" plugin
Setting up repositories
Reading repository metadata in from local files
0 packages excluded due to repository priority protections

Available Packages
Name : bind
Arch : i386
Epoch : 30
Version: 9.3.3
Release: 10.el5
Size : 954 k
Repo : base
Summary: The Berkeley Internet Name Domain (BIND) DNS (Domain Name System) server.
Description:
BIND (Berkeley Internet Name Domain) is an implementation of the DNS
(Domain Name System) protocols. BIND includes a DNS server (named),
which resolves host names to IP addresses; a resolver library
(routines for applications to use when interfacing with DNS); and
tools for verifying that the DNS server is operating properly.

-> ll /var/named
ls: /var/named: No such file or directory

> ll /etc/name*
ls: /etc/name*: No such file or directory

As you can see I have no bind packages installed. Now I will install bind as shown here:

-> yum -y install bind
Loading "installonlyn" plugin
Loading "priorities" plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
//cut here to make thread readable
Installed: bind.i386 30:9.3.3-10.el5
Complete!

-> /etc/init.d/named start
Locating //etc/named.conf failed:
[FAILED]
-> ll /etc/named.conf
ls: /etc/named.conf: No such file or directory

Obviously the configuration missing is the cause of my problem, but why did the package not include any default working config like it once did?

Just looking to get a DNS proxy on this box. I would like to configure it so that queries are only responded to the IP of the server and 127.0.0.1 - is this possible (excluding using a firewall)?

you can look in this file for example
/usr/share/doc/bind-9.3.3/sample/etc/named.conf
and on this manual
http://www.centos.org/docs/5/html/5.1/Deployment_Guide/ch-bind.html

This is a FAQ. Please take a look at FAQ #3 here:

http://wiki.centos.org/FAQ/CentOS5

So I did these commands as this page said I could do...

\cp -Rf /usr/share/doc/bind*/sample/etc /etc
\cp -Rf /usr/share/doc/bind*/sample/var/named /var/named
/etc/init.d/named start

-> /etc/init.d/named start
Starting named: [FAILED]

But now it won't start, still. I'm at least getting errors in /var/log/messages

Feb 13 22:48:18 essex named[4414]: starting BIND 9.3.3rc2 -u named
Feb 13 22:48:18 essex named[4414]: found 1 CPU, using 1 worker thread
Feb 13 22:48:18 essex named[4414]: loading configuration from '/etc/named.conf'
Feb 13 22:48:18 essex named[4414]: listening on IPv4 interface lo, 127.0.0.1#53
Feb 13 22:48:18 essex named[4414]: listening on IPv4 interface eth0, 74.218.88.169#53
Feb 13 22:48:18 essex named[4414]: /etc/named.conf:101: configuring key 'ddns_key': bad base64 encoding
Feb 13 22:48:18 essex named[4414]: loading configuration: bad base64 encoding
Feb 13 22:48:18 essex named[4414]: exiting (due to fatal error)

/etc/named.conf:
100 key ddns_key
101 {
102 algorithm hmac-md5;
103 secret "use /usr/sbin/dns-keygen to generate TSIG keys";
104 };
105 view "external"
106 {

To fix the problem I posted above you simply execute:

/usr/sbin/dns-keygen

then copy the output of that command into the double quotes on line 103