Zones, Crossbow, and a virtual DMZ

One of the aspects of OpenSolaris that I've been most excited about has been Zones. Unfortunately, due to some complications in my home setup, I couldn't make the networking come out right. Since I first saw the description of what the Crossbow project has brought to the table, I had high hopes that it could do what I needed it to do.

My OpenSolaris server at home wears a number of different hats. Firstly, it's my firewall and performs NAT and filtering. It has two interfaces: ipbr0 connected to the outside, and bge0 connected to the inside. Secondly it's my private file server. ZFS was the biggest reason I migrated this machine from Linux to OpenSolaris, and it's worked wonderfully. Thirdly, I do some web development on the side so it serves as my test web server.

As you can see, my configuration is a little involved. Before Crossbow, I tried to build a zone for the web server, but I couldn't make it talk to both the inside and outside networks. I could make it talk to the inside network fine, but that keeps it from being publicly accessible. I could probably have made it talk to the outside if I paid my cable provider for a second IP, but then I couldn't use it inside. There might have been a way to make it work, but I never found it.

Before I go any further, I want to thank Ben Rockwood. His post about Crossbow at http://www.cuddletech.com/blog/pivot/entry.php?id=1001 was extremely helpful in figuring out how to get started with Crossbow.

The first step was to update my system to build 105. To do this, I needed to point my system to the dev repository by executing "pkg set-authority -O http://pkg.opensolaris.org/dev/ opensolaris.org". This took a minute or two to complete, but it ran without errors. After that, I did a "pkg image-update" to upgrade to the latest packages and rebooted. That took care of getting the bits required for Crossbow.

The next step was to create the virtual switch (etherstub) and network interfaces (vnic). Below are the commands that I used to create an etherstub named dmz1 and two virtual nics named dmz1_host1 and dmz1_web1. It appears that the last character of the vnic name must be a digit.

# dladm create-etherstub dmz1
# dladm create-vnic -l dmz1 dmz1_host1
# dladm create-vnic -l dmz1 dmz1_web1
# dladm show-link
LINK        CLASS    MTU    STATE    OVER
iprb0       phys     1500   up       --
bge0        phys     1500   up       --
dmz1        etherstub 9000  unknown  --
dmz1_host1  vnic     9000   up       dmz1
dmz1_web1   vnic     9000   up       dmz1

Once you've created the etherstub and the vnics, you need to configure them. First you need to plumb the and assign an IP to the host interface.

# ifconfig dmz1_host1 plumb
# ifconfig dmz1_host1 192.168.0.1 netmask 255.255.255.0
# ifconfig dmz1_host1 up

The next step is to create and install the zone. This is very similar to all the examples of zone configuration that I've found, but the key difference is setting the ip-type to exclusive. This is marked in bold below. Before beginning, I created a zfs filesytem at /data/zones. I've configured the zone so that it will share the /www filesystem from the host. If you don't need this, simply skip the lines from "add fs" through the next "end" statement.

# zonecfg -z web1
web1: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:web1> create
zonecfg:web1> set zonepath=/data/zones/web1
zonecfg:web1> set ip-type=exclusive
zonecfg:web1> add net
zonecfg:web1:net> set physical=dmz1_web1
zonecfg:web1:net> end
zonecfg:web1> add fs
zonecfg:web1:fs> set dir=/www
zonecfg:web1:fs> set special=/www
zonecfg:web1:fs> set type=lofs
zonecfg:web1:fs> end
zonecfg:web1> info
zonename: web1
zonepath: /data/zones/web1
brand: ipkg
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: exclusive
net:
address not specified
physical: dmz1_web1
defrouter not specified
zonecfg:web1> verify
zonecfg:web1> commit
zonecfg:web1> exit
# zoneadm -z web1 install
A ZFS file system has been created for this zone.
Authority: Using http://pkg.opensolaris.org/dev/.
Image: Preparing at /data/zones/web1/root ...  done.
Cache: Using /var/pkg/download.
Installing: (output follows)
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                  53/53   7972/7972   77.15/77.15

PHASE                                        ACTIONS
Install Phase                            12074/12074
PHASE                                          ITEMS
Reading Existing Index                           9/9
Indexing Packages                              53/53

Note: Man pages can be obtained by installing SUNWman
Postinstall: Copying SMF seed repository ... done.
Postinstall: Working around http://defect.opensolaris.org/bz/show_bug.cgi?id=741
Done: Installation completed in 90.608 seconds.

Next Steps: Boot the zone, then log into the zone console
     (zlogin -C) to complete the configuration process
# zoneadm -z web1 boot
# zlogin -C web1

Once you've connected to the console of the new zone using zlogin, you will be walked through some configuration options. This should be the same as a standard sysidconfig process. When asked for an IP address for the interface dmz1_web1, enter an IP the falls within the same subnet as the dmz1_host1 interface. Use the same subnet mask and use the IP address of dmz1_host1 as the gateway.

Since my server is also functioning as a firewall, I need to configure access rules. To start with, I'm simply allowing free access between my internal network and the dmz. Later, I'll configure specific access rules. For now, I added "pass in quick on dmz1_host1 keep state" and "pass in quick on dmz1_host1 keep state" to /etc/ipf/ipf.conf. To activate these rules, I ran "ipf -Fa -f /etc/ipf/ipf.conf". You may also need to adjust your NAT rules to allow access to the outside world.

If everything has gone according to plan, you should be able to log into your new zone and access both the Internet and your internal network. Next, install and enable apache.

# pkg install SUNWapch22
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    4/4   1342/1342     6.07/6.07

PHASE                                        ACTIONS
Install Phase                              1656/1656
PHASE                                          ITEMS
Reading Existing Index                           9/9
Indexing Packages                                1/4
Indexing Packages                                4/4
# svcadm enable http:apache22
#

Now try to access the new zone from a system on your internal network by pointing your browser to http://192.168.0.2.

I hope that this post is able to help people get started with Zones and Crossbow. These are very flexible technologies that work very nicely together.

Reference:
http://www.cuddletech.com/blog/pivot/entry.php?id=1001
http://www.sun.com/software/solaris/howtoguides/containersLowRes.jsp
http://blog.thilelli.net/post/2006/10/05/Zones-and-ZFS-Integration-and-New-Features-in-OpenSolaris