Wednesday, October 29, 2008

HOWTO: Get tun/tap compile for x64 OpenSolaris

I've been trying to figure out how to get tun/tap working on OpenSolaris for a while so that I can use OpenVPN. I finally put together all the different pieces to make it work. There may be some things here that aren't strictly required, but this is what finally got it working for me. I've included the main references that helped get it working below.

Firstly, I used the Sun Studio C compiler so you'll need to install that to use this process. On OpenSolaris, you can do this by executing "pkg install sunstudioexpress". The download is over 500MB, so it will take some time.

The next step is to download and extract the tun/tap driver sources. These are freely available from http://vtun.sourceforge.net/tun/. At this time, the current version is 1.1. They will extract into a directory named tun-1.1. You'll also need to download a new version of tun.c from http://openvpn.net/solaris/tun.c and replace the tun-1.1/solaris/tun.c file. The version that is distributed with the tun/tap sources compiles but doesn't work correctly.

After downloading and updating the sources, now we need to configure the build environment. This is where the majority of the work was in getting tun/tap to work correctly. Firstly, make sure that the Sun Studio compilers are in your path by executing "export PATH=/opt/sunstudioexpress/bin/:$PATH". Now cd into tun-1.1/solaris and execute "./configure". Unfortunately, the Makefile that is generated doesn't work correctly. I've included the entire contents of my working Makefile below. The changes I've made are to use the Sun Studio C compiler, use a different set of CFLAGS, add some LDFLAGS, and install the driver files into some additional locations.

References:
Fixing the "relocation error: R_AMD64_32" error
http://opensolaris.org/jive/thread.jspa?messageID=185654

Fixing the failed to attach error:
http://openvpn.net/archive/openvpn-users/2005-08/msg00002.html


# Generated automatically from Makefile.in by configure.
#
# Universal TUN/TAP device driver.
#
# Multithreaded STREAMS tun pseudo device driver.
#
# Copyright (C) 1999-2000 Maxim Krasnyansky
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# $Id: Makefile.in,v 1.7 2000/06/20 03:14:17 maxk Exp $
#
CONFIGURE_FILES = Makefile config.status config.cache config.h config.log

CC = cc
LD = ld

DEFS = -DTUN_VER=\"\"
CFLAGS = $(DEFS) -O2 -m64 -D_KERNEL -I. -xmodel=kernel -KPIC
LDFLAGS = -64
ADD_DRV = /usr/sbin/add_drv
REM_DRV = /usr/sbin/rem_drv
DRV_DIR = /kernel/drv/
DRV_DIR2 = /kernel/drv/amd64
DRV_DIR3 = /platform/i86pc/kernel/drv

INSTALL = /usr/bin/ginstall -c

all: module

module: tun.o
$(LD) $(LDFLAGS) -r -o tun tun.o

tun.o: tun.c if_tun.h
$(CC) $(CFLAGS) -c tun.c

inst: module
$(INSTALL) -m 644 -o root -g root if_tun.h /usr/include/net
$(INSTALL) -m 644 -o root -g root tun $(DRV_DIR)
$(INSTALL) -m 644 -o root -g root tun.conf $(DRV_DIR)
$(INSTALL) -m 644 -o root -g root tun $(DRV_DIR2)
$(INSTALL) -m 644 -o root -g root tun.conf $(DRV_DIR2)
$(INSTALL) -m 644 -o root -g root tun $(DRV_DIR3)
$(INSTALL) -m 644 -o root -g root tun.conf $(DRV_DIR3)
-$(REM_DRV) tun >/dev/null 2>&1
$(ADD_DRV) tun

clean:
rm -f tun *.o *~

distclean:
rm -f $(CONFIGURE_FILES)