HOWTO: Get tun/tap compile for x64 OpenSolaris

I've been trying to figure out how to get tun/tap working on OpenSolaris for a while so that I can use OpenVPN. I finally put together all the different pieces to make it work. There may be some things here that aren't strictly required, but this is what finally got it working for me. I've included the main references that helped get it working below.

Firstly, I used the Sun Studio C compiler so you'll need to install that to use this process. On OpenSolaris, you can do this by executing "pkg install sunstudioexpress". The download is over 500MB, so it will take some time.

The next step is to download and extract the tun/tap driver sources. These are freely available from http://vtun.sourceforge.net/tun/. At this time, the current version is 1.1. They will extract into a directory named tun-1.1. You'll also need to download a new version of tun.c from http://openvpn.net/solaris/tun.c and replace the tun-1.1/solaris/tun.c file. The version that is distributed with the tun/tap sources compiles but doesn't work correctly.

After downloading and updating the sources, now we need to configure the build environment. This is where the majority of the work was in getting tun/tap to work correctly. Firstly, make sure that the Sun Studio compilers are in your path by executing "export PATH=/opt/sunstudioexpress/bin/:$PATH". Now cd into tun-1.1/solaris and execute "./configure". Unfortunately, the Makefile that is generated doesn't work correctly. I've included the entire contents of my working Makefile below. The changes I've made are to use the Sun Studio C compiler, use a different set of CFLAGS, add some LDFLAGS, and install the driver files into some additional locations.

References:
Fixing the "relocation error: R_AMD64_32" error
http://opensolaris.org/jive/thread.jspa?messageID=185654

Fixing the failed to attach error:
http://openvpn.net/archive/openvpn-users/2005-08/msg00002.html

# Generated automatically from Makefile.in by configure.
#
#  Universal TUN/TAP device driver.
#
#  Multithreaded STREAMS tun pseudo device driver.
#
#  Copyright (C) 1999-2000 Maxim Krasnyansky 
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#  GNU General Public License for more details.
#
#  $Id: Makefile.in,v 1.7 2000/06/20 03:14:17 maxk Exp $
#
CONFIGURE_FILES = Makefile config.status config.cache config.h config.log

CC = cc
LD = ld

DEFS =  -DTUN_VER=\"\"
CFLAGS = $(DEFS) -O2 -m64 -D_KERNEL -I. -xmodel=kernel -KPIC
LDFLAGS = -64
ADD_DRV = /usr/sbin/add_drv
REM_DRV = /usr/sbin/rem_drv
DRV_DIR = /kernel/drv/
DRV_DIR2 = /kernel/drv/amd64
DRV_DIR3 = /platform/i86pc/kernel/drv

INSTALL = /usr/bin/ginstall -c

all: module

module: tun.o
       $(LD) $(LDFLAGS) -r -o tun tun.o

tun.o: tun.c if_tun.h
       $(CC) $(CFLAGS) -c tun.c

inst: module
       $(INSTALL) -m 644 -o root -g root if_tun.h /usr/include/net
       $(INSTALL) -m 644 -o root -g root tun $(DRV_DIR)
       $(INSTALL) -m 644 -o root -g root tun.conf $(DRV_DIR)
       $(INSTALL) -m 644 -o root -g root tun $(DRV_DIR2)
       $(INSTALL) -m 644 -o root -g root tun.conf $(DRV_DIR2)
       $(INSTALL) -m 644 -o root -g root tun $(DRV_DIR3)
       $(INSTALL) -m 644 -o root -g root tun.conf $(DRV_DIR3)
       -$(REM_DRV) tun >/dev/null 2>&1
       $(ADD_DRV) tun 

clean:
       rm -f tun *.o *~

distclean:
       rm -f $(CONFIGURE_FILES)

Is Sun Solaris on its deathbed?

As a long-time Linux user and a new Solaris user, I found the article “Is Sun Solaris on it's deathbed” to be misguided. Firstly, I think that there definitely are some people who are taking a fresh look at Solaris and especially OpenSolaris. In my case, my exposure to an extremely diverse Linux environment has shown me the appeal of the consistency and compatibility that Solaris offers. Even the differences between Solaris 10 and OpenSolaris, which seem to bother many Solaris admins, seem minor compared to the differences between Linux distributions.

I think there are a few factors related to finding experienced Linux administrators versus Solaris administrators. Most people agree that the best way to learn something is to use it in production. Linux has really only been accepted in the enterprise for about 5 years, which is pretty short in comparison to Solaris. An admin who has worked with Linux for 5 years can be considered experienced. On the other hand, an admin who has worked with Solaris for 5 years is a relative newcomer.

Solaris is known for reliability, and the deployments where it is used have tended to reflect that. This has led the companies who deploy it to demand experienced admins. As stated above, being an experienced Solaris admin is a much higher bar than with Linux. This leads those companies compete for the best of a limited supply of experienced administrators. It also makes it a very difficult specialty to break into.

To address these issues, I think Sun needs to do more to showcase the benefits of Solaris as well as to encourage more people to learn the system. Either one of these alone isn't going to address the problem. By doing both, you create both more demand for the system as well as more people qualified to administrate it. This is one area where Linux has been remarkably effective.

I think the demand for DTrace and other advanced tools relates to the kind of environments where a system is used. It used to be that applications scaled vertically. It was expensive to add capacity because at some point the whole system needed to be replaced. Administrators were required to learn how to ensure maximum reliability and to wring every last drop of performance out of the system due to the expense of an upgrade. This demanded advanced tools like DTrace.

Today, more and more applications are scaled horizontally. Rather than spend time pouring over the internals of the system followed by extensive tuning to improve performance by 10%, it's easier to just add another server and ignore the problem. Because of this mindset, Linux hasn't required the kind of tools that Solaris brings to the table. I think that efficiency efforts may be changing this mindset, because people are realizing that the cost of a server over it's lifetime is much more than the initial purchase price.

What can be done about this? I think that Sun needs to do a better job of showing what these tools bring to the table. They should also show that most applications deployed on Linux can be deployed on Solaris with minimal effort and that the tools Solaris offers will let companies get more out of their hardware investment.

I also think that Sun should clarify the relationship between OpenSolaris and Solaris. If I were to start a business, in the beginning I'm more likely to use OpenSolaris on commodity hardware and find my own solutions to any problems that arise. As the business grew, I'd be more likely to be interested in hardware optimized for Solaris and long-term support. I think clarifying the road map could make companies considering OpenSolaris more comfortable in that decision.

The final thing that I think Sun should do is to calm people's fears about using Solaris. Many startups have minimal money to spend, and so they don't want to start with one platform only to run into a wall that requires them to shell out a ton of money. People know that Linux is Free, and that if they want to support themselves they can use it without paying for it and that nobody will come in and take it away. I think more small businesses would use Solaris and OpenSolaris if Sun can show that they are safe choices.

A work re-work: a rough draft

So what am I doing with this? So far, I've purchased an electronic organizer. I've opted for an Apple iPod Touch, which is very slim but has a nice calendar and supports third-party applications. The calendar application supports multiple calendars, so I can have both my work and personal calendars available, but not mixed together.

I've also purchased Things through the iTunes App Store from Cultured Code. It is a GTD-oriented task management application, and so far it seems to be working well. I've started to input all the projects that I have in progress at the moment. I'm also recording any task that I think of as soon as I'm able. The list is depressingly long and growing fast, but at the same time it's also a relief to have one place to look for work to get done. I haven't been using it long, but already it's a relief to know that I have a record of stuff that I need to get done.

I'm also going to try an experiment at the office this week. I'm going to start closing Outlook. I have my calendar available on my iPod Touch, so I don't need to worry about missing meetings. I'll try to check my mail every hour or two, and process everything then. I plan to notify the people who I deal with most often, but everybody else will just have to wait. If it's urgent, they'll be calling anyway.

A work re-work: time management

I've tried various organization techniques, such as blocking out time for functional areas of my job, trying to pre-schedule my weekly activities, etc. Unfortunately, those haven't really helped. For example, I find that when my calendar tells me that it's time to work on storage I don't know what I should be doing. I know there are things that need to be done, I don't know what I can do at the current time to actually move things forward.

That leads me to today. In an attempt to turn things around, I'm attempting to become better at managing my time. If I don't succeed, I think I'm going to suffer from major burnout. I've already had weeks where I just don't want to do anything at all.

My company has various online training courses, and I started with an online time management course. Unfortunately, it was targeted for people in a more traditional office job where they may have 5-10 major tasks to accomplish in a week, with maybe 20% change based on shifting requirements. Unfortunately, I've found IT to be quite different, with 50-80 smaller tasks to accomplish in a week, with 50-80% change based on shifting requirements.

I'm currently working my way through Time Management for System Administrators by Thomas Limoncelli. While I'm only about half way through the book, there are some valuable tips here. One of the most useful, I expect, is the Mutual Interruption Shield. If you work with other people, set aside times when you are not to be interrupted except in an emergency. The other person fields all questions and requests, which allows you to focus completely on the task at hand. Then you switch places. This gives both of you real focus time. He also advocates establishing routines. This allows you to get the routine work done without having to think about it. The author also provides a simple system for keeping track of tasks that need to get done, and when they're due.

I've also started reading Getting Things Done by David Allen. I'm not very far in, but this book seems a bit more philosophical. The organizational system that the author advocates is definitely more complex, but it may also be more robust. One of the core tenants of this system, as well as Time Management for System Administrators, is to get the lists of things to do out of your head and into some kind of trusted organizer. This can be paper or digital, just so long as it can be counted on.

A work re-work: background

I've been in my current job for nearly three years, and I've been very productive from early on. When I first started, I was able to work almost entirely on technical projects. This was largely because I was new and there were very few other demands placed on me. I established some good habits such as logging my work daily in OneNote, tracking tasks that needed to be done, etc. I became “the organized one,” a title I never thought would be applied to me.

For the past six months or eight months, I've felt like my productivity has been slipping. Some weeks I look back on Friday and wonder what I've accomplished. I'm spending more and more time communicating about work, and less time doing it. What good habits I developed have started to slip. I will miss days or weeks of work logs, and my task list is getting stale.

Unfortunately, at the same time the amount of work that I'm expected to complete has expanded during this same period. So now I have more to do, but I'm getting less done. I would guess that these are related. As I've become more busy, I've let my productivity tools slide. That has put me into a vicious cycle of getting more and more busy with less and less support from my tools. It hasn't reached crisis stage yet, but I know that there is a problem coming if I don't change something.

This cycle has left me feeling thoroughly unfulfilled at work. I've read both “Now, Discover Your Strengths” by Marcus Buckingham and Donald O. Clifton, and “Go Put Your Strengths to Work” by Marcus Buckingham. These have helped me to get a better idea of what I'm good at and where I need to be. Unfortunately, I'm still scrambling to try to get my current work done. This leaves me precious little time to try to leverage myself into a position where I play more to my strengths.

OpenSolaris power management

One aspect of OpenSolaris that I've taken an interest in is power management. In my research, I've come across several useful things.

One of the first things I tried to find was the current CPU speed to see if power management was doing anything. I finally stumbled across this post by Mark Haywood. In that post he shows these commands:

$ kstat -m cpu_info -s supported_frequencies_Hz
module: cpu_info                        instance: 0
name:   cpu_info0                       class:    misc
       supported_frequencies_Hz        2800000000:3200000000

$ kstat -m cpu_info -s current_clock_Hz
module: cpu_info                        instance: 0
name:   cpu_info0                       class:    misc
       current_clock_Hz                2800000000

These report the supported frequencies and the current frequency of the CPU. If the supported frequencies field shows a range as indicated above, OpenSolaris supports power management for your CPU.

Another very handy tool is PowerTop. It captures statistics about how much CPU time is idle, how much time it spends running at different speeds, and what is causing the CPU to wake up from idle most frequently.

OpenSolaris 2008.05 installation and the fault manager

Last week I had to install an OpenSolaris 2008.05 system at work. The first system I tried to use wouldn't boot the kernel. I did some searching, and found that adding -kv to the kernel command line in grub enables more verbose output during boot. Adding -d enables the kernel debugger. The system was hanging so early in the boot process that I wasn't able to diagnose very much. BIOS and firmware updates didn't seem to help either.

After that, I moved to another system. That was where I had my first encounter with the Solaris Fault Manager system. During a reboot after an image update, it showed an error on a PCI device but continued working fine. After the next reboot, there was a message on the console about a device being retired and the system did not come up on the network. A look at dmesg showed that the e1000g0 devices had been unregistered. I did some searching, and came across this Sun Documentation page that describes the faulty device retirement feature. Basically, you can use the fmadm faulty command to list devices that have show failures. In my case, I issued the fmadm repair command with the fault ID. After another reboot, the system came back up with the e1000g0 device working properly.