Hudson Continuous Integration Installed
I just got an installation of Hudson continuous integration running on my (older) Tomcat server, and I thought I'd knock out a few notes to both remind me what to do in the future, and possibly help someone else who's hit the same problems I've hit.
Installing the software itself was pretty easy. I simply hit the link and downloaded the latest WAR file. I then used the Tomcat manager to deploy the WAR file. Tomcat churned for a little while, and then the app was started. Easy enough.
I run my Tomcat behind an Apache server with a number of virtual hosts. I wanted this one exposed on a particular host, so I modified the Apache config files to JKMount the desired path on the desired server. A quick restart and that worked, too. I checked the other virtual hosts, and they did not also map that path, as was expected.
A quick aside (that took me several hours to fix) was that I noticed my Apache was running as "nobody" and my Tomcat was (egads!) running as "root." I made a new httpd user and group, changed the Apache config file and directory permissions, and restarted without a hiccup. I tried doing the same quickly with Tomcat, but it has no such launcher, so I added an "su - httpd" to the start up script, changed those file permissions, and went through a few cycles of trying to trying to get the scripts to stop complaining. I've configured my /usr/bin/false script to text my cellphone whenever a user configured with this shell tries to login. Evidently, the Tomcat startup scripts are executing as login shells, so I kept getting the notices. I spent way too much time trying to fix this, so I eventually decided that since it was indeed running, I could live with the text messages and work on finding the solution ('cause that's not the shell that this user is using...what else is Tomcat hitting?) later.
One of the reasons I wanted to use Hudson over the old favorite Cruise Control is that there's no security in Cruise Control, but there is in Hudson. That and the quick WAR deployment.
The first gotcha came in the form of a bit of chicken-and-egg configuration. When I turned on security and saved the configuration, Hudson prompted me to login. Curiously, I'd not created a user, nor was I given an opportunity to do so. To make things worse, I'd turned off the "users can create accounts" option. I found a quick article on the Internet that pointed me to the XML configuration item to change. I tried that, and created an account. It, however, wouldn't let me do anything because I had no permissions. Another quick Internet search and I divined that I should have added that user to the Hudson configuration before getting started. I blew away the Hudson config file, restarted the app in the Tomcat manager, and tried again. Third time is the charm; I created the username with full permissions as I configured the app, leaving the "create" option on. I created a user with the same name using the link on the login page. Upon logging in, I was rewarded once again with the Hudson interface without warnings.
Then I got into creating my project. Here's where I think my style gets a little ugly with the Hudson project management. I prefer separate projects for my main source and my test source. This helps me keep the JUnit classes, test data and properties, and test classes and dependencies out of the main source tree. How many times has it happened that a class will get accidentally created in the source tree destined for production, or a class will accidentally use a JAR or properties file intended for test only, because Eclipse can't separate the build paths. Ant does a swell job, and since NetBeans uses Ant, it also does a swell job, but this is one of the shortcomings I keep meaning to fix for Eclipse...
Anyway, it was swell that Hudson allowed me to get both projects from my Subversion server. It conveniently put them in separate, but neighboring directories, which matches my style (although I can use Ant properties to give different paths if necessary...I'm slick like that).
My first ten builds all failed due to some bit of misunderstanding in the Hudson config, my Ant file, or something in the middle. The project builds successfully in the IDE (I use Eclipse), using both the IDE tools and the Ant scripts in each project's root. I figured I'd be ready to go.
For example, apparently, Hudson doesn't contain Ant (Cruise Control does). I did find in the configuration a place where I could name an Ant path. The next few builds failed because while I had only one Ant configured, it wasn't selected as the default, and needed to be explicitly chosen in the project config.
I had to tweak settings between a couple of other build attempts to get the path to Tomcat set right in the Ant properties (5.5 on the server, 6.0 on my dev box...upgrade needed, yes; in 6.0 it's TOMCAT_HOME/lib, while on 5.5 it's TOMCAT_HOME/common/lib). And then to get the JUnit target to run before the WAR build target (spaces, not commas in the target lines); the default target just builds, but doesn't test or create a WAR.
Then a try or two to get the JUnit test reports to show. I was a little concerned because I hadn't specified which project's build.xml to use (the one calls the other, so I only need one), but using the right Ant-style wildcard to the folder created and used by Ant (**/ant-build/junit-reports/*.xml worked).
I believe, but haven't verified by any kind of testing, that I can put the project folder in the path name for the build file, archives, and the JUnit reports. Right now I'm just throwing the build to the wind (finds it and works), and using the wildcards for the archives and reports.
Finally, a build succeeded! However, I was unsatisfied with the 10 failures before the first success, so I deleted the project and started over. Since I didn't have to configure Hudson (declare Ant, etc), and I'd made a few notes, the configuration was put together right the first time, and its first build was successful.
After scoping out the various bits on the Hudson, I found that I could see the console output (as with Cruise Control), and noticed that I see the "unauthorized account" messages I was getting when tweaking my Tomcat user. This is certainly my server configuration and not something with Hudson. Again, firing up Ant works, so I'm guessing it's something to do with something running as the httpd user. In my next pass, I plan to upgrade the Solaris server to v10, and put my Tomcat in a zone of its own, or other-how jail the user. Right now the server's running v9, which didn't have zones, and I've not made the effort to do some kind of chroot for the servers. I've probably just been lucky that no one has taken advantage of my obscurity and challenged my security.
I carefully perused the project settings looking for a place to have the EMMA output tracked. I use EMMA for code coverage, as we all should (or something similar). I found there is an EMMA plug-in. The Hudson wiki docs show how to configure Maven, but I've already got my Ant configured. I didn't have the output in XML form, though, so I added that to my build, saved my project configuration, and waited for the first detected and automatic build.
The build succeeded, but is now off by one; there's a JUnit history with two 100%, and a code-coverage history with one mark, but no lines. I guess as I make more changes the project will fill this out. I'm happy with the successful build and integration of the EMMA with the Hudson dashboard.
So now my first install of Hudson is complete, with my first project added to it. The tiny hiccups with the config and users were frustrating, but pretty small. As with so many projects, the documentation has gaps where the writers either make assumptions about "everyone knowing that," or they don't have the time to fill all of the holes.
For future reference, here's a quick Hudson initial configuration checklist, well, for getting it working on Tomcat. Probably similar for other containers.
- Have Ant (or Maven, or whatever) on the system. Know where it is, and be sure the Tomcat can execute it.
- Have the Tomcat secured as not the super-user. This is just a good idea, especially if Ant is being used as Ant can execute arbitrary commands, and doing this as root can cause much undesired havoc.
- Deploy using the Manager. It's very easy, unless the Manager isn't running; then just unpack the WAR file. Configure Apache, if in front, correctly. Check the Hudson docs if you don't want its workspace folder to be in the default (user's home) folder. This is very server-specific, and really has little to do with Hudson, other than getting to it.
- When configuring the server settings, if you're going to use Hudson user security, create the user permissions with the user (I haven't tried groups, or maybe it is a group...) first, then create the user, before disabling user self-creation
- Configure the JDK. I'm not sure if this is necessary, or if Hudson will just use the JDK it's running with inside of Tomcat. I didn't see any place in the project configurations to select a JDK, either because I've only defined one, or because it's just using it.
- Configure the Ant for use in the projects. It's really just a matter of providing a path. If Maven is used instead, there are options for that, too. Otherwise,
- Make a project with as many of the settings as possible completed in advance. Once saved, it will try to run (when next scheduled) and failures may mess up your mojo if configuration is incomplete.
- While configuring the project, be sure to configure the polling, SCM (source code management), build steps (script, Ant, Maven, whatever), and users as complete as possible. This is why continuous integration packages are used, after all.
I would like to spend a little more time with my wildcard paths. It seems I should be able to say "project/builddir/etc/file.xml" instead. Only because I break up the production and test projects, and I think it might remove some ambiguity. Same for the build.xml; I can probably specify "project1/build.xml" to be sure that that's the one that gets executed.
I'd like also to understand better the ties between users and e-mail addresses. From what I saw in my Internet searching, it seems that Hudson will create users based on check-out information, but what happens if the projects are diverse and have overlapping user names? I'm not too worried about that just yet as I'm just starting this project that I've added first, but there are other projects I'd like to add and they come from different sources. There doesn't seem to be a concept of aliases in Hudson, although I've not looked too deeply at the plug-ins. It also seems to be the case that an account can only have one e-mail address. There are other options than the "Hudson built-in" security, but I didn't want to play with all of that just yet.
I'm also curious about how to make this work in the virtual servers. I can do some trickery with Tomcat virtual hosting and provide separate managers for each domain, but that results in multiple copies of the app installed and running, which I'd like to avoid.
There's also one nagging message on the Hudson configuration page telling me that my container isn't enforcing UTF-8, but following the suggestion in the documentation didn't help. It's only on the configuration page, so I'll let that one slide, too. The gripe it gives is that project names with other character sets will cause problems; I say that Hudson should either URL encode them, avoid using the project names in URLs, or otherwise enforce UTF-8 itself.
It is an open-source project, so perhaps I'll look into these and see if I can make fixes myself. I probably won't as I've got plenty of other work to finish, and I didn't correct any of the Cruise Control bits I didn't like (no security at alll, nor virtual host support either).
That's the meat of it, methinks. A good start, anyway. Now that I've got my CI set up, I can start digging into the work of working on the projects. I'll get a few more projects added, and a few more users on the system and see how it goes.
HI
looks very interesting!
bookmarked your blog.
john brightman