I have a dream that one day I will not need to have a dedicated machine to do software development. That I will be free from the shackles of physical hardware and do all my development in the cloud.
I work as a consultant, so I change places quite often. It should not matter whether I’m at work, at home or at a client site. When virtualization first came along, I was pleased. My prayers were heard. In practice, the virtual machines required serious physical hardware to run at any decent speed and that still chained me to a big machine.
Then came the cloud. Hmm, not so much different from running my own machine… on the surface. In practice, it gives me a lot of freedom. I just rent my powerful machine instead of buying it.
I work with java, grails and .NET and for my purposes I use Windows 7 64bit most of the time. Sure, Linux is great too, but not for .NET work. Many of the cloud hosts do not support running windows so those are out of the question. I need a very responsive system, so I can’t feel the difference between running a remote desktop on my LAN or on the cloud.
That sets a very high requirement for the 4 big factors of cloud performance:
Network latency, Disk IOPS, RAM and CPU
Network latency is the most important. I need the host to register my mouse movement and my typing quickly. Cable quality, router speed and distance all matter. For my purposes I need to run it really close to home. I live in Copenhagen, Denmark and the network latency to Rackspace and Amazon is too great. Even their european servers are too far away in ping-time. I’ve tried host with 50ms ping time or less, and it’s still too much. I tried a great cloudhost called Cloudsigma in Zürich, but even though their performance was great, their latency was not good enough for me.
Up until now there was nothing to do. Host a server at home maybe…? Nah, I don’t want to run a powerful server at home. Not because I can’t, but because it’ll be a dinosaur in two years anyway. And what if i need 10 machines for a projekt? Or another 32 GB of ram? Nope, not for me. Cloud it is.
Cloud.dk is a new danish provider of cloud computers. Their datacenter is around 3 ms away from me, and their disk/ram/cpu are pretty good too. I’ll show some benchmarks later.
Here’s a little how-to of how to get a Windows 7 running on their infrastructure. I’ll assume that you have an account and know how to fire up a virtual machine from one of their existing images.
Download the ISO image of your operating system. I used my MSDN account to download Windows 7 Enterprise with SP1 in the 64 bit version
Make the ISO available via HTTP. In my case i uploaded it to an S3 bucket and made it public.
Go into the cloud.dk control panel and go to “images” and then “fetch image” http://i.imgur.com/NkhRk.jpg
Instruct cloud.dk to fecth the image from your httpserver. Take a look at the following image http://i.imgur.com/Y8LGG.jpg. Be sure to change the URL and be sure to add a default user.
That will start a job that fetches the image. Be sure to have a fast HTTP server, as the job will time out if it runs too long. You can go to the jobs tab to see the status. http://i.imgur.com/YHytl.jpg
When the download is complete, you’ll have a new image in the list: http://i.imgur.com/Z8BsX.jpg. If you don’t see this, your download has failed for some reason.
Make a new machine. Use the new image as the primary disk and add another 50GB or 100 GB disk if you need the extra space. I recommend at least 4 GB/ 2 vCPU if you plan on running Visual Studio 2010. http://i.imgur.com/jRpR0.jpg
Switch the primary and secondary disks: http://i.imgur.com/ht0gQ.jpg. Think of this as putting the harddisk as the primary and the uploaded iso image as the DVD-ROM. IF YOU FORGET THIS STEP, THE INSTALL WILL FAIL as it will be unable to partition correctly.
Start the server: http://i.imgur.com/6A921.jpg. Let the machine run for a couple of minutes and then start the console: http://i.imgur.com/oziFm.jpg
If you have java installed, it’ll start a program (webstart) that acts as a vnc console. You should see the windows install screen like this: http://i.imgur.com/p9Lrp.jpg
Install windows as you normally would.
Enable Remote Desktop in windows 7
Close the console
Login with remote desktop
Your Win 7 should be ready for you to setup as you like.
My install (from boot to remote desktop login) took around 12 minutes! That’s fast! And that time even included waiting for the human (me) to react
Shutdown the machine. If you logged on with remote desktop you can shutdown with this command:
shutdown -s -f -t 3
After that, you wait for the VM to be completely off. When it’s off, remove the ISO image from the server: http://i.imgur.com/8W6KM.jpg
After that, delete the disk image. http://i.imgur.com/cn77v.jpg. It’s a copy of the uploaded image anyway, so no need to waste cloud storage on it. You can still make new machines with the uploaded image.
That’s it. No more required steps. I do have a few suggestions for activities though:
- Get all windows updates
- Set up the cloud.dk firewall. RDP uses port 3389
- Enable power management and set to hibernate after 1 hour without use. Execute the command “powercfg -H ON” in an elevated prompt to enable hibernation feature of of Win7 if it’s not enabled already.
- Figure out how to start and stop your machine from the commandline API
- Disable the pesky UAC feature.
Oh, I almost forgot I promised some benchmarks. Remember this is the 4 GB / 2 vCPU machine configuration.
- Geekbench: 4580 – A decent score, as expected.
- Crystal Disk Mark 4K QD32: 43.77 MB/s read and 11.84 MB/s write. An impressive score for a cloud machine. They’re using a TSM system, where their SAN has a mix og ssd’s, harddrives, large ram caches and migrates automatically. Seems to work out nicely for them
- Speedtest.net: Ping 5ms, download 186.64 Mbps, uploda 180.86 Mbps
- Pingtest.net: Packet loss 0%, PIng 4 ms, jitter 1 ms.
- Details here: http://i.imgur.com/qb9Jr.jpg
So: in summary, a decent RAM/CPU score, a good network score and a disk score that’s similar to a low end SSD. That’s just numbers, and good ones too, but…
What’s my “feel” for it? Well, it feels very snappy and fast. Complete windows install in 12 minutes. Windows updates download and install quickly. Any sort of manipulation of the user interface feels almost as if it was a local machine. Everything loads quickly and programs start up fast. It feels almost like it’s a SSD
Enjoy your new workstation in the cloud!
I hate next-next-next installers. Up until now, I had to endure the horrors of that, while installing the Java JDK on windows.
Thank to my colleague Erik, I don’t have to do that anymore. With his help, I discovered how to do a silent command line install of the entire JDK:
UPDATE: this article turned obsolete. Instead I’ll provide the link to the “Installation notes” for the windows JDK 7:
I had succes with this command line:
jdk-7u2-windows-i586.exe /s ADDLOCAL="ToolsFeature,SourceFeature" INSTALLDIR="%CD%\jdk7u2"
Just a quick post about something i had to to do today. It’s all in the Apache docs, but bits of it was hard to find. I was working on a redhat 4 64-bit
Grab the source from http://httpd.apache.org/download.cgi
I picked the Unix Source: httpd-2.2.17.tar.gz
Unpack with tar xzf httpd-2.2.17.tar.gz
./configure --prefix /path/to/apache --enable-mods-shared=all --enable-proxy --enable-ssl
If you leave out "--prefix ..." then the installation directory is set to /usr/local/apache2
Does all the compiling
Installs into the directory that you wrote in --prefix
Edit the /conf/httpd.conf if you want to set a another port or change some other setting
Controlling the Apache server:
- /bin/apachectl -k start
- /bin/apachectl -k stop
- /bin/apachectl -k restart
- /bin/apachectl -k graceful-stop
I am now a ScrumMaster with certificates and everything.
We did a two day ScrumMaster certification training course with our entire company (around 40 or 50 people). It was a very good course and our teachers were both knowledgeable, experienced and very inspiring. I can recommend them if you’in the market for some training.
Our trainers were by Gertrud Bjørnvig and Jim Coplien and their website is at: https://sites.google.com/a/gertrudandcope.com/www/
And here’s my proof and pretty certificate
All in all a very good experience, and I feel much more enthusiastic about Scrum than I did before.
I’ve set up my grails build on Hudson, and now I want to deploy to my Tomcat server. Every time my tests complete, I want a new deploy. And I want to be completely automatic too.
First step is to navigate to your grails app and try:
grails install-plugin tomcat
It’s probably already installed, as it’s the default for grails these days.
Then we need to install Tomcat if you don’t have it already.
First, make sure it’s the Sun JDK, argh, I mean Oracle, but the Ubuntu packages still say Sun. The OpenJDK doesn’t really work well with grails and tomcat.
sudo aptitude install sun-java6-jdk
Then install tomcat:
sudo aptitude install tomcat6
Then edit the tomcat user file to enable some users:
sudo nano /etc/tomcat6/tomcat-users.xml
I suggest you have the following lines active, but please change the passwords:
<user username=”manager” password=”manager” roles=”standard,manager”/>
<user username=”grails” password=”grails” roles=”standard,manager”/>
Restart tomcat to activate the changes.
sudo /etc/init.d/tomcat6 restart
Test by going to:
You should now specify the username/password and url for the to) in your grails project. Open the grails-app/conf/Config.groovy file and add this at the bottom of the file:
That should set up easy deployment directly from grails. We’ll simply be using grails’ own plugin for deployment, so we’ll have no real need to understand how ant or tomcat handles it. It’s all nicely camouflaged by grails.
The deployment sequence is like this: First you need to undeploy the app from tomcat (not needed the first time)
grails tomcat undeploy
And then deploy the app
grails tomcat deploy
You should now be able to use your app on your testserver.
To add them to your Hudson project just add those two lines as a post-build shell script (or batch command if you’re on windows) and you’re good to go.
If undeployment fails, there may be some file locks. I experienced that on windows, but not on the Ubuntu 10.10 server. If (and only if) you have this problem, you can edit the
sudo nano /etc/tomcat6/context.xml
And change the <context> line to:
<Context antiJARLocking=”true” antiResourceLocking=”true” >
That fixed the issue for me.
PS: To control tomcat use these commands:
sudo /etc/init.d/tomcat6 stop
sudo /etc/init.d/tomcat6 start
sudo /etc/init.d/tomcat6 restart
Before we set up the project, we need to install some plugins. Access your hudson admin interface, probably at http://<hostname>:9090
Go to Hudson -> Manage Hudson -> Manage plugins -> Available (it’s a tab) and get these required plugins:
Hudson Mercurial plugin or get the one for git/svn/cvs depending on what you use. I’m a mercurial fan.
If you like, you can also install some of these, but they’re not required:
After that you need start a new project by going to the link called Hudson -> New project
Choose a job name, and the Build a free-style software project. After you click OK, you get a lot of options. Only some of them are needed:
Discard Old Builds -> Days to keep: 3, Max # of builds to keep: 10
In the Source Code Management part you select Mercurial and then enter the Repository URL. For a private bitbucket it’s like this:
Make sure to include the username:password if it’s a private project. For a public one, it’s not needed.
In the Build Triggers section, choose Poll SCM and enter this into the schedule field (it’s std cron notation):
# every minute
* * * * *
In the Build section, choose Add Build Step and Build With Grails. You’ll se a lot of grails related fields but you only need to enter the following into the Targets field:
Eventually, you’ll probably want to run some other targets, but this one is a good starter.
In the Post-build Actions section, select the Publish JUnit test result report. Enter the following into the Test report XMLs field:
You’re almost done, just click Save in the bottom.
Now run the project and check the outcome. After two runs, you should start to see a unittest graph.
Make a change to your code, push to the repo and see if hudson picks it up. It should poll once every minute and pick up the code change and start a build automatically
First off, after trying out most of the free build servers on the market, I’ve found that Hudson ( http://www.hudson-ci.org/ ) is the best for me. It’s by far the easiest to configure and it runs pretty smoothly. The plugin ecosystem is vibrant and there are constantly new and interesting plugins. I’ve used it for java, grails and even msbuild projects with success.
I’m learning about headless linux servers and hosting on them, so I’m using this an opportunity to document the sometimes tricky install procedures. I’m using Ubuntu 10.10 Server in a 32-bit edition. I must admit that I’m impressed by the performance of the headless linux server. It’s running in VMWare Player 3 with 512mb ram, on a USB drive, on my underpowered Lenovo T500 laptop… and it feels snappy!
I’ll assume that you’ve already done the steps for installing grails. If not, you at least need to do a succesful:
sudo aptitude install sun-java6-jdk
I got some of this installation info from http://hudson-ci.org/debian/. First we need to install a prerequisite:
sudo aptitude install daemon
Then we need to fetch a key (and install it) and a deb package (and install it). After that we update the apt repo list and do a “normal” install
wget -O /tmp/key http://hudson-ci.org/debian/hudson-ci.org.key
sudo apt-key add /tmp/key
wget -O /tmp/hudson.deb http://hudson-ci.org/latest/debian/hudson.deb
sudo dpkg --install /tmp/hudson.deb
sudo apt-get update
sudo apt-get install hudson
Once the package is installed, normal updating should work. Check that the install is okay by visiting:
Running hudson on port 8080 is not cool, since so many other servers want to run there (Tomcat for instance). So let’s change the port number:
First, stop the hudson server:
sudo /etc/init.d/hudson stop
To change change port number for hudson to 9090, run this line:
sudo perl -pi -e 's/HTTP_PORT=(.*)\n/HTTP_PORT=9090\n/g' /etc/default/hudson
It does a search and replace in the /etc/default/hudson and replaces the port number. Call this to verify the new port number:
cat /etc/default/hudson |grep HTTP_PORT=
Then you can start Hudson again:
sudo /etc/init.d/hudson start
In case you need to debug something or if you’re a log geek you can find the hudson log is in /var/log/hudson/hudson.log. You may want to run
tail -f /var/log/hudson/hudson.log
to keep viewing it
PS: Thanks to Theis Borg for helping me with the linux scripting!