makeitfaster

performance, scalability and optimizing processes

Development workstation in the Cloud

leave a comment »

I have a dream that one day I will not need to have a dedicated machine to do software development. That I will be free from the shackles of physical hardware and do all my development in the cloud.

I work as a consultant, so I change places quite often. It should not matter whether I’m at work, at home or at a client site. When virtualization first came along, I was pleased. My prayers were heard. In practice, the virtual machines required serious physical hardware to run at any decent speed and that still chained me to a big machine.

Then came the cloud. Hmm, not so much different from running my own machine… on the surface. In practice, it gives me a lot of freedom. I just rent my powerful machine instead of buying it.

I work with java, grails and .NET and for my purposes I use Windows 7 64bit most of the time. Sure, Linux is great too, but not for .NET work. Many of the cloud hosts do not support running windows so those are out of the question. I need a very responsive system, so I can’t feel the difference between running a remote desktop on my LAN or on the cloud.

That sets a very high requirement for the 4 big factors of cloud performance:

                  Network latency, Disk IOPS, RAM and CPU

Network latency is the most important. I need the host to register my mouse movement and my typing quickly. Cable quality, router speed and distance all matter. For my purposes I need to run it really close to home. I live in Copenhagen, Denmark and the network latency to Rackspace and Amazon is too great. Even their european servers are too far away in ping-time. I’ve tried host with 50ms ping time or less, and it’s still too much. I tried a great cloudhost called Cloudsigma in Zürich, but even though their performance was great, their latency was not good enough for me.

Up until now there was nothing to do. Host a server at home maybe…? Nah, I don’t want to run a powerful server at home. Not because I can’t, but because it’ll be a dinosaur in two years anyway. And what if i need 10 machines for a projekt? Or another 32 GB of ram? Nope, not for me. Cloud it is.

Cloud.dk is a new danish provider of cloud computers. Their datacenter is around 3 ms away from me, and their disk/ram/cpu are pretty good too. I’ll show some benchmarks later.
Here’s a little how-to of how to get a Windows 7 running on their infrastructure. I’ll assume that you have an account and know how to fire up a virtual machine from one of their existing images.

Step 1
Download the ISO image of your operating system. I used my MSDN account to download Windows 7 Enterprise with SP1 in the 64 bit version

Step 2
Make the ISO available via HTTP. In my case i uploaded it to an S3 bucket and made it public.

Step 3
Go into the cloud.dk control panel and go to “images” and then “fetch image” http://i.imgur.com/NkhRk.jpg

Step 4
Instruct cloud.dk to fecth the image from your httpserver. Take a look at the following image http://i.imgur.com/Y8LGG.jpg. Be sure to change the URL and be sure to add a default user.
That will start a job that fetches the image. Be sure to have a fast HTTP server, as the job will time out if it runs too long. You can go to the jobs tab to see the status. http://i.imgur.com/YHytl.jpg

Step 5
When the download is complete, you’ll have a new image in the list: http://i.imgur.com/Z8BsX.jpg. If you don’t see this, your download has failed for some reason.

Step 6
Make a new machine. Use the new image as the primary disk and add another 50GB or 100 GB disk if you need the extra space. I recommend at least 4 GB/ 2 vCPU if you plan on running Visual Studio 2010. http://i.imgur.com/jRpR0.jpg
Switch the primary and secondary disks: http://i.imgur.com/ht0gQ.jpg. Think of this as putting the harddisk as the primary and the uploaded iso image as the DVD-ROM. IF YOU FORGET THIS STEP, THE INSTALL WILL FAIL as it will be unable to partition correctly.
Start the server: http://i.imgur.com/6A921.jpg. Let the machine run for a couple of minutes and then start the console: http://i.imgur.com/oziFm.jpg
If you have java installed, it’ll start a program (webstart) that acts as a vnc console. You should see the windows install screen like this: http://i.imgur.com/p9Lrp.jpg

Step 7
Install windows as you normally would.

Step 8
Enable Remote Desktop in windows 7
Close the console
Login with remote desktop :)
Your Win 7 should be ready for you to setup as you like.
My install (from boot to remote desktop login) took around 12 minutes! That’s fast! And that time even included waiting for the human (me) to react

Step 9
Shutdown the machine. If you logged on with remote desktop you can shutdown with this command:
shutdown -s -f -t 3
After that, you wait for the VM to be completely off. When it’s off, remove the ISO image from the server: http://i.imgur.com/8W6KM.jpg
After that, delete the disk image. http://i.imgur.com/cn77v.jpg. It’s a copy of the uploaded image anyway, so no need to waste cloud storage on it. You can still make new machines with the uploaded image.

Step 10
That’s it. No more required steps. I do have a few suggestions for activities though:
- Get all windows updates
- Set up the cloud.dk firewall. RDP uses port 3389
- Enable power management and set to hibernate after 1 hour without use. Execute the command “powercfg -H ON” in an elevated prompt to enable hibernation feature of of Win7 if it’s not enabled already.
- Figure out how to start and stop your machine from the commandline API
- Disable the pesky UAC feature.

Oh, I almost forgot I promised some benchmarks. Remember this is the 4 GB / 2 vCPU machine configuration.

  • Geekbench: 4580 – A decent score, as expected.
  • Crystal Disk Mark 4K QD32: 43.77 MB/s read and 11.84 MB/s write. An impressive score for a cloud machine. They’re using a TSM system, where their SAN has a mix og ssd’s, harddrives, large ram caches and migrates automatically. Seems to work out nicely for them
  • Speedtest.net: Ping 5ms, download 186.64 Mbps, uploda 180.86 Mbps
  • Pingtest.net: Packet loss 0%, PIng 4 ms, jitter 1 ms.
  • Details here: http://i.imgur.com/qb9Jr.jpg

So: in summary, a decent RAM/CPU score, a good network score and a disk score that’s similar to a low end SSD. That’s just numbers, and good ones too, but…
What’s my “feel” for it? Well, it feels very snappy and fast. Complete windows install in 12 minutes. Windows updates download and install quickly. Any sort of manipulation of the user interface feels almost as if it was a local machine. Everything loads quickly and programs start up fast. It feels almost like it’s a SSD :)

Enjoy your new workstation in the cloud!

Written by Sebastian Vilstrup

October 18, 2011 at 15:24

Posted in Uncategorized

Java JDK silent install on windows

leave a comment »

I hate next-next-next installers. Up until now, I had to endure the horrors of that, while installing the Java JDK on windows.
Thank to my colleague Erik, I don’t have to do that anymore. With his help, I discovered how to do a silent command line install of the entire JDK:

UPDATE: this article turned obsolete. Instead I’ll provide the link to the “Installation notes” for the windows JDK 7:

http://www.oracle.com/technetwork/java/javase/documentation/install-windows-152927.html

I had succes with this command line:

 

jdk-7u2-windows-i586.exe /s ADDLOCAL="ToolsFeature,SourceFeature" INSTALLDIR="%CD%\jdk7u2"

 

Written by Sebastian Vilstrup

March 25, 2011 at 14:36

Posted in Java

Compiling and installing Apache webserver from source on redhat linux

leave a comment »

Just a quick post about something i had to to do today. It’s all in the Apache docs, but bits of it was hard to find. I was working on a redhat 4 64-bit

  • Grab the source from http://httpd.apache.org/download.cgi
    • I picked the Unix Source: httpd-2.2.17.tar.gz
  • Unpack with tar xzf httpd-2.2.17.tar.gz
  • Cd httpd-2.2.17
  • ./configure --prefix /path/to/apache --enable-mods-shared=all --enable-proxy --enable-ssl
    • If you leave out "--prefix ..." then the installation directory is set to /usr/local/apache2
  • make
    • Does all the compiling
  • make install
    • Installs into the directory that you wrote in --prefix
  • Edit the /conf/httpd.conf if you want to set a another port or change some other setting


Controlling the Apache server:

  • /bin/apachectl -k start
  • /bin/apachectl -k stop
  • /bin/apachectl -k restart
  • /bin/apachectl -k graceful-stop

Written by Sebastian Vilstrup

March 25, 2011 at 10:38

Congratulations are in order :)

leave a comment »

I am now a ScrumMaster with certificates and everything.

We did a two day ScrumMaster certification training course with our entire company (around 40 or 50 people). It was a very good course and our teachers were both knowledgeable, experienced and very inspiring. I can recommend them if you’in the market for some training.

Our trainers were by Gertrud Bjørnvig and Jim Coplien and their website is at: https://sites.google.com/a/gertrudandcope.com/www/

And here’s my proof and pretty certificate :)

All in all a very good experience, and I feel much more enthusiastic about Scrum than I did before.

 

 

 

Written by Sebastian Vilstrup

January 31, 2011 at 16:30

Posted in Uncategorized

Deploying from grails to tomcat

leave a comment »

I’ve set up my grails build on Hudson, and now I want to deploy to my Tomcat server. Every time my tests complete, I want a new deploy. And I want to be completely automatic too.

First step is to navigate to your grails app and try:

grails install-plugin tomcat

It’s probably already installed, as it’s the default for grails these days.

Then we need to install Tomcat if you don’t have it already.

First, make sure it’s the Sun JDK, argh, I mean Oracle, but the Ubuntu packages still say Sun. The OpenJDK doesn’t really work well with grails and tomcat.

sudo aptitude install sun-java6-jdk

Then install tomcat:

sudo aptitude install tomcat6

Then edit the tomcat user file to enable some users:

sudo nano /etc/tomcat6/tomcat-users.xml

I suggest you have the following lines active, but please change the passwords:

<role rolename=”manager”/>

<user username=”manager” password=”manager” roles=”standard,manager”/>

<user username=”grails” password=”grails” roles=”standard,manager”/>

Restart tomcat to activate the changes.

sudo /etc/init.d/tomcat6 restart

Test by going to:

http://<hostname&gt;:8080/manager

You should now specify the username/password and url for the to) in your grails project. Open the  grails-app/conf/Config.groovy file and add this at the bottom of the file:

tomcat.deploy.username=”grails”

tomcat.deploy.password=”grails”

tomcat.deploy.url=”http://<hostname&gt;:8080/manager”

That should set up easy deployment directly from grails. We’ll simply be using grails’ own plugin for deployment, so we’ll have no real need to understand how ant or tomcat handles it. It’s all nicely camouflaged by grails.

The deployment sequence is like this: First you need to undeploy the app from tomcat (not needed the first time)

grails tomcat undeploy

And then deploy the app

grails tomcat deploy

You should now be able to use your app on your testserver.

To add them to your Hudson project just add those two lines as a post-build shell script (or batch command if you’re on windows) and you’re good to go.

If undeployment fails, there may be some file locks. I experienced that on windows, but not on the Ubuntu 10.10 server. If (and only if) you have this problem, you can edit the

sudo nano /etc/tomcat6/context.xml

And change the <context> line to:

<Context antiJARLocking=”true” antiResourceLocking=”true” >

That fixed the issue for me.

PS: To control tomcat use these commands:

sudo /etc/init.d/tomcat6 stop

sudo /etc/init.d/tomcat6 start

sudo /etc/init.d/tomcat6 restart

Written by Sebastian Vilstrup

December 20, 2010 at 14:18

Posted in Uncategorized

Building a grails project in hudson

with one comment

I’ve learned how to set up setup grails and how to install hudson on Ubuntu 10.10 Server. Now I want to use hudson to build my grails project.

Before we set up the project, we need to install some plugins. Access your hudson admin interface, probably at http://<hostname>:9090

Go to Hudson -> Manage Hudson -> Manage plugins -> Available (it’s a tab) and get these required plugins:

Hudson Grails plugin

Hudson Mercurial plugin or get the one for git/svn/cvs depending on what you use. I’m a mercurial fan.

If you like, you can also install some of these, but they’re not required:

Radiator View Plugin

Green Balls

ChuckNorris Plugin

Monitoring

Build-timeout Plugin

Hudson Tray Application

 

After that you need start a new project by going to the link called Hudson -> New project

Choose a job name, and the Build a free-style software project. After you click OK, you get a lot of options. Only some of them are needed:

Discard Old Builds -> Days to keep: 3, Max # of builds to keep: 10

In the Source Code Management part you select Mercurial and then enter the Repository URL. For a private bitbucket it’s like this:

https://<username&gt;:<password>@bitbucket.org/<username>/<project>

Make sure to include the username:password if it’s a private project. For a public one, it’s not needed.

In the Build Triggers section, choose Poll SCM and enter this into the schedule field (it’s std cron notation):

# every minute

* * * * *

In the Build section, choose Add Build Step and Build With Grails. You’ll se a lot of grails related fields but you only need to enter the following into the Targets field:

test-app

Eventually, you’ll probably want to run some other targets, but this one is a good starter.

In the Post-build Actions section, select the Publish JUnit test result report. Enter the following into the Test report XMLs field:

target/test-reports/*.xml

You’re almost done, just click Save in the bottom.

Now run the project and check the outcome. After two runs, you should start to see a unittest graph.

Make a change to your code, push to the repo and see if hudson picks it up. It should poll once every minute and pick up the code change and start a build automatically

Written by Sebastian Vilstrup

December 14, 2010 at 16:40

Install the Hudson build server on Ubuntu 10.10 Server

with 6 comments

First off, after trying out most of the free build servers on the market, I’ve found that Hudson ( http://www.hudson-ci.org/ ) is the best for me. It’s by far the easiest to configure and it runs pretty smoothly. The plugin ecosystem is vibrant and there are constantly new and interesting plugins. I’ve used it for java, grails and even msbuild projects with success.

I’m learning about headless linux servers and hosting on them, so I’m using this an opportunity to document the sometimes tricky install procedures. I’m using Ubuntu 10.10 Server in a 32-bit edition. I must admit that I’m impressed by the performance of the headless linux server. It’s running in VMWare Player 3 with 512mb ram, on a USB drive, on my underpowered Lenovo T500 laptop… and it feels snappy!

I’ll assume that you’ve already done the steps for installing grails. If not, you at least need to do a succesful:

 sudo aptitude install sun-java6-jdk

I got some of this installation info from http://hudson-ci.org/debian/. First we need to install a prerequisite:

 sudo aptitude install daemon

Then we need to fetch a key (and install it) and a deb package (and install it). After that we update the apt repo list and do a “normal” install

 wget -O /tmp/key http://hudson-ci.org/debian/hudson-ci.org.key
 sudo apt-key add /tmp/key
 wget -O /tmp/hudson.deb http://hudson-ci.org/latest/debian/hudson.deb
 sudo dpkg --install /tmp/hudson.deb
 sudo apt-get update
 sudo apt-get install hudson

Once the package is installed, normal updating should work. Check that the install is okay by visiting:

 http://<hostname>:8080

Running hudson on port 8080 is not cool, since so many other servers want to run there (Tomcat for instance). So let’s change the port number:

First, stop the hudson server:

 sudo /etc/init.d/hudson stop

To change change port number for hudson to 9090, run this line:

sudo perl -pi -e 's/HTTP_PORT=(.*)\n/HTTP_PORT=9090\n/g' /etc/default/hudson

It does a search and replace in the /etc/default/hudson and replaces the port number. Call this to verify the new port number:

cat /etc/default/hudson |grep HTTP_PORT=

Then you can start Hudson again:

 sudo /etc/init.d/hudson start

In case you need to debug something or if you’re a log geek you can find the hudson log is in /var/log/hudson/hudson.log.  You may want to run

 tail -f /var/log/hudson/hudson.log

to keep viewing it

PS: Thanks to Theis Borg for helping me with the linux scripting!

Written by Sebastian Vilstrup

December 14, 2010 at 14:16

Follow

Get every new post delivered to your Inbox.