Let me start by saying that the ideas discussed here are my own, and not necessarily that of my employer (EMC). This is my own personal blog.

A great article by Andrew Oliver has been doing the rounds called “Never ever do this to Hadoop”. Andrew argues that the best architecture for Hadoop is not external shared storage, but rather direct attached storage (DAS). The article can be found here: http://www.infoworld.com/article/2609694/application-development/never–ever-do-this-to-hadoop.html

I want to present a counter argument to this. Now having seen what a lot of companies are doing in this space, let me just say that Andrew’s ideas are spot on, but only applicable to traditional SAN and NAS platforms. There is a new next generation storage architecture that is taking the Hadoop world by storm (pardon the pun!). EMC has done something very different which is to embed the Hadoop filsyetem (HDFS) into the Isilon platform. This approach changes every part of the Hadoop design equation.

Ok, here is an issue I have been struggling with for a while and finally came across a fix. When installing Pivotal Cloud Foundry for a second or third time, often the installation will drop out on the step “creating director user” with the following error:

Target already set to `microbosh-b5f7fa48777480cf307e’ {“type”: “step_finished”, “id”: “microbosh.logging_into_director”} {“type”: “step_started”, “id”: “microbosh.creating_director_user”}

Running bosh login director f539d3aebcc43dc2d2f0

Here’s another little issue that I couldn’t find any documentation on, but managed to work around. When installing […]

If you encounter this issue while Installing Pivotal CF then here is a potential fix. A web search showed no public fix to this so this might help others and the fix is fairly simple. The exact error I received is below and this occurred while installing Pivotal Cloud Foundry v1.2.2. on ESX 5.5.

`block in wait_for_task’: The operation is not allowed in the current connection state of the host. (RuntimeError)

Ok, so our app has become popular and too many users are accessing it. We need to scale. If only we had a….wait…a PaaS. Ok so here is what to do.

First we could do this via the GUI, but we are going to use the command line just for fun.

Now that you have built your app we need to add a database. Use the following command to see what services you can create:

> cf marketplace

You should see something like this.
service   plans       description
p-hd      Standard    Pivotal HD is the industry’s most full-featured Hadoop distribution.
p-mysql   100mb-dev   MySQL service for application development and testing 

Here’s a tutorial on how to get started with Pivotal CF. In part 1 we will focus on deploying a pre-built app to the cloud.

Before you begin

Download the “CF” command line tool from: https://console.run.pivotal.io/tools

Download the sample Java app that we will be using for this exercise: booking-mvc.war.zip

Unzip the file

1. set your target API to the api address of the cloud foundry interface. In this case our domain is cf.mattzwol.com

> cf api api.cf.mattzwol.com

note: you may need an extra command here in the line above to skip ssl verification. It will tell you if you do. In this case use > cf api  api.cf.mattzwol.com –skip-ssl-validation