Tag Archives: configuration

Weaponizing Hadoop

We are usually left to bash for scripting Hadoop functions. It’s the default in Linux and it’s usually good enough. There are enough “bash-isms” that will cause your Java/pig/database people serious heart ache. If you’re new to Hadoop, go ahead … Continue reading

Posted in Administration, Deployment, Development, syndicated | Tagged , , , | Leave a comment

Building Clusters without installing an Operating System

I’m about to tell you how we build Hadoop clusters without install an OS on the DataNodes.  We simply PXE boot them, assign them to a cluster and they join automatically.  Read on (and on) to see how we do … Continue reading

Posted in Administration, Deployment | Tagged , , , , , | Leave a comment

Working without a net

In a previous post, I mentioned that we’d be using a radically tuned EXT4 FS for our Hadoop DataNodes.  Well we did it.  Then I turned off journaling! It was scary, exciting and it seems to be working pretty well. … Continue reading

Posted in Administration, Tuning | Tagged , , | 1 Comment

Silly Rabbit, HMC is for Kids!

I was using HMC 1.0 to install and configure a hadoop test cluster.  I wanted to make some changes to /usr/lib/hadoop/conf because the property that I needed to change is not exposed via Manage Services Map Reduce in HMC.  I tried … Continue reading

Posted in Uncategorized | Tagged , , | Leave a comment