Category Archives: Tuning

Performance and Tuning discussions

Experimenting w/ Neo4j

Graph databases are a really neat concept. We’ve started playing with Neo here as we attempt to link customers with visits and actions based on those visits. It seems like a really good fit at first glance. Our challenge is … Continue reading

Posted in Data, Development, Tuning | Leave a comment

hello woRld!

R is the latest Hadoop darling. It is an open source language that “is widely used among statisticians and data miners for developing statistical software and data analysis. Polls and surveys of data miners are showing R’s popularity has increased … Continue reading

Posted in Administration, Deployment, Development, Tuning | Tagged , , , | Leave a comment

GlusterFS and Hadoop, not replacing HDFS

Enterprise Hadoop must cooperate with many other forms of data transmission and ingestion. Any form of MFT, Mqueue or file landing zone requires disk space. Not HDFS disk, just disk that we can mount, MFT, SFTP, etc. to until we … Continue reading

Posted in Administration, Deployment, Tuning | Leave a comment

Replication FAIL

We’ve had our clusters running for a few months without significant issues. Or at least so we thought. I’m not sure of the why and how yet, but it seems that even rack topology scripts running, replication factor of 3 … Continue reading

Posted in Administration, Deployment, Tuning | Leave a comment

Working without a net

In a previous post, I mentioned that we’d be using a radically tuned EXT4 FS for our Hadoop DataNodes.  Well we did it.  Then I turned off journaling! It was scary, exciting and it seems to be working pretty well. … Continue reading

Posted in Administration, Tuning | Tagged , , | 1 Comment

It’s scary… but sometimes things go really well!

If you’ve been in IT for more than a few minutes, you know that things seldom go as planned.  Sometimes they go sideways and you’re left to clean up an abomination.  Been there, done that, expect it at least 15% of … Continue reading

Posted in Administration, Deployment, Development, Tuning | Tagged , , | 1 Comment

Calculating Bandwidth and Binning Time

I found Michael Baker’s blog here and wanted to summarize it here specifically for the calculating bandwidth part. Start with Packetpig custom loaders that allow you to access specific information in packet captures. There are a number of them but … Continue reading

Posted in Tuning | Leave a comment