Working without a net

In a previous post, I mentioned that we’d be using a radically tuned EXT4 FS for our Hadoop DataNodes.  Well we did it.  Then I turned off journaling!

It was scary, exciting and it seems to be working pretty well.  No crazy FS corruption, even with systems power cycled w/ no proper shutdown.

We have yet to test this while data is being written, but I have a feeling that normal fsck will mark HDFS Block files as bad and well move on with life.  After all, HDFS knows how to replicate blocks.  As long as it can see a replica is bad, it should mark it so, make another copy and move on.  And besides, if you’re really good at what you do, the net doesn’t matter anyway, cuz you won’t use it. ;)

About Grease Monkey

30+ Years of IT Geekiness, Linux Fanboy and Open Source patriot.
This entry was posted in Administration, Tuning and tagged , , . Bookmark the permalink.

One Response to Working without a net

  1. Good article – I’m still a bit bothered with not having journaling on. What level of performance improvement do you expect (Is it worth it)? BTW – well s/r we’ll or we will.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>