In a previous post, I mentioned that we’d be using a radically tuned EXT4 FS for our Hadoop DataNodes. Well we did it. Then I turned off journaling!
It was scary, exciting and it seems to be working pretty well. No crazy FS corruption, even with systems power cycled w/ no proper shutdown.
We have yet to test this while data is being written, but I have a feeling that normal fsck will mark HDFS Block files as bad and well move on with life. After all, HDFS knows how to replicate blocks. As long as it can see a replica is bad, it should mark it so, make another copy and move on. And besides, if you’re really good at what you do, the net doesn’t matter anyway, cuz you won’t use it. 😉
~~ GM 