Comments for All Things Hadoop

Comment on Faster Datanodes with less wait io using df instead of du by Edward Capriolo

Edward Capriolo — Sat, 21 May 2011 15:01:17 +0000

Good job! I mentioned this on the hadoop mailing list and someone did mention that you can tune your VFS cache so it caches more inodes and less data ( though I already have 7 GB vfs cache). It is nice to see that someone else noticed this is a problem.

Comment on Cloudera, Yahoo and the Apache Hadoop Community Security Branch Release Update by Allen Wittenauer

Allen Wittenauer — Fri, 06 May 2011 04:36:24 +0000

FWIW, LinkedIn runs base Apache 0.20.2 with 3-4 patches. These patches are fixes to the capacity scheduler and non-Linux portability (Mac OS X and Solaris). The “huge team” that builds and supports our production code base is me, either writing new code or grabbing patches from JIRA with some occasional help from the authors of those patches. Now that Jakob is an employee, our next internal release might have two people that support it. Two is not exactly huge. (Most of the other work that LinkedIn does with Hadoop is on the periphery–Azkaban, Pig, etc.)

Anyway, it was important to me that LI run what Apache runs for a variety of reasons. The big one being so that our options are open. In retrospect, this was a great decision given the incompatibilities that are now flooding into the ecosystem between these forks and trunk. It will be interesting to see how the various vendors force upgrades on users to deal with them.

(I’m working on dropping the phrase “Hadoop distribution” from my vocabulary. They are forks, no matter how much marketing wants to say otherwise).

The other thing to keep in mind is that if one has been paying attention, the tension is not new. These battle lines were drawn a long, long time ago.

Comment on Hadoop Streaming Made Simple using Joins and Keys with Python by [repost] Medialets Architecture – Defeating the Daunting Mobile Device Data Deluge » New IT Farmer

Fri, 11 Mar 2011 13:48:21 +0000

[...] Python – a lot of MapReduce is being moved into Python using Python streaming. [...]

Comment on Hadoop Streaming Made Simple using Joins and Keys with Python by Quora

Quora — Thu, 06 Jan 2011 04:55:21 +0000

What issues do developers generally face with Map-Reduce, the programming model, when they use Hadoop?…

There are challenges specifically to MapReduce and then other challenges using Hadoop (which implements MapReduce) for developers, generally speaking. On the MapReduce side it is an entirely different thought process you need to have when attacking pro…

Comment on NoSQL HBase and Hadoop with Todd Lipcon from Cloudera by NoSQL Daily – Thu Oct 14 › PHP App Engine

NoSQL Daily – Thu Oct 14 › PHP App Engine — Thu, 14 Oct 2010 08:15:43 +0000

[...] NoSQL HBase and Hadoop with Todd Lipcon from Cloudera « All Things Hadoop [...]

Comment on Running Hadoop MapReduce With Cassandra NoSQL by NoSQL Daily – Mon Sep 27 › PHP App Engine

NoSQL Daily – Mon Sep 27 › PHP App Engine — Mon, 27 Sep 2010 08:15:50 +0000

[...] Running Hadoop MapReduce With Cassandra NoSQL « All Things Hadoop [...]

Comment on Podcast by Some podcasts « Data story

Some podcasts « Data story — Wed, 15 Sep 2010 11:35:41 +0000

[...] Hadoop related podcasts: [...]

Comment on NoSQL HBase and Hadoop with Todd Lipcon from Cloudera by NoSQL Daily – Tue Sep 7 › PHP App Engine

NoSQL Daily – Tue Sep 7 › PHP App Engine — Tue, 07 Sep 2010 17:08:53 +0000

[...] NoSQL HBase and Hadoop with Todd Lipcon from Cloudera « All Thin… – <4> [...]

Comment on Running Hadoop MapReduce With Cassandra NoSQL by Cassandra and Hadoop | Mawazo

Cassandra and Hadoop | Mawazo — Mon, 02 Aug 2010 05:42:46 +0000

[...] http://allthingshadoop.com/2010/04/24/running-hadoop-mapreduce-with-cassandra-nosql/ [...]

Comment on Hadoop Cluster Setup, SSH Key Authentication by charmalloc

charmalloc — Wed, 21 Jul 2010 16:30:43 +0000

If you have not already posted to the user group you should http://hadoop.apache.org/mailing_lists.html