Overheard - MapReduce is a screwdriver

“Just because you’ve got the best hammer in the entire world doesn’t make everything a nail. If you’ve got a screw, even a cheap, old, rusty screwdriver is going to do a better job. And MapReduce is a lot better than a cheap, old, rusty screwdriver.” Mark C. Chu-Carroll, Databases are hammers; MapReduce is a screwdriver

RDBs are absolutely brilliant things. They’re amazing tools, which can be used to build amazing software. I’ve done a lot of work using RDBs, and without them, I wouldn’t have been able to do some of the work that I’m proudest of. I don’t want to cut down RDBs at all: they’re truly great. But not everything is a relational database, and not everything is naturally suited towards being treated as if it were relational. The criticisms of MapReduce all come down to: “But it’s not the way relational databases would do it!” - without every realizing that that’s the point. RDBs don’t parallelize very well: how many RDBs do you know that can efficiently split a task among 1,000 cheap computers? RDBs don’t handle non-tabular data well: RDBs are notorious for doing a poor job on recursive data structures. MapReduce isn’t intended to replace relational databases: it’s intended to provide a lightweight way of programming things so that they can run fast by running in parallel on a lot of machines. That’s all it was intended to do.