Hadoop 測試

Hadoop 測試

MRUnit單元測試Mapper和Reducer類在內(nèi)存上獨立運行, PipelineMapReduceDriver單線程運行.

LocalJobRunner單線程運行, 且僅有一個 Reducer能夠啟動conf.set("mapred.job.tracker", "local"); conf.set("fs.default.name", "file:////"); FileSystem fs = FileSystem.getLocal(conf);

MiniMRCluster, MiniYarnCluster, MiniDFSCluster 多線程