(undone) MIT6.824 Lab1

时间:2024-10-03 16:08:20

参考:http://nil.csail.mit.edu/6.824/2021/labs/lab-mr.html


task1: 熟悉讲义,尤其是搞明白如何运行测试程序(完成) ------------------------------------------------ start

先看 Introduction
我们的目标:构建一个MapReduce系统。
细节:实现一个工作进程,调用应用程序的Map和Reduce函数,并处理文件的读写;同时还将实现一个协调进程,负责分配任务给工作进程并应对失败的工作进程。构建的内容类似于MapReduce论文中描述的内容。(注意:实验中使用“协调者”而不是论文中的“主节点”。)
在这里插入图片描述

细节在后边
在这里插入图片描述
先根据讲义下载 LAB 代码

git clone git://g.csail.mit.edu/6.824-golabs-2021 6.824

看讲义:代码中已经有了 word-count 和 text indexer 两个应用。同时还有一个极简单的 MapReduce 实现(在一个进程中,一次只运行一个 map 和 一个reduce)。
我们电脑上没 go ,先安装,安装步骤如下 (WSL2 ubuntu20.04)

wget -qO- https://golang.org/dl/go1.15.8.linux-amd64.tar.gz | sudo tar xz -C /usr/local
sudo ln -s /usr/local/go/bin/go /usr/local/bin/go

安装完毕,我们试着按照讲义运行 word count 这个应用

cd ~/6.824
cd src/main
go build -race -buildmode=plugin ../mrapps/wc.go
go run -race mrsequential.go wc.so pg*.txt
more mr-out-0

出现了单词统计结果:
在这里插入图片描述
看来顺利运行了

看看后边的内容
这一段概述了要做的事情(作业目标)
在这里插入图片描述

下面内容介绍了一些代码大致位置:
1.main函数,包括对 协调者 和 工作者 的调动,在 main/mrcoordinator.go 里(不应修改)
2.我们的实现应该放在 mr/coordinator.go, mr/worker.go 和 mr/rpc.go 里
更多内容看后边
在这里插入图片描述
仔细阅读了后边的内容,总结了运行测试程序的方式:
1.修改 mr/coordinator.go Done 函数中的 返回值,把 return false 改成 return true
2.在 main 文件夹下,运行 bash test-mr.sh
看到输出如下:

*** Starting wc test.
2024/09/29 10:39:19 rpc.Register: method "Done" has 1 input parameters; needs exactly three
sort: cannot read: 'mr-out*': No such file or directory
cmp: EOF on mr-wc-all which is empty
--- wc output is not the same as mr-correct-wc.txt
--- wc test: FAIL
*** Starting indexer test.
2024/09/29 10:39:21 rpc.Register: method "Done" has 1 input parameters; needs exactly three
sort: cannot read: 'mr-out*': No such file or directory
cmp: EOF on mr-indexer-all which is empty
--- indexer output is not the same as mr-correct-indexer.txt
--- indexer test: FAIL
*** Starting map parallelism test.
2024/09/29 10:39:22 rpc.Register: method "Done" has 1 input parameters; needs exactly three
cat: 'mr-out*': No such file or directory
--- saw 0 workers rather than 2
--- map parallelism test: FAIL
cat: 'mr-out*': No such file or directory
--- map workers did not run in parallel
--- map parallelism test: FAIL
*** Starting job count test.
2024/09/29 10:39:24 rpc.Register: method "Done" has 1 input parameters; needs exactly three
cat: 'mr-out*': No such file or directory
test-mr.sh: line 170: [: : integer expression expected
--- job count test: PASS
*** Starting early exit test.
2024/09/29 10:39:25 rpc.Register: method "Done" has 1 input parameters; needs exactly three
sort: cannot read: 'mr-out*': No such file or directory
sort: cannot read: 'mr-out*': No such file or directory
--- early exit test: PASS
*** Starting crash test.
2024/09/29 10:39:26 rpc.Register: method "Done" has 1 input parameters; needs exactly three
sort: cannot read: 'mr-out*': No such file or directory
cmp: EOF on mr-crash-all which is empty
--- crash output is not the same as mr-correct-crash.txt
--- crash test: FAIL
*** FAILED SOME TESTS

本 task 完成

task1: 熟悉讲义,尤其是搞明白如何运行测试程序(完成) ------------------------------------------------ end

task2: 该明白测试程序的原理,尤其是如何测试代码是并行运行的 ------------------------------------------------ start

TODO:here

task2: 该明白测试程序的原理,尤其是如何测试代码是并行运行的 ------------------------------------------------ end

TODO: 看看 word count 和 MapReduce 的联动

TODO: here