如何加快我的mongodb测试？

From my test, i get around 500 inserts, 200 queries, 400 updates per sec. I wonder on what i can tweak to increase these numbers.

从我的测试中,我得到大约500个插入,200个查询,每秒400次更新。我想知道我可以调整什么来增加这些数字。

I have read that others can achieve thousands or even tens of thousand of inserts in their tests, which is far better than my test. I wonder if i am missing something basic?

我已经读过,其他人可以在他们的测试中获得数千甚至数万个插件,这比我的测试要好得多。我想知道我是否遗漏了一些基本的东西?

So, here are the facts :

所以,这是事实:

I am using a win 32-bit mongodb v2.0.3 with default configuration

我正在使用默认配置的win 32位mongodb v2.0.3

Java driver(2.7.3) with spring mongo, (i dont force fsync)

带有spring mongo的Java驱动程序(2.7.3),(我不强制使用fsync)

Doing a combination of insertion and atomic updates like push, pull, inc, dec, set,

执行插入和原子更新的组合,如push,pull,inc,dec,set,

And repeat all these for 500k times.

并重复所有这些500k次。

The intention is to simulate user actions like insertion and updates

目的是模拟插入和更新等用户操作

No specific indexes are defined, but i think by default there'll be always the unique index on the id ?

没有定义特定的索引,但我认为默认情况下id上总会有唯一索引?

The java app which runs in eclipse IDE is running in the same machine as the mongod server

在eclipse IDE中运行的java应用程序与mongod服务器在同一台机器上运行

H/W spec : core i5, mem 4GB, thinkpad edge

硬件规格:核心i5,内存4GB,thinkpad优势

I notice the java process takes around 280MB and is stable in that number in the course of the loop

我注意到java进程大约需要280MB,并且在循环过程中该数字是稳定的

The start time is : 2012-03-08 21:50:16

开始时间是:2012-03-08 21:50:16

And i monitor using mongostat, and after reaching the time of 22:05:10, i terminate my unfinished application .. and here's the last output of the mongostat

我使用mongostat进行监控,在到达22:05:10的时间后,我终止了我未完成的应用程序..这里是mongostat的最后一个输出

insert  query update delete getmore command flushes mapped  vsize    res faults locked % idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
   499    200    400      0       0     100       0  1023m  1.06g   581m    145      8.5          0       0|0     0|0   645k    97k     3   22:05:01
   503    201    403      0       0     102       0  1023m  1.06g   582m    154     10.7          0       0|0     0|1   651k    98k     3   22:05:02
   520    208    415      0       0     105       0  1023m  1.06g   582m    176     11.1          0       0|0     0|0   671k   101k     3   22:05:03
   504    202    403      0       0     102       0  1023m  1.06g   582m    167      7.2          0       0|0     0|0   651k    98k     3   22:05:04
   524    209    419      0       0     106       0  1023m  1.06g   582m    147      8.3          0       0|0     0|0   675k   102k     3   22:05:05
   534    213    428      0       0     107       0  1023m  1.06g   583m    176      7.4          0       0|0     0|0   690k   103k     3   22:05:06
   531    213    424      0       0     108       0  1023m  1.06g   584m    160      4.9          0       0|0     0|0   685k   104k     3   22:05:07
   533    213    427      0       0     107       0  1023m  1.06g   584m    164      6.9          0       0|0     0|0   689k   103k     3   22:05:08
   518    208    414      0       0     105       0  1023m  1.06g   585m    158      7.3          0       0|0     0|0   669k   101k     3   22:05:09
   521    208    417      0       0     105       0  1023m  1.06g   585m    154      4.7          0       0|0     0|0   673k   101k     3   22:05:10

And then i checked my insertion number :

然后我检查了我的插入号码:

> db.myCollection.find().size();
90575

And this is an example of my document inserted, which is also updated etc in the process

这是我插入的文档的一个示例,在此过程中也会更新等

> db.myCollection.findOne().pretty();
{
        "_id" : "b146189a-56a4-4035-8245-c4bd6dc2bd22",
        "something1" : "my class is cool !",
        "something2" : {
                "value" : "this is a statement blah blah",
                "name" : "myStatement"
        },
        "something3" : {
                "size" : {
                        "value" : 0,
                        "name" : "size"
                },
                "value" : [
                        "6810cb0c-fa3e-4ca9-8a27-8432f2d1e828",
                        "a8276d05-a796-4c43-bc74-edc06d074099"
                ],
                "name" : "myids"
        },
        "something4" : {
                "myattr" : {
                        "value" : "something",
                        "name" : "name"
                },
                "attr" : {
                        "content" : {
                                "value" : "another another body body content content",
                                "name" : "content"
                        },
                        "contentId" : "b146189a-56a4-4035-8245-c4bd6dc2bd22",
                        "name" : "something"
                },
                "subsubchildchild" : {
                        "size" : {
                                "value" : 0,
                                "name" : "size"
                        },
                        "value" : [ ],
                        "name" : "subBodies"
                },
                "myId" : "b146189a-56a4-4035-8245-c4bd6dc2bd22",
                "name" : "hiccups"
        },
        "something5" : {
                "value" : false,
                "name" : "hahaha"
        },
        "something6" : {
                "name" : "okay this is just a test"
        },
        "something7" : {
                "value" : false,
                "name" : "remove me !"
        },
        "something8" : {
                "size" : {
                        "value" : 0,
                        "name" : "size"
                },
                "value" : [ ],
                "name" : "guess what"
        },
        "something9" : {
                "size" : {
                        "value" : 0,
                        "name" : "anotherSize"
                },
                "value" : [ ],
                "name" : "tarantula"
        },
        "something10" : {
                "value" : 8,
                "name" : "my exam score"
        },
        "something11" : {
                "size" : {
                        "value" : 0,
                        "name" : "justAnotherSize"
                },
                "value" : [ ],
                "name" : "myReference"
        },
        "something12" : {
                "size" : {
                        "value" : 0,
                        "name" : "size"
                },
                "value" : [ ],
                "name" : "myOtherReference"
        },
        "something13" : {
                "value" : "8b78fff0-50f5-4992-9972-89f9d944fee7",
                "name" : "user"
        },
        "something14" : {
                "dateTime" : "2012-03-08 21:50:17.480000000"
        },
        "something15" : {
                "value" : false,
                "name" : "lovely"
        }
}

And here's my db stat :

这是我的db stat:

> db.stats();
{
        "db" : "qa",
        "collections" : 7,
        "objects" : 815197,
        "avgObjSize" : 622.2093211824872,
        "dataSize" : 507223172,
        "storageSize" : 610770944,
        "numExtents" : 57,
        "indexes" : 5,
        "indexSize" : 64197952,
        "fileSize" : 1056702464,
        "nsSizeMB" : 16,
        "ok" : 1
}

Also another question from curiosity .. judging from my main collection size, which has around 90k records, and other nonsubstantial collections which shouldnt be big in size, is it reasonable to have around 1TB of fileSize in this case ? Is there anything i can do to help reduce my filesize ?

好奇心的另一个问题......从我的主要收藏规模来看,它有大约90k的记录,以及其他非实质性的收藏品,它们的大小不大,在这种情况下,大约1TB的fileSize是否合理?有什么我可以帮助减少我的文件大小?

Please share your thoughts.

请分享你的想法。

1 个解决方案

#1

You seem to be getting a lot of faults on the mongostat. Any idea why?

你似乎在mongostat上遇到了很多错误。知道为什么吗?

Doing a combination of insertion and atomic updates like push, pull, inc, dec, set

执行插入和原子更新的组合,如push,pull,inc,dec,set

How are you issuing these updates? By _id?

你是如何发布这些更新的?通过_id?

I have read that others can achieve thousands or even tens of thousand of inserts in their tests, which is far better than my test. I wonder if i am missing something basic?

我已经读过,其他人可以在他们的测试中获得数千甚至数万个插件,这比我的测试要好得多。我想知道我是否遗漏了一些基本的东西?

According to mongostat you only have 3 connections active and your lock % is only about 10%.

根据mongostat,您只有3个连接处于活动状态,而您的锁定%仅为10%左右。

Are you multi-threading your inputs?

您是否多线程输入?

Are you ruining this all on the same computer?

你是否在同一台计算机上破坏了这一切?

How is system IO?

系统IO如何?

Are you doing WriteConcern.Safe?

你在做WriteConcern.Safe吗?

These are all considerations that could be affecting your throughput.

这些都是可能影响吞吐量的因素。

is it reasonable to have around 1TB of fileSize in this case ?

在这种情况下,拥有大约1TB的fileSize是否合理?

Base on your db.stats(), you only have about 600 MB of disk in use.

基于db.stats(),您只有大约600 MB的磁盘在使用中。

"storageSize" : 610770944 // = 610,770,944

Your average object size is 622 bytes, but you have 815,197 objects, not the 90k documents you claim to have.

您的平均对象大小为622字节,但您有815,197个对象,而不是您声称拥有的90k文档。

Is there anything i can do to help reduce my filesize ?

有什么我可以帮助减少我的文件大小?

Yes, reduce the size of the Keys in your JSON documents. For example:

是的,减小JSON文档中Keys的大小。例如:

"something1" : "my class is cool !"  => ~28 bytes
"s1": "my class is cool !"           => ~20 bytes

Ensure that you are correctly storing the shortened names and let you Data access framework do the lifting of mapping these to more reasonable names.

确保您正确存储缩短的名称,并让数据访问框架解除将这些名称映射到更合理的名称。

#1