MongoDB : Data Storage Internals

Previous: Mongodb Replication Configuration

How is Data Stored?

Your application will have information in memory. It then talks to the server and the server will have data stored permanently on the disk.  So how does the engine store this data? Mongo uses memory mapped files. The server cannot store all its information in memory, but it likes to think that all of the information is readily available in memory. So what it does is create a humongous array (byte arrays) and maps it using memory mapped files. When ever  information is needed from a portion of that array, the operating system takes care of loading it or storing it to the disk.

Image Source

When you have to store information , you hand it over to the server. The server stores it in memory and that memory gets managed and serialized to disk. When you want to read data, the server will attempt to access a portion of the large byte array which will be loaded as needed from the operating system. Since this operating system functionality is a core function native to any OS, it is highly optimized, highly stable and fast. so the MongoDB server does not have to deal with these issues and can focus on serving data.

But how does your schema less data get saved on disk from the byte array? The  answer is BSON. BSON data format has several advantages. The key advantages being

1. There is very little marshaling necessary from BSON elementary data types to memory (C data types; Mongo is written in C,C++). That makes reading and writing really fast
2. BSON documents are very easy to traverse. They are very easy to skip over because there is length prefixing and its very easy to drill into a document and ind particular fields. There is a slight overhead in field name storage but that’s how the BSON structure allows you to be schema-less. It allows you to store arbitrary  object structures and not have to adhere to a particular schema.

Rules for saving data in Mongo

1. Every document that you save must have an “_ID” field. (Spelled exactly ‘_ID’). If you don’t specify an ID, Mongo will assign you one.
2. The rest of the fields or document structure is up-to you. Mongo imposes no other rules
3. The size of a document is limited to 16 MB. If you need more size the document will have to be split across as multiple documents. You can configure the size using the nsSize option. The WiredTiger storage engine is not subject to this limitation.

Next up: Operations in Mongo (insert,update,delete etc..)

For all your application development needs, visit www.verbat.com for a fiscally conscious proposal that meets your needs ( So I can keep this blog going as well!!!!)

Alternatively click through the link   if you found this article interesting. (This will help the companies Search engine rankings)