# Notes on JSON Modeling for Document Databases Notes from a presentation by [https://twitter.com/mgroves](@mgroves) . Replay available [https://www.youtube.com/channel/UC5Cjdv38sS_yNEue-QUAF7g](here) Agenda: - Why NoSQL? - JSON Data Modeling - Accessing Data - Migrating Data - Summary / Q&A ### Why NoSQL "Change is inevitable" and relational databases don't handle change as well. NoSQL databases scale much easier Types of NoSQL Databases: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614812965257/Vp2usB-b0.png) ### Document DB - Get, Set, Replace, Delete by key(s) Simplest type of a key-value store Scalability comes in the form of multiple nodes in a cluster No inherent relationship between data. Provides flexibility Performance Use Cases: Caching, session, user profile, content management Other users: gaming, advertising, travel booking, loyalty programs, fraud monitoring, social media, finance, customer 360, IoT, communication ### JSON Data Modeling Properties of Real-World Data: example: customer - Name - DoB - Billing - Connections - Purchases ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813429625/o0Dbpom2I.png) The old way of relational databases: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813494830/V1t4t0gMd.png) As schemas get more complex, it gets harder to add tables and columns. RDBMS vs DocumentDB: Standard Customer table: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813569704/bD1bcAg5d.png) Introduce another table: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813613133/091KpFLYo.png) RDBMS would require a "Join" at this point, but with the document model, it's still just one piece document Adding more rows is easy by adding items to the array (quotes are not optional for valid JSON) Creating the connections array refers to another document with a documentkey ID. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813730393/Uk8oZvFDT.png) The full model: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813795353/XabNDvO4o.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813855475/ApoP3zneM.png) Ways to control documents: You can version documents with different ways to break out the model Does not require a change to pk or fk. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614813991217/SYf5K6gBB.png) Can also reversion with a web application. Get data from old document and save it to the new (user initiated) Tools to help with modeling ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814098405/OxnS1Gird.png) jsoneditoronline.org is the only free one. ### Accessing Data SQL is accessed via SQL Query. NoSQL has many many options to access. - Example using C#: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814199061/yN4n5Cmh3.png) Recommendations for keys: - Natural Keys (something that has a value that makes sense, but that won't change) - Human Readable - Deterministic - Semantic Deterministic means that a "march" through the data makes sense. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814339933/CpJSJZfO-.png) Example: Find author, pull up author's blogs, go to blog posts, pull blog posts comments Relationships: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814534162/x4ArAL3Wq.png) #### Subdocument Access (API) Update via subdocument api, can speed up updates without accessing the whole document #### Other ways to access the data ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814645463/HS7UTXQba.png) N1QL (SQL for Document DB) Example: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814742571/tK4R7FEa0.png) Search with index and use suggestions to improve performance. Full Text Search - find all the results that have the word you're looking for ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814952007/6ZAsXZqtq.png) Good for lots of things like inverted index, geographical ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614814993644/ChxybP7Mn.png) Use the right concept to access the data, and model the data accordingly. ### Migrating ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614815241514/34Uy5PMQ1.png) Tools: ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614815340756/CeTqScFfR.png) ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614815422843/YYQ4cfs8h.png) Do it with a simple CSV output and import. Once data is in place, start using or transforming that data into a future state. ![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1614815497839/cl-SxzQmi.png) Align your data with the ways you want to access it. CouchBase ships with samples. ### DEMO Demo showed creating a bucket and running a C# program to convert the data from SQL Server to Couchbase. Bucket contains Scopes, Scopes contain collections, collection contains documents ### Take Away - Pick the right application - Proof of concept - Match the data access method to the requirements - Links: https://blog.couchbase.com/sql-to-nosql-automated-migration/ https://blog.couchbase.com/proof-of-concept-move-relational/ missed the last one [Couchbase Download](couchbase.com/downloads) [CouchBase Influitive](https://couchbase.influitive.com/join/MGCTA) [Link to github repo of demo](https://github.com/mgroves/SqlServerToCouchbase)