MySQL NoSQL Document Store

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Mark Swarbrick
MySQL Principle Presales Consultant
Mark.Swarbrick@Oracle.com
MySQL 8.0
a Document Store with all the benefits of a
transactional RDBMS

Safe Harbor Statement
The following is intended to outline our general product direction. It is
intended for information purpose only, and may not be incorporated into
any contract. It is not a commitment to deliver any material, code, or
functionality, and should not be relied up in making purchasing decisions.
The development, release and timing of any features or functionality
described for Oracle´s product remains at the sole discretion of Oracle.
Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.

• Transactional (InnoDB) Data Dictionary
• Persistent runtime configuration changes (SET PERSIST)
• Auto-increment counter value will now persist across server restarts!
• Indexing of PS tables
• SQL roles (collection of privileges)
• MySQL privilege tables are now InnoDB
• INVISIBLE indexes
• Common Table Expressions
• Windows Functions
What’s new
3

• New default: utf8mb4 (UTF 9)
• Expanded GIS Support
• Expanded Query Hints Support
• Improved Scan Query Performance
• Improved BLOB Storage
• Improved InnoDB Memcached Interface
• Descending indexes
• Scalability Improvements
• Parser Refactoring
• New! Document Store
• Improvements to Temporary Tables
• C++11 and Toolchain Improvements
• Replication Applier Thread Progress
Reports
• GTID_PURGED always settable
• …
4
MySQL 8.0: All this and more….

• Increased Reliability
• Built on InnoDB
– No FRM files
– No DB.OPT files
– No TRG files
– No TRN files
– No PAR files
– MyISAM not required
• Atomic DDL
– Better Replication
– Simplifies server edge cases
• MDL for Foreign Keys
• Flexible Metadata API
– Easier path to adding new features
• Better cross-platform experience
– No dependencies on filesystem
semantics
5
MySQL 8.0: Native Data Dictionary

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 6
MySQL 8.0: Performance Schema Indexes
• Allows for more efficient access to Performance
Schema tables
• A total of 115 indexes across 93 tables
• Adds zero overhead
– A physical index is not maintained internally
– Implementation of indexes tricks the optimizer into better execution plan
Feature Request
from DBAs

MySQL 8.0: Cloud Friendly
• New variable innodb_dedicated_server
– according to the amount of memory detected on the server, automatically configure
innodb_buffer_pool_size, innodb_log_file_size, innodb_flush_method
• Persistent runtime configuration changes
– SET PERSIST variable_name = value;
– Any GLOBAL DYNAMIC configuration variable can be modified
– the settings will be persisted to a file named mysqld-auto.cnf (JSON format)
– New table performance_schema.variables_info
7

MySQL 8.0: Invisible Indexes
• Indexes are “hidden” to the MySQL Optimizer
– Not the same as “disabled indexes”
– Contents are fully up to date and maintained by DML
• Two use cases:
– Soft Delete (Recycle Bin)
– Staged Rollout
Feature Request
from DBAs

MySQL 8.0: Atomicity in Privileges
• Privilege Tables now 100% InnoDB
• User Management DDLs Atomic
– CREATE USER
– ALTER USER
– RENAME USER
– DROP USER
– GRANT
– REVOKE

GIS
• Geography Support
• Spatial Reference Systems (SRS) Support
• SQL/MM Information Schema views
• Standard compliant axis ordering in import/export functions
• Helper functions to manipulate and convert data:
• st_y(geom, y)
• st_srid(geom, srid)
10

MySQL 8.0: Windows Functions
• Window functions gather results into a set of rows,
returning a single aggregated value for each of the rows
in the result set. Gives you a very practical tool for
estimations and rankings.
• Perform some calculation on a set of rows
• Over(partition by ...)
• Simplifies writing complex SQL:
• OVER(), OTHER(partition)
11
Feature Request
from Developers

12
Feature Request
from Developers

13
Feature Request
from Developers
• Use the usual aggregate functions or...
• RANK(), DENSE_RANK(), PERCENT_RANK()
• CUME_DIST()
• NTILE()
• ROW_NUMBER()
• FIRST_VALUE(), LAST_VALUE(), NTH_VALUE()
• LEAD(), LAG()

•Evolution from MySQL 5.5 to 8.0
Performance Schema
MySQL 8.0
Histograms
Indexes
Data Locks instrumentation
SQL Errors instrumentation
Variables Instrumentation
Table plugin
Improved Defaults
MySQL 5.7
Memory Instrumentation
Prepared Statements Instrumentation
Transactions Instrumentation
Scalable Memory Allocation
Bundled SYS schema
Lower Overhead
MySQL 5.6
Statement Instrumentation
Lower Overhead
MySQL 5.5
Event Waits
Mutexes
Files
Threads
Feature Request
from DBAs

• Improved JSON Support
– Better perfomances, new functions (aggregate, formatting...), support for ranges
• Better Handling of Hot Rows
• Unicode 9.0
• Improvements to Query Consistency (Optimizer Histograms)
• Extended GIS
What’s new
15

•Improved to consider buffer pool fit
Optimizer Cost Model
Feature Request
from DBAs
SELECT * FROM Country
WHERE population > 20000000;
Model for a table scan:
# pages in table *
(IO_BLOCK_READ_COST |
MEMORY_BLOCK_READ_COST)
# records * ROW_EVALUATE_COST
= 25.4 100% in memory
= 29.9 100% on disk
Model for a range scan:
# records_in_range *
(IO_BLOCK_READ_COST |
MEMORY_BLOCK_READ_COST)
# records_in_range *
ROW_EVALUATE_COST + #
records_in_range *
ROW_EVALUATE_COST
= 22.5 100% in memory
= 60 100% on disk
Model accounts for
memory fit. For data on
disk an IO block read
defaults to 1.0. In
memory defaults to 0.25.
Much larger
performance
difference for range
scan not in memory
(good)

query: INSERT INTO `t1` VALUES (...)
db: mysqlslap
total_latency: 54.43 s
exec_count: 58377
lock_latency: 1.70 s
..
digest: 4e0c5b796c4052b0da4548fd7cb694be
first_seen: 2017-04-16 20:59:16
last_seen: 2017-04-16 21:00:34
latency_distribution:
0us+
10us+ #############################
100us+ ####################
1ms+ #
10ms+
100ms+
1s+
10s+
New! Performance Schema Histograms
17
+---------+--------------------------------+-------+
| bucket | visualization | count |
+---------+--------------------------------+-------+
| 0us+ | # | 1253 |
| 10us+ | ############################## | 43102 |
| 100us+ | ################# | 25013 |
| 1ms+ | # | 2003 |
| 10ms+ | | 325 |
| 100ms+ | | 17 |
| 1s+ | | 0 |
| 10s+ | | 0 |
+---------+--------------------------------+-------+
8 rows in set (0.08 sec)
Showing distribution of query time from a run of mysqlslap
Generated with a quick CTE
over
events_statements_histogram_
global
Feature Request
from DBAs
Available on a per statement
digest level. Can quickly
aggregate top-N statements
with latency distribution.

MySQL as a Document Store – a FULL Stack
• Native JSON Datatype, Indexes on JSON Documents
• Native Collections with Key Value Semantics
• Interactive Shell “MySQL Shell” – Javascript, Python, SQL modes
• Connectors include NoSQL CRUD APIs
– Java, New NodeJS, NET, C++/C, PHP, Python
– Method Chaining and Pipelining
– Supports Combined Document and Relational
Store, Retrieve, Search and Managing JSON documents
18

JSON Functions
19
MySQL 5.7 and 8.0
JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
JSON_MERGE()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()
JSON_PRETTY()
JSON_STORAGE_SIZE()
JSON_STORAGE_FREE()
JSON_ARRAYAGG()
JSON_OBJECTAGG()

RDBMS & MySQL
Relational vs NoSQL databases

Relational Databases
Data Integrity
normalization
constraints (foreign keys, ...)
Atomicity, Consistency, Isolation, Durability
ACID compliant
transactions
SQL
powerfull query language

Schemaless
no schema design, no normalization, no foreign keys, no data types, ...
very quick initial development
Flexible data structure
embedded arrays or objects
valid solution when natural data can´t be
modeled optimally into a relational model
objects persistence without the use of any ORM - mapping object-oriented
JSON
close to frontend
native in JS
easy to learn
NoSQL or Document Store

How DBAs see data How Developers see data
{
"GNP" : 249704,
"Name" : "Belgium",
"government" : {
"GovernmentForm" :
"Constitutional Monarchy, Federation",
"HeadOfState" : "Philippe I "
} ,
"_id" : "BEL",
"IndepYear" : 1830,
"demographics" : {
"Population" : 10239000,
"LifeExpectancy" : 77.8000030517578
} ,
"geography" : {
"Region" : "Western Europe",
"SurfaceArea" : 30518,
"Continent" : "Europe"
}
}

DMBS or NoSQL ?
Why not both ?

The MySQL Document Store !
SQL is now optional !

A solution for all
34 / 112
Developers:
[✔] Schemaless or/and Schema
[✔] Rapid Prototyping/Simpler APIs
[✔] Document Model
[✔] Transactions
Operations:
[✔] Performance Management/Visibility
[✔] Robust Replication, Backup, Restore
[✔] Comprehensive Tooling Ecosystem
[✔] Simpler application schema upgrades
Business Owner:
[✔] Don’t lose my data = ACID transactions
[✔] Capture all my data = Extensible/Schemaless
[✔] Products On Schedule/Time to Market = Rapid Development

MySQL Document Store
the Solution
35 / 112

35 / 112
MySQL Document Store

X DevAPI Connectors for
C++, Java, .Net, Node.js, Python, PHP
working with Communities
New MySQL Shell
Command Completion
Python, JavaScript & SQL modes
Admin functions
New Util object
A new high-level session concept that can scale from
single MySQL Server to a multiple server environment
Non-blocking, asynchronous calls follow common
language pat erns
Supports CRUD operations
37 / 112

MySQL Documents and Collections
• Collections are containers for documents
– These documents share a purpose
– Possibly share one or more indexes
– Each collection has a unique name
– Exists within a single schema
• Within a Collection you can
– Add(), Find(), Modify(), and Remove() - JSON documents
• Collections can be
– Create(), List(), Drop()

Operation Document Relational
Create Collection.add() Table.insert()
Read Collection.find() Table.select()
Update Collection.modify() Table.update()
Delete Collection.remove() Table.delete()
• Use SQL, CRUD APIs – Document (NoSQL) and Relational (SQL), or “All of the Above”
– All of this is in addition to the Classic APIs
MySQL X DevAPI included in Shell and Connectors

It relies on the proven MySQL InnoDB’s strength & robustness:
• innodb_flush_log_at_trx_commit = 1
• innodb_doublewrite = ON
• sync_binlog = 1
• transaction_isolation = REPEATABLE-READ | READ-COMMITTED | ...
We do care about your data!
MySQL Document Store is Full ACID
ACID transactions = Don’t lose my data

MySQL Shell 8.0.11+
MySQL
Server 5.7
MySQL 8.0
Upgrade Checker
Prompt Themes
Auto Completion
&
Command History
MySQL
Server 8.0
Document Store
X DevAPI
InnoDB ClusterSQL CLI
Output Formats
(Table, JSON, Tabbed)
Batch Execution
JavaScript
Python
SQL

MySQL Document Store: Architecture

MySQL X devAPI connector for:
• Java
- https://insidemysql.com/connector-j-8-0-11-the-face-for-your-brand-new-document-oriented-database/
• .NET
- https://insidemysql.com/introducing-connector-net-with-full-support-for-mysql-8-0/
• Node.JS
- https://insidemysql.com/introducing-connector-node-js-for-mysql-8-0/
• C++
- https://insidemysql.com/what-is-new-in-connector-c-8-0/
• Python
- https://insidemysql.com/using-mysql-connector-python-8-0-with-mysql-8-0/
• PHP
- https://insidemysql.com/introducing-the-mysql-x-devapi-php-extension-for-mysql-8-0/
• ODBC
- https://insidemysql.com/what-is-new-in-connector-odbc-8-0/
https://insidemysql.com/mysql-8-0-welcome-to-the-devapi/
Write application using X DevAPI

what do I gain ?
Takeaways

This is the best of the two worlds in one product !
Data integrity
ACID Compliant
Transactions
SQL
schemaless
flexible data structure
easy to start (CRUD)

Resources
Topic Link(s)
MySQL as a Document Store https://dev.mysql.com/doc/refman/8.0/en/document-store.html
MySQL Shell User Guide https://dev.mysql.com/doc/mysql-shell-excerpt/8.0/en/index.html
MySQL Shell Documentation https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/
https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/
X Dev API User Guide https://dev.mysql.com/doc/x-devapi-userguide/en/
X Plugin https://dev.mysql.com/doc/refman/8.0/en/x-plugin.html
MySQL Engineering Blog https://insidemysql.com/mysql-8-0-welcome-to-the-devapi/
https://insidemysql.com/mysql-document-store-crud-quick-start/
MySQL JSON Data Type https://dev.mysql.com/doc/refman/8.0/en/json.html
Blogs http://dasini.net/blog/2015/11/17/30-mins-avec-json-en-mysql/
http://dasini.net/blog/2015/11/30/json-et-colonnes-generees-avec-mysql
http://mysqlserverteam.com/tag/json/
http://mysqlserverteam.com/category/docstore/
38

MySQL NoSQL Document Store

More Related Content

What's hot

What's hot (20)

Similar to MySQL NoSQL Document Store

Similar to MySQL NoSQL Document Store (20)

More from Mark Swarbrick

More from Mark Swarbrick (20)

Recently uploaded

Recently uploaded (20)

MySQL NoSQL Document Store