Whats New in Mysql 5.7 Geir Høydalsvik, Sr. Director, Mysql Engineering

Whats New in MySQL 5.
7
Geir Hydalsvik, Sr. Director, MySQL Engineering
Simon Mudd, Senior DBA, Booking.com
Copyright 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracles products remains at the sole discretion of Oracle.
Agenda
First Geir will talk about Features
Highlights of 5.7 Improvements and Additions
Next Simon will provide Bookings Perspective

Upgrading and Putting 5.7 into production
Copyright 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Internal/Restricted/Highly Restricted
Benefit Highlights of MySQL 5.7

Faster
Server quicker queries, more IO, more connections
InnoDB engine, Faster Connect/Disconnect, More Optimized Queries
No need to use MyISAM can have speed and consistency both now
Replication scales higher, faster slave processing

Parallel replications (parallel replication) order of magnitude faster(?)
GIS searching quickly search geographic data

Native InnoDB Spatial indexes
Compression
Leverages new advanced Operating File System capabilities
Improves performance, saves space
Benefit Highlights of MySQL 5.7 (cont)

Easier
Easier Configurability dynamic online changes
More Flexible
Native JSON support datatype, binary storage format
Virtual Columns/Functional indexes index any data and find quickly
Simpler to Manage
More instruments, Easy access to data using SYS schema
More Secure
By default and more options to secure the server
MySQL 5.7 Sysbench Benchmark: SQL Point Selects / sec

3x Faster than MySQL 5.6
1,600,000 QPS
MySQL 5.7: Sysbench OLTP Read Only (SQL Point Selects)

1 800 000
Queries per Second
1 600 000
MySQL 5.7
1 400 000
1 200 000
1 000 000
MySQL 5.6
800 000
600 000
400 000
200 000
MySQL 5.5
0
8
16
32
64
128
Connections
256
512
1 024
Intel(R) Xeon(R) CPU E7-8890 v3

4 sockets x 18 cores-HT (144 CPU threads)
2.5 Ghz, 512GB RAM
Linux kernel 3.16
6
MySQL 5.7 Sysbench Benchmark: Mixed OLTP Read Only

Near 1M QPS
MySQL 5.7: Sysbench OLTP Read Only (Mixed)

1 200 000
Queries per Second
1 000 000
MySQL 5.7
800 000
600 000
MySQL 5.6
400 000
200 000
MySQL 5.5
0
8
16
32
64
128
Connections
256
512
1 024

2.5 Ghz, 512GB RAM
Linux kernel 3.16
7
MySQL 5.7 Sysbench Benchmark: Connect / sec

82% Faster than MySQL 5.6
100K Connect / Sec
MySQL 5.7: Sysbench OLTP Read Only (Connect)
120 000
Connect / sec
100 000
MySQL 5.7
80 000
MySQL 5.6
60 000
40 000
MySQL 5.5
20 000
0
8
16
32
64
128
Connections
256
512
1 024

2.5 Ghz, 512GB RAM
Linux kernel 3.16
8
MySQL 5.7 Sysbench Benchmark: Complex OLTP Read Write

1.5x Faster than MySQL 5.6
MySQL 5.7: Sysbench OLTP Read Write (Complex)
700 000
Queries per Second
600 000
MySQL 5.7
500 000
400 000
MySQL 5.6
300 000
200 000
MySQL 5.5
100 000
0
8
16
32
64
128
Connections
256
512
1 024

2.5 Ghz, 512GB RAM
Linux kernel 3.16
9
MySQL 5.7: JSON Overview

Native JSON data type
Native internal binary format for efficient processing & storage
Built-in JSON functions

Allowing you to efficiently store, search, update, and manipulate Documents
JSON Comparator
Allows for easy integration of Document data within your SQL queries
Indexing of Documents using Generated Columns

InnoDB supports indexes on both stored and virtual Generated Columns
New expression analyzer automatically uses the best functional index available
10
MySQL 5.7: Optimizer Improvements

Queries execute faster, while using less CPU and disk space!
Optimizer and Parser refactoring
New Cost based Optimizer
Cleanly separate the parsing, optimizing, and

execution stages
New hint framework

Easier to manage
With support for additional new hints
Improved JSON EXPLAIN
EXPLAIN for running thread

Generated Columns
Configurable and tunable

mysql.server_cost and mysql.engine_cost tables
API for determining where data resides: on disk or in
cache
Support for InnoDB based internal temp

tables
SQL Standard compliant
ONLY_FULL_GROUP_BY mode
is ON by default
Many specific new optimizations
11
MySQL 5.7: Query Rewrite Plugin

New pre and post parse query rewrite APIs
Users can write their own plug-ins
Provides a post-parse query plugin

Rewrite problematic queries without the need to make application changes
Add hints
Modify join order
Many more
Improve problematic queries from ORMs, third party apps, etc

Eliminates many legacy use cases for proxies
12
MySQL 5.7: Performance Schema

Memory Instrumentation
Statement Instrumentation
Additional Information
Aggregates statistics by
Stored Procedures
Replication slave status
Stored Functions
MDL lock instrumentation
Prepared Statements
Status and variables per

thread
Type of memory used

(caches, internal buffers, )
Thread/account/user/host
indirectly performing the
memory operation
Attributes include
Memory used (bytes)
Operation counts
High/Low Water Marks
Transactions
Server stage tracking
Track long running SQL

Improved configuration
and ease-of-use
All while reducing total

footprint and overhead
13
MySQL 5.7: SYS Schema

SYS views make it easier to
- Monitor server health, user, host statistics

- Spot, diagnose, and tune performance issues
Provides insights into
- IO hot spots, Locking, Costly SQL statements
- Schema, table and index statistics
SYS is similar to
- Oracle V$ catalog views
- Microsoft SQL DMVs (Dynamic Mgmnt Views)
14
MySQL 5.7: GIS Improvements

Replaced custom code with Boost.Geometry
For spatial calculations

For spatial analysis
Enabling full OGC compliance
Were also Boost.Geometry contributors!
InnoDB R-tree based spatial indexes

Full ACID, MVCC, & transactional support
Index records contain minimum bounding box
GeoHash
GeoJSON
Helper functions such as ST_Distance_Sphere() and ST_MakeEnvelope()
15
MySQL 5.7: InnoDB Improvements

Native Partitioning
Eliminates previous limitations
Eliminates resource usage problems
Transportable tablespace support
Native Full-Text Search

Including full CJK support!
Native Spatial Indexes

Transparent page compression
Support for 32K and 64K pages
Use with transparent page compression for
very high compression ratios
General TABLESPACE support

Store multiple tables in user defined shared
tablespaces
Support for MySQL Group Replication

High priority transactions
Improved support for cache preloading

Load your hottest data loaded at startup
Configurable fill-factor
Allows for improvements in storage footprint
Improved bulk-data load performance

Resize the InnoDB Buffer Pool online
16
MySQL 5.7: Security Improvements

AES 256 Encryption now the default
SSL
Password rotation policies

Can be set globally, and at the user level
Deployment: enable secure unattended

install by default
Random password set on install
Remove anonymous accounts
Deployment without test account, schema,
demo files
Enabled by default
Auto-detection of existing keys and certs
Auto generation of keys and certs when needed
New helper utility: mysql_ssl_rsa_setup
New --require_secure_transport option to
prevent insecure communications
Added SSL support to binary log clients
Extended Proxy User Support
Easier instance initialization and setup:

mysqld initialize
New detection and support for systemd
Added Built-in Authentication Plugins support

for Proxy Users
Allows multiple users to share a single set of
managed privileges
17
MySQL 5.7: Replication Improvements

8-10x Faster slave throughput
GTID enhancements
On-line, phased deployment of GTIDs
Binary logging on slave now optional
Enhanced Semi-synchronous replication

Write guaranteed to be received by slave
before being observed by clients of the master
Option to wait on Acks from multiple slaves
Multi-Source Replication
Consolidate updates from multiple Masters
into one Slave
Dynamic slave filters
Often removes slave as a bottleneck; keep pace

with master with 8+ slave threads
Option to preserve Commit order
Automatic slave transaction retries
250%
Slave throughput vs. 96 Thread Master

200%
150%
100%
50%
0%
1
24
48
Slave Threads
18
- Perspectives on MySQL 5.7

- Upgrading to MySQL 5.7
Simon Mudd, Senior DBA
19
About Booking.com
Booking.com is part of the Priceline Group
World leader in booking accommodation online

Established in 1996 and based in Amsterdam
Over 801,000 properties in 221 countries
42 languages (website and customer service)
170 offices worldwide
We Use MySQL:
Thousands of servers: ~85% replicating

Over 110 masters: ~25 have 50+ slaves, ~8 have 100+ slaves
20
What we look for when upgrading to a new major version?

Basically motivation for upgrades:
Longer uptime and more dynamic configuration (avoid mysqld restart)
Better performance related to
Replication
Concurrency
Better stability under load
Seamless upgrade path from existing 5.6 setup to using 5.7

More features and flexible configuration
Incorporation of features requested while using 5.6
Better visibility of what MySQL is doing and where it is busy
21
What does 5.7 provide?

Specific Features of Interest:
Parallel Replication
GTID migration possible on live system and no binlogs needed on slaves
Dynamic InnoDB buffer pool size reconfiguration
Dynamic replication configuration
Improved cache preloading on startup (configurable percentage to load)
InnoDB Fill-size should be good for statistics tables which are write-once
Loss-less semi-sync
Plus others mentioned previously
22
Why booking.com tests MySQL 5.7 DMR releases

Testing of DMR releases has allowed us to:
Preview potential new features and give feedback

Check performance meets expectations
Catch unexpected behaviour early
Resources required to do this are not significant
23
How to upgrade - in pictures

+---+
Master
| M |
+---+
|
+----+---+---------+
|
|
|
+----+ +----+
+----+
Slave
| s1 | |Slave
s2
|
...
|
1
2sn |
+----+ +----+
+----+
Slave
N
Everyone starts this way

All same version of MySQL (say 5.6.27)
24
How to upgrade - in pictures start with slaves

Master
MySQL 5.7
Slave
1
Slave
2
Slave
N
Upgrade slaves
25
Upgrading MySQL - upgrade steps

If more than 1 master and slave the procedure is basically the same.
1. Make a new slave or upgrade an existing one
2. Configure /etc/my.cnf for 5.7
3. Change binaries
4. Start mysqld with replication stopped
5. Run: mysql_upgrade
6. Restart mysqld normally
7. Test
26
How to upgrade - in pictures Final Steps

1. Move slaves under
5.7 intermediate
master
Master
2. Move application
writes to 5.7
master
Intermediate
Master
MySQL 5.7
Slave 1
MySQL 5.7
Slave 2
MySQL 5.7
Slave 3
MySQL 5.7
Slave N
MySQL 5.7
27
What are the Risks?

For initial testing risks are minimal.
If something breaks:
throw it away, fix internal settings or

report a bug
With Cloud infrastructure it is easy to build a spare & test for a few hours
As you continue testing MySQL 5.7
You control the rate of progress of deploying 5.7 slaves
Easy to back out until right at the end (with 5.7 intermediate master)
No application downtime except for the final move when you write to the
new 5.7 master.
28
Moving MySQL 5.7 into production observations

The setup of a new environment is simpler
the new defaults require less cleanup
Configuration changes in /etc/my.cnf

Few required changes
log_syslog
innodb_buffer_pool_dump_pct
show_compatibility_56
Pay attention to changes in default values between 5.6 and 5.7

We have had to add some extra settings
Upgrading to 5.7 vs previous upgrades
Slight changes in procedures but generally they are simpler

Just run mysql_upgrade, no need for --skip-grant-tables
29
Moving MySQL 5.7 into production observations (cont.)

Grant management
Procedures need to be MySQL version aware (new disable accounts vs old change
password)
New ALTER USER is replication safe, but cant use on master until all servers run 5.7
Improve monitoring
Add more checks
Collect 5.7 specific metrics from performance_schema / sys
Replication
Memory usage
Locks
30
Results of our MySQL 5.7 DMR testing

2 years following the evolution of MySQL 5.7 DMR
First installed in 06/2013 (5.7.1_m11)
We provided feedback on several issues, e.g.
SQL_MODE changes caused problems with replication (5.6 to 5.7)

ALTER, CREATE, DROP USER were not replication safe (5.6 to 5.7)
Unexpected crashes on startup in some early releases of 5.7
Parallel Replication
MySQL engineering responded to these issues
No longer an issue in the recently announced GA version

Avoided potential breakage in a GA version where functional changes would have
been impossible/undesirable
Now 5.7 is GA we are in a good state to start to roll out more aggressively
31
MySQL Community Reception @ Oracle OpenWorld

Celebrate, Have Fun and Mingle with Oracles MySQL Engineers & Your Peers
Tuesday, October 27: 7 pm
Jillians at Metreon: 175 Fourth Street, San Francisco

At the corner of Howard and 4th st.; only 2-min walk from Moscone Center (same
place as last year)
Join us!
32
Oracle Sessions
Today, Tues Oct 27
CON3716 - InnoDB: What's new in 5.7
Sunny Bains | Room 254 at 4.00 pm
CON2353 - MySQL Server Performance Tuning 101

Ligaya Isler-turmelle | Room 262
CON2257 - MySQL Sys Schema Deep Dive - Mark Leith
CON5097 - An Overview Of The New Replication Features in MySQL 5.7 and Beyond!
Luis Soares | Room 254 at 5:15 pm
CON4043 - MySQL Performance: Demystified Tuning & Best Practices - Dimitri Kravtchuk
33
Oracle Sessions
Tomorrow, Weds 0ct 28
CON3379 - MySQL 5.7: What is new in optimizer?
Manyi Lu | Room 262 at 12.15 pm
CON2131 - JSON support in MySQL 5.7

Evgeny Potemkin | Room 262 at 1.30 pm
CON2174 - MySQL: Support Virtual Column and Effective Functional Index in InnoDB
Jimmy Yang 245 | Room 262 at 2.45 pm
CON4975 - Scalable High-Availability Systems using MySQL Fabric

Mats Kindahl | Room 262 at 4.00 pm
34
Oracle Sessions
Thurs Oct 29
CON5349 - Update Everywhere With MySQL Group Replication
Nuno Carvalho | Room 250 at 9.30 am
CON4507 - Oracle Enterprise Manager for MySQL Database Latest Features

Carlos Proal Aguilar | Room 262 at 10.45 am
CON4062 - Securing your MySQL Environment with MySQL Enterprise Monitor

Mark Matthews | Room 262 at 12.00 pm
CON1559 - What's New in MySQL 5.7 Security

Georgi Kodinov | Room 262 at 1.15 pm
CON2625 - OpenStack and MySQL

Matt Lord | Room 262 at 1.15 pm
35
Booking.com Sessions
CON4098 - Binlog Servers at Booking.com
Jean-Francois Gagne | Room 250 Tuesday at 4.00 pm
BOF5528 - Riding the Binary Logs: Forthcoming Evolution in the

Replication Stream
Jean-Francois Gagne | Moscone South262 Tuesday at 6.15 pm
CON4073 - The Scalability Axes of MySQL Architectures

Nicolai Plum (Booking) | Room 250 Wednesday at 2.45 pm
36
MySQL Central @ OpenWorld

October 25-29, 2015 | San Francisco
Keynotes
Conferences Sessions
Birds-of-a-feather sessions
Tutorials
Hands-on Labs
Demos
Receptions & Customer Appreciation Event
Access to Oracle OpenWorld Extensive Content
37

Whats New in Mysql 5.7 Geir Høydalsvik, Sr. Director, Mysql Engineering

Uploaded by

Copyright:

Available Formats

You might also like

Whats New in Mysql 5.7 Geir Høydalsvik, Sr. Director, Mysql Engineering

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Whats New in Mysql 5.7 Geir Høydalsvik, Sr. Director, Mysql Engineering

Uploaded by

Copyright:

Available Formats

Whats New in MySQL 5.

Safe Harbor Statement

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

Next Simon will provide Bookings Perspective

Benefit Highlights of MySQL 5.7

Replication scales higher, faster slave processing

GIS searching quickly search geographic data

Benefit Highlights of MySQL 5.7 (cont)

MySQL 5.7 Sysbench Benchmark: SQL Point Selects / sec

MySQL 5.7: Sysbench OLTP Read Only (SQL Point Selects)

Queries per Second

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

Intel(R) Xeon(R) CPU E7-8890 v3

MySQL 5.7 Sysbench Benchmark: Mixed OLTP Read Only

MySQL 5.7: Sysbench OLTP Read Only (Mixed)

Queries per Second

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

Intel(R) Xeon(R) CPU E7-8890 v3

MySQL 5.7 Sysbench Benchmark: Connect / sec

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

Intel(R) Xeon(R) CPU E7-8890 v3

MySQL 5.7 Sysbench Benchmark: Complex OLTP Read Write

Queries per Second

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

Intel(R) Xeon(R) CPU E7-8890 v3

MySQL 5.7: JSON Overview

Built-in JSON functions

Indexing of Documents using Generated Columns

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

MySQL 5.7: Optimizer Improvements

New Cost based Optimizer

Cleanly separate the parsing, optimizing, and

New hint framework

Improved JSON EXPLAIN

EXPLAIN for running thread

Configurable and tunable

Support for InnoDB based internal temp

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

MySQL 5.7: Query Rewrite Plugin

Provides a post-parse query plugin

Improve problematic queries from ORMs, third party apps, etc

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

MySQL 5.7: Performance Schema

Replication slave status

MDL lock instrumentation

Status and variables per

Type of memory used

Server stage tracking

Track long running SQL

All while reducing total

MySQL 5.7: SYS Schema

- Monitor server health, user, host statistics

Copyright 2015, Oracle and/or its affiliates. All rights reserved. |

MySQL 5.7: GIS Improvements

For spatial calculations

InnoDB R-tree based spatial indexes

MySQL 5.7: InnoDB Improvements