MySQL Performance Tuning: Top 10 Tips
64 likes55,775 views
This document provides tips for tuning a MySQL database to optimize performance. It discusses why tuning is important for cost effectiveness, performance, and competitive advantage. It outlines who should be involved in tuning including application designers, developers, DBAs and system administrators. The document covers what can be tuned such as applications, databases structures, and hardware. It provides best practices for when and how much to tune a database. Specific tuning techniques are discussed for various areas including application development, database design, server configuration, and storage engine optimizations.
MySQL Performance Tuning: Top 10 Tips
- 1. OSS Cube|OSI Days 2010 1 1 MySQL Performance Tuning: Top 10 Tips Presented by :- Sonali Minocha Osscube, New Delhi
- 2. OSS Cube|OSI Days 2010 2 2 Why Tune a Database? Cost-effectiveness o A system that is tuned can minimize the need to buy additional hardware and other resources to meet the needs of the end users. o Tuning may demonstrate that the system being used is excessive for the end users and downsizing is the better option. This may result in multiple levels of savings to include maintenance. · Performance o A high-performance, well-tuned system produces faster response time and better throughput within the organization. This increases the productivity of the end users. o A well-tuned system benefits the organization’s customers, poor response time causes lot of unhappiness and loses business. · Competitive Advantage o Tuning a system for optimal performance gives the end users the ability to glean more critical information faster than the competitors thus giving the company as a whole an advantage. o Tuning the access to the data helps business analysts, who are utilizing business intelligence initiatives based on corporate data, make faster and more precise decisions.
- 3. OSS Cube|OSI Days 2010 3 3 Who Tunes? • All persons involved with the MySQL software should be concerned with performance and involved in tuning. Consider all phases of the System Development Life Cycle (SDLC) as opportunities to create and enhance an effective, well designed and efficient system. • · Application Designers • · Application Developers • · Database Administrators • · System Administrators
- 4. OSS Cube|OSI Days 2010 4 4 What is Tuned? Careful design of systems and applications is essential to the optimal performance of any database. In most cases the greatest gain in performance can be achieved through tuning the application. The most opportune time to consider performance issues is when the application is in the very early stages of the SDLC. • · Application Design • · Application Development • · Database Structures • · Hardware
- 5. OSS Cube|OSI Days 2010 5 5 When should Tuning be accomplished? • · From the beginning • · Routine tune-ups • · Response to problems/bottlenecks • · Proactive Tuning • · End-of-life • o Bottom line, performance tuning is never finished. As long as the database system is in • place, the DBA will continually be adjusting the database architecture to meet the • continual needs of the end user. • Tuning Outcome • The effect of tuning should be readily apparent to the end user. For this reason, the tuning outcome • should be to: • · Minimize or decrease the response time • · Provide high throughput scalability at comparable response times • Tuning Targets • Tuning goals can be defined in numerous ways. In many cases the goals are set by the users of the • system in order for them to perform their daily functions efficiently. • · How much data needs to be processed? • · How quickly can the information be returned to the user? • · What is the total number of users that can efficiently access the data?
- 6. OSS Cube|OSI Days 2010 6 6 How much tuning is enough? • Too Little Tuning • Too Much Tuning • Other Questions to Consider • · Is the cost of improving the database architecture or adding additional resources cost effective for the return on investment that the data is providing to the organization? • · Are changes made to database systems that are in production? • · Is the integrity of the data being compromised for speed? • There comes a point when the system is in balance, and it is better not to adjust settings to achieve infinitesimally small performance improvements. • Caution is required when making changes once a system is in production. For best results, always try to build a test environment
- 7. OSS Cube|OSI Days 2010 7 7 Stages of Tuning • Application design • Application development • Database configuration • Application maintenance and growth • Troubleshooting
- 8. OSS Cube|OSI Days 2010 8 8 Measurement Methods • Benchmarking • Profiling • Benchmarking Vs Profiling
- 9. OSS Cube|OSI Days 2010 9 9 Application Development (Optimizing Queries)
- 10. OSS Cube|OSI Days 2010 10 10 Indexes • In MySQL there are several types of indexes: – Tree Indexes • B-Trees – FULLTEXT indexes (based on words instead of whole columns) • B+Trees (InnoDB) • T-Trees (NDB) • Red-black binary trees (MEMORY) • R-Trees (MyISAM, spatial indexes) – Hash indexes (MEMORY and NDB) • The use of indexes to find rows speedes up most queries • Writes become slower with each added index
- 11. OSS Cube|OSI Days 2010 11 11 Query Execution Plan (EXPLAIN) • With EXPLAIN the query is sent all the way to the optimizer, but not to the storage engine • Instead EXPLAIN returns the query execution plan • EXPLAIN tells you: – In which order the tables are read – What types of read operations that are made – Which indexes could have been used – Which indexes are used – How the tables refer to each other – How many rows the optimizer estimates to retrieve from each table
- 12. OSS Cube|OSI Days 2010 12 12 EXPLAIN Types system The table has only one row const At the most one matching row, treated as a constant eq_ref One row per row from previous tables ref Several rows with matching index value ref_or_null Like ref, plus NULL values index_merge Several index searches are merged unique_subquery Same as ref for some subqueries index_subquery As above for non-unique indexes range A range index scan index The whole index is scanned ALL A full table scan
- 13. OSS Cube|OSI Days 2010 13 13 EXPLAIN Extra Using index The result is created straight from the index Using where Not all rows are used in the result Distinct Only a single row is read per row combination Not exists A LEFT JOIN missing rows optimization is used Using filesort An extra row sorting step is done Using temporary A temporary table is used Range checked for each The read type is optimized individually for each combination of rows record from the previous tables
- 14. OSS Cube|OSI Days 2010 14 14 Optimizer Hints STRAIGHT_JOIN Forces the optimizer to join the tables in the given order SQL_BIG_RESULTS Together with GROUP BY or DISTINCT tells the server to use disk-based temp tables SQL_BUFFER_RESULTS Tells the server to use a temp table, thus releasing locks early (for table-locks) USE INDEX Hints to the optimizer to use the given index FORCE INDEX Forces the optimizer to use the index (if possible) IGNORE INDEX Forces the optimizer not the use the index
- 15. OSS Cube|OSI Days 2010 15 15 Selecting Queries to Optimize • The slow query log – Logs all queries that take longer than long_query_time – Can also log all queries that don’t use indexes with --log-queries-not-using-indexes – To log slow administrative commands use --log-slow-admin-statements – To analyze the contents of the slow log use mysqldumpslow • The general query log can be use to analyze: – Reads vs. writes – Simple queries vs. complex queries – etc
- 16. OSS Cube|OSI Days 2010 16 16 Database Designing (Optimizing Schemas)
- 17. OSS Cube|OSI Days 2010 17 17 Normalization • Normalization is a key factor in optimizing your database structure – Good normalization prevents redundant data from being stored in the same tables – By moving redundant data to their own table, this reduces storage requirements and overhead when processing queries – Transactional databases should be in the 3rd normal form • For data warehousing and reporting system a star-schema might be a better solution
- 18. OSS Cube|OSI Days 2010 18 18 Table Optimizations • Use columns that are as short as possible; – INT instead of BIGINT – VARCHAR(10) instead of VARCHAR(255) – etc. • Pay special attention to columns that are used in joins • Define columns as NOT NULL if possible • For hints on saving space, use PROCEDURE ANALYSE() • For data warehousing or reporting systems consider using summary tables for speed
- 19. OSS Cube|OSI Days 2010 19 19 Index Optimizations • An index on the whole column is not always necessary – Instead index just a prefix of a column – Prefix indexes take less space and the operations are faster • Composite indexes can be used for searches on the first column(s) in the index • Minimize the size of PRIMARY KEYs that are used as references in other tables – Using an auto_increment column can be more optimal • A FULLTEXT index is useful for – word searches in text – searches on several columns
- 20. OSS Cube|OSI Days 2010 20 20 MyISAM-Specific Optimizations • Consider which row format to use, dynamic, static or compressed – Speed vs. space • Consider splitting large tables into static and dynamic parts • Important to perform table maintenance operations regularly or after big DELETE/UPDATE operations – Especially on tables with dynamic row format • Change the row-pointer size (default 6b) for large (>256Tb) tables or smaller (< 4Gb) tables
- 21. OSS Cube|OSI Days 2010 21 21 InnoDB-Specific Optimizations • InnoDB uses clustered indexes – The length of the PRIMARY KEY is extremely important • The rows are always dynamic – Using VARCHAR instead of CHAR is almost always better • Maintenance operations needed after – Many UPDATE/DELETE operations • The pages can become underfilled
- 22. OSS Cube|OSI Days 2010 22 22 MEMORY-Specific Optimizations • Use BTREE (Red-black binary trees) indexes – When key duplication is high – When you need range searches • Set a size limit for your memory tables – With --max_heap_table_size • Remove unused memory – TRUNCATE TABLE to completely remove the contents of the table – A null ALTER TABLE to free up deleted rows
- 23. OSS Cube|OSI Days 2010 23 23 Optimizing the Server
- 24. OSS Cube|OSI Days 2010 24 24 Performance Monitoring • Server performance can be tracked using native OS tools – vmstat, iostat, mpstat on Unix – performance counters on Windows • The mysqld server tracks crucial performance counters – SHOW STATUS gives you a snapshot – Can use Cricket, SNMP, custom scripts to graph over time – MySQL Administrator • Default graphs • Allows you to create your own graphs • Queries can be tracked using log files – Can collect every query submitted to the server – Slow queries can be logged easily
- 25. OSS Cube|OSI Days 2010 25 25 Monitoring Threads in MySQL • To get a snapshot of all threads in MySQL – SHOW FULL PROCESSLIST – The state column shows what’s going on for each query • Performance problems can often be detected by – Monitoring the processlist – Verifying status variables • Imminent problems can be eliminated by – Terminating runaway or unnecessary threads with KILL
- 26. OSS Cube|OSI Days 2010 26 26 Tuning MySQL Parameters • Some MySQL options can be changed online • The dynamic options are either – SESSION specific • Changing the value will only affect the current connection – GLOBAL • Changing the value will affect the whole server – Both • When changing the value SESSION/GLOBAL should be specified • Online changes are not persistant over a server restart – The configuration files have to be changed as well • The current values of all options can be found with SHOW SESSION/GLOBAL VARIABLES
- 27. OSS Cube|OSI Days 2010 27 27 Status Variables • MySQL collects lots of status indicators – These can be monitored with SHOW STATUS • The variables provide a way of monitoring the server activity • They also act as guides when optimizing the server • The variables can also be viewed with – mysqladmin extended-status – MySQL Administrator • Provides graphical interface for monitoring the variables • Can be very efficient for tracking the health of the server
- 28. OSS Cube|OSI Days 2010 28 28 Some Global Options • table_cache (default 64) – Cache for storing open table handlers – Increase this if Opened_tables is high • thread_cache (default 0) – Number of threads to keep for reuse – Increase if threads_created is high – Not useful if the client uses connection pooling • max_connections (default 100) – The maximum allowed number of simultaneous connections – Very important for tuning thread specific memory areas – Each connection uses at least thread_stack of memory
- 29. OSS Cube|OSI Days 2010 29 29 MyISAM Global Options • key_buffer_size (default 8Mb) – Cache for storing indices – Increase this to get better index handling – Miss ratio (key_reads/key_read_requests) should be very low, at least < 0.03 (often < 0.01 is desirable) • Row caching is handled by the OS
- 30. OSS Cube|OSI Days 2010 30 30 MyISAM Thread-Specific Options • myisam_sort_buffer_size (default 8Mb) – Used when sorting indexes during REPAIR/ALTER TABLE • myisam_repair_threads (default 1) – Used for bulk import and repairing – Allows for repairing indexes in multiple threads • myisam_max_sort_file_size – The max size of the file used while re-creating indexes
- 31. OSS Cube|OSI Days 2010 31 31 InnoDB-Specific Optimization 1/2 • innodb_buffer_pool_size (default 8Mb) – The memory buffer InnoDB uses to cache both data and indexes – The bigger you set this the less disk i/o is needed – Can be set very high (up to 80% on a dedicated system) • innodb_flush_log_at_trx_commit (default 1) – 0 writes and sync’s once per second (not ACID) – 1 forces sync to disk after every commit – 2 write to disk every commit but only sync’s about once per second
- 32. OSS Cube|OSI Days 2010 32 32 InnoDB-Specific Optimization 2/2 • innodb_log_buffer_size (default 1Mb) – Larger values allows for larger transactions to be logged in memory – Sensible values range from 1M to 8M • innodb_log_file_size (default 5Mb) – Size of each InnoDB redo log file – Can be set up to buffer_pool_size
- 33. OSS Cube|OSI Days 2010 33 33 Q&A
- 34. OSS Cube|OSI Days 2010 34 34 Thank you