Top 10 Tips For Optimizing SQL Server Performance
Top 10 Tips For Optimizing SQL Server Performance
Written by
Kevin Kline Technical Strategy Manager for SQL Server Solutions Quest Software
WHITE PAPER
Contents
Introduction ..................................................................................................................................................................... 2
10. What Problem Are We Trying to Solve ..................................................................................................................... 3
What Are Baselining and Benchmarking? ......................................................................................................................................... 3
What Baselining Cant Do .................................................................................................................................................................. 4
8. Why Changing Sp_Configure Settings Probably Wont Help ..................................................................................... 8
7. I Have a Bottleneck What Do I Do Now? ................................................................................................................. 9
6. SQL Profiler Is Your Friend ...................................................................................................................................... 11
Start a Trace .................................................................................................................................................................................... 11
5. Zen and the Art of Negotiating with Your SAN Administrator .................................................................................... 15
4. The Horror of Cursors (and Other Bad T-SQL) ........................................................................................................ 17
3. Plan Reuse Recycling for SQL .............................................................................................................................. 19
2. The Mystery of the Buffer Cache .............................................................................................................................. 21
1. The Tao of Indexes ................................................................................................................................................... 23
sys.dm_db_index_operational_stats ................................................................................................................................................ 23
sys.dm_db_index_usage_stats ........................................................................................................................................................ 24
Introduction
Performance optimization on SQL Server is difficult. A vast array of information exists on how to address performance problems in general. However, there is not much detailed information on specific issues and even less information on how to apply that specific knowledge to your own environment. In this white paper, Ill discuss the 10 things I think you should know about SQL Server performance. Each item is a nugget of practical knowledge that you can immediately apply to your environment.
For the period of a few days, we will sample the level of the fuel in the fuel tank and plot it in a graph shown below.
The plot displays the fuel remaining in the fuel tank over time. We can see that the baseline behavior represents the level of fuel in the tank that decreases slowly at first and then starts to accelerate more quickly toward the end of the measured time period. In general, this is the normal behavior for fuel in a fuel tank over time.
Assuming this graph represents normal behavior, we can measure and plot a different behavior and compare the two graphs. We would easily see the change in behavior. Emerging trends may also be easily identified since we can plot against time. A baseline cannot, however, provide any qualitative measure of efficiency. From the chart above, you cannot draw any conclusions about how efficient the car is you must investigate elsewhere for this information. The baseline can tell you only whether you used more (or less) fuel between two days. Similarly for SQL Server, a baseline can tell you only that something is outside that range of normally observed behavior. It cannot tell you whether the server is running as efficiently as possible. The point is that you should not start with baselining. You need to make sure that your application workload is running as efficiently as possible. Once performance has been optimized, you can then take a baseline. Also, you cannot simply stop with baselining. You should keep your application running as efficiently as possible and use your baseline as an early warning system that can alert you when performance starts to degrade.
Operational Monitoring
Operational monitoring checks for general resource usage. It helps answer questions like: Is the server about to run out of resources like CPU, disk space, or memory? Are the data files able to grow? Do fixed-size data files have enough free space for data? You could also collect data for trending purposes. A good example would be collecting the sizes of all the data files. From this information, you could trend the data file growth rates. This would allow you to more easily forecast the resource requirements you may have in the future. To answer the three questions posed above, you should look at the following counters:
Bottleneck Monitoring
Bottleneck monitoring focuses more on performance-related matters. The data you collect helps answer questions such as: Is there a CPU bottleneck? Is there an I/O bottleneck? Are the major SQL Server subsystems, such as the Buffer Cache and Procedure Cache, healthy? Do we have contention in the database? To answer these questions, we would look at the following counters:
It is a similar story for an I/O bottleneck: SELECT TOP 50 (total_logical_reads + total_logical_writes) AS total_logical_io, (total_logical_reads / execution_count) AS avg_logical_reads, (total_logical_writes / execution_count) AS avg_logical_writes, (total_physical_reads / execution_count) AS avg_phys_reads, substring( st.text, (qs.statement_start_offset / 2) + 1, ((CASE qs.statement_end_offset WHEN -1 THEN datalength(st.text) ELSE qs.statement_end_offset END qs.statement_start_offset) / 2) + 1) AS statement_text, * FROM sys.dm_exec_query_stats AS qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS st ORDER BY total_logical_io DESC
10
Start a Trace
The goal is to classify workload, so I have chosen these four SQL-related events: RPC:Completed SP:Completed SQL:BatchCompleted SQL:StmtCompleted
Figure 1 shows the Trace Properties Dialog. I have also chosen all possible columns for each of these event types.
[Figure 1.]
11
Figure 2 shows the General tab in the same dialog. I have configured the trace to store into a table on a server other than the server I am tracing. I have also configured the trace to stop after an hour.
[Figure 2.] Once the trace is finished, the data should now be available in the database table that I configured. For those who wish to use server side tracing, we will also assume from this point that the trace data now exists in a table. On a server with a large amount of throughput, there will be a large number of rows in the trace table. In order to make sense of all this data, it will be necessary to aggregate. I suggest aggregating by at least the SQL text or TextData column. You can include other columns in your aggregation, such as user or client host name, but for now I will concentrate on TextData. TextData is a text column, which means I cant do a GROUP BY on it. So I will convert it to something we can do a GROUP BY on. In order to do this, I will create a column on the trace table called TextDataTrunc. Figure 3 illustrates the populating of this column with a simple UPDATE.
To get a more accurate aggregation, it would be better to process the TextData column and replace the parameters and literals with some token that allows the SQL statements to be hashed. The hash could then be stored, and the aggregation could be performed on the hash value. This could be done with a C# user-defined function on SQL Server 2005. Illustrating how to do this is beyond the scope of this paper, so I am using the quick and dirty method.
12
[Figure 3.] Once the UPDATE is complete, I can query the table to get the data we require. For example, say you wanted to know the SQL that had been executed the most I could use a simple query: SELECT TextDataTrunc, COUNT(TextDataTrunc) AS ExecCount, SUM(cpu) AS TotalCPU Avg(cpu) AS AvgCPU FROM dbo.tracedata WHERE EventClass = 12 GROUP BY TextDataTrunc ORDER BY ExecCount DESC Figure 4 shows an example of this:
13
[Figure 4.] The values to use for the EventClass column can be found in SQL Server books online under the topic sp_trace_setevent.
14
15
[Figure 5.] Using these numbers, you can quickly narrow down which files are responsible for consuming I/O bandwidth and ask questions such as: Is this I/O necessary? Am I missing an index? Is it one table or index in a file that is responsible? Can I put this index or table in another file on another volume?
16
WHILE @@FETCH_STATUS = 0 BEGIN UPDATE Patient SET ConfirmFlag = 'N' WHERE CURRENT OF PatientConfirmRec
CLOSE PatientConfirmRec DEALLOCATE PatientConfirmRec This is real code in a real production system. It can actually be reduced to: UPDATE Patient SET ConfirmFlag = 'N' WHERE PolicyGUID = @PolicyGUID
17
This refactored code will, of course, run much more efficiently, allow the optimizer to work its magic, and take far less CPU time. In addition, it will be far easier to maintain. Its important to schedule a code review of the T-SQL in your applications, both stored code and client side, and to try to refactor such nonsense. Bad T-SQL can also appear as inefficient queries that do not use indexes, mostly because the index is incorrect or missing. Its important to learn how to tune queries using query plans in SQL Server Management Studio. Figure 6 shows an example of a large query plan:
[Figure 6] A detailed discussion of query tuning using query plans is beyond the scope of this white paper. However, the simplest way to start this process is by turning SCAN operations into SEEKs. SCANs will read every row in the table. For large tables, it is expensive in terms of I/O, whereas a SEEK will use an index to go straight to the required row. This, of course, requires an index to use, so if you find SCANs in your workload, you could be missing indexes. There are a number of good books on this topic, including: Professional SQL Server Execution Plan Tuning by Grant Fritchey SQL Server Internals & Troubleshooting (paperback) by Christian Bolton, Brent Ozar, Justin Langford, James Rowland-Jones, Jonathan Kehayias, Cindy Gross, and Steven Wort
18
[Figure 7] Figure 7 shows two code examples. Though they are contrived, they illustrate the difference between building a statement through string concatenation and using prepared statements with parameters.
19
SQL Server cannot reuse the plan from the bad example. If a parameter had been a string type, this function could be used to mount a SQL injection attack. The good example is not susceptible to a SQL injection attack because a parameter is used, and SQL Server is able to reuse the plan.
20
21
SELECT o.name, i.name, bd.* FROM sys.dm_os_buffer_descriptors bd INNER JOIN sys.allocation_units a ON bd.allocation_unit_id = a.allocation_unit_id INNER JOIN sys.partitions p ON (a.container_id = p.hobt_id AND a.type IN (1, 3)) OR (a.container_id = p.partition_id AND a.type = 2) INNER JOIN sys.objects o ON p.object_id = o.object_id INNER JOIN sys.indexes i ON p.object_id = i.object_id AND p.index_id = i.index_id You can also use the new index DMVs to find out which tables/indexes have large amounts of physical I/O.
22
sys.dm_db_index_operational_stats
sys.dm_db_index_operational_stats contains information on current low-level I/O, locking, latching, and access method activity for each index. Use this DMV to answer the following questions: Do I have a hot index? Do I have an index on which there is contention? The row lock wait in ms/page lock wait in ms columns can tell us whether there have been waits on this index. Do I have an index that is being used inefficiently? Which indexes are currently I/O bottlenecks? The page_io_latch_wait_ms column can tell us whether there have been I/O waits while bringing index pages into the buffer cache a good indicator that there is a scan access pattern. What sort of access patterns are in use? The range_scan_count and singleton_lookup_count columns can tell us what sort of access patterns are used on a particular index.
[Figure 8. ] Figure 8 illustrates the output of a query that lists indexes by the total PAGE_IO_LATCH wait. This is very useful when trying to determine which indexes are involved in I/O bottlenecks.
23
sys.dm_db_index_usage_stats
sys.dm_db_index_usage_stats contains counts of different types of index operations and the time each type of operation was last performed. Use this DMV to answer the following questions: How are users using the indexes? The user_seeks, user_scans, user_lookups columns can tell you the types and significance of user operations against indexes. What is the cost of an index? The user_updates column can tell you what the level of maintenance is for an index. When was an index last used? The last_* columns can tell you the last time an operation occurred on an index.
[Figure 9.] Figure 9 illustrates the output of a query that lists indexes by the total number of user_seeks. If you instead wanted to identify indexes that had a high proportion of scans, you could order by the user_scans column. Now that you have an index name, wouldnt it be good if you could find out what SQL statements used that index? On SQL Server 2005 and newer versions, you can.
24
Conclusion
On reflection, there are far more than 10 things you should know about SQL Server performance. However, this white paper offers a good starting point and some practical tips about performance optimization that you can apply to your SQL Server environment. So, remember these 10 things when optimizing SQL Server performance: 10. Benchmarking facilitates comparisons of workload behavior and lets you spot abnormal behavior because you have a good indication of what normal behavior is. 9. Performance Counters give you quick and useful information about currently running operations. 8. Changing server settings usually yields limited returns. 7. DMVs help you identify performance bottlenecks quickly. 6. Learn to use SQL Profiler and traces. 5. SANs are more than just I/O. 4. Cursors and other bad T-SQL frequently return to haunt applications. 3. Maximize plan reuse for better SQL Server caching. 2. Learn how to read the SQL Server buffer cache and how to minimize cache thrashing. And the number one tip for optimizing SQL Server performance: 1. Master indexing by learning how indexes are used and how to counteract the characteristics of bad indexes.
25
26
2010 Quest Software, Inc. ALL RIGHTS RESERVED. This document contains proprietary information protected by copyright. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose without the written permission of Quest Software, Inc. (Quest). The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document. If you have any questions regarding your potential use of this material, contact: Quest Software World Headquarters LEGAL Dept 5 Polaris Way Aliso Viejo, CA 92656 www.quest.com email: legal@quest.com Refer to our Web site for regional and international office information.
Trademarks
Quest, Quest Software, the Quest Software logo, AccessManager, ActiveRoles, Aelita, Akonix, AppAssure, Benchmark Factory, Big Brother, BridgeAccess, BridgeAutoEscalate, BridgeSearch, BridgeTrak, BusinessInsight, ChangeAuditor, ChangeManager, Defender, DeployDirector, Desktop Authority, DirectoryAnalyzer, DirectoryTroubleshooter, DS Analyzer, DS Expert, Foglight, GPOADmin, Help Desk Authority, Imceda, IntelliProfile, InTrust, Invirtus, iToken, I/Watch, JClass, Jint, JProbe, LeccoTech, LiteSpeed, LiveReorg, LogADmin, MessageStats, Monosphere, MultSess, NBSpool, NetBase, NetControl, Npulse, NetPro, PassGo, PerformaSure, Point,Click,Done!, PowerGUI, Quest Central, Quest vToolkit, Quest vWorkSpace, ReportADmin, RestoreADmin, ScriptLogic, Security Lifecycle Map, SelfServiceADmin, SharePlex, Sitraka, SmartAlarm, Spotlight, SQL Navigator, SQL Watch, SQLab, Stat, StealthCollect, Storage Horizon, Tag and Follow, Toad, T.O.A.D., Toad World, vAutomator, vControl, vConverter, vFoglight, vOptimizer, vRanger, Vintela, Virtual DBA, VizionCore, Vizioncore vAutomation Suite, Vizioncore vBackup, Vizioncore vEssentials, Vizioncore vMigrator, Vizioncore vReplicator, WebDefender, Webthority, Xaffire, and XRT are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. Other trademarks and registered trademarks used in this guide are property of their respective owners. UpdatedApril, 2011
27
WHITE PAPER
800.306.9329 (United States and Canada) If you are located outside North America, you can find your local office information on our Web site.
EMAIL MAIL
sales@quest.com Quest Software, Inc. World Headquarters 5 Polaris Way Aliso Viejo, CA 92656 USA
5 Polaris Way, Aliso Viejo, CA 92656 | PHONE 800.306.9329 | WEB www.quest.com | EMAIL sales@quest.com
If you are located outside North America, you can find local office information on our Web site.
2011 Quest Software, Inc. ALL RIGHTS RESERVED. Quest, Quest Software, the Quest Software logo [and product name] are registered trademarks of Quest Software, Inc. in the U.S.A. and/or other countries. All other trademarks and registered trademarks are property of their respective owners. WPD-Optimize-SQL-ServerPerf-US-KS-20110407