Skip to content

Find Most Executed Stored Procedures

An important step in optimizing a system is to take a holistic approach and look at stored procedures that are called very often. These procedures can often be the backbone of a system. Sometimes optimizing these procedures can have a dramatic effect on relieving the physical bottlenecks of the system and improving end-user experience.

The following DMV query shows the execution count of each stored procedure, sorted by the most executed procedures first.

SELECT
	DatabaseName		= DB_NAME(st.dbid)
	,SchemaName			= OBJECT_SCHEMA_NAME(st.objectid,dbid)
	,StoredProcedure	= OBJECT_NAME(st.objectid,dbid)
	,ExecutionCount		= MAX(cp.usecounts)
FROM sys.dm_exec_cached_plans cp
CROSS APPLY sys.dm_exec_sql_text(cp.plan_handle) st
WHERE DB_NAME(st.dbid) IS NOT NULL
AND cp.objtype = 'proc'
GROUP BY
	cp.plan_handle
	,DB_NAME(st.dbid)
	,OBJECT_SCHEMA_NAME(objectid,st.dbid)
	,OBJECT_NAME(objectid,st.dbid)
ORDER BY MAX(cp.usecounts) DESC

These execution counts are an aggregate from the last time SQL Server has been restarted.

The object name ‘x’ contains more than the maximum number of prefixes. The maximum is 2.

This error usually comes up when you are trying to perform a SELECT..INTO across a linked server. The problem is that a 4 part name cannot be used as the “INTO” parameter. This will fail:

SELECT *
INTO [remoteserver].RemoteDB.dbo.sysdtspackages
FROM MSDB.dbo.sysdtspackages

The trick is to create the table on the remote server, then perform an INSERT INTO.

INSERT INTO [remoteserver].RemoteDB.dbo.sysdtspackages
SELECT *
FROM MSDB.dbo.sysdtspackages

Note, I never recommend doing an insert using SELECT *, however since you just created the table, and it’s a one time shot. This works fine.

SQL Server Join Algorithms

If you read execution plans enough, you’ve probably realized that when SQL Server joins tables together, it uses different internal algorithms. The three algorithms are:

  • Loop Join
  • Merge Join
  • Hash Join

These alogorithms that are used are based upon factors of the underlying data.

Merge Join


For the most part, this is the most efficient method of joining tables. As the name implies, both tables are essentially merged together, much like a zipper being zipped. It typically occurs when both tables that are being joined, are joined on keys that are presorted and contain all the same keys in both tables (for example joining a primary key with a foreign key). When one table contains keys that the other table does not have, the chance of the merge join being used is less likely.

The physical profile of a merge join is very little CPU usage, and very little reads compared to other types of joins.

Loop Join

The loop join is more CPU intensive than a merge join. This join typically occurs when one worktable is quite a bit smaller than the other. As the word loop implies, the smaller table being joined is looped until it finds the matching key in the outer (larger) table. This join is most efficient when the resulting output is smaller than 5000 rows. When larger than that, the CPU and reads make the join less efficient.

Hash Join


A hash join is the least efficient of all joins, however that does not mean it is not necessary. When large tables are being joined and not all the common keys being joined exist in both sides and/or they are not sorted, this join type may be chosen. As the word Hash implies, a hash is created against all rows for the common key in each table, essentially breaking the table into approximately 7 buckets. After the table has been broken into these separate buckets, the buckets themselves are joined together resulting in a smaller comparison of a result set.

Within the Hash Join, there are 3 separate types of hash joins (In memory, Grace, and recursive). These hash joins are segragated according to how much memory is required to create them. Essentially the hash joins follow the same methodology as lock escalation. When a hash join is taking too much memory, then the next highest type of hash join is escalated to and pushes the buckets to disk.

Using Query hints for Join types

Increasingly, I do not recommend using query hints for SQL Server. With SQL Server 2000 I was more apt to use query hints. However, as the optimizer gets smarter, and the more you upgrade, you never know what the query optimizer is going to choose for the query or why it is choosing it. That being said, yes you can specify query hints for join types. To do so, see the following queries:

SELECT *
FROM Sales.SalesOrderDetail sod
INNER MERGE JOIN Sales.SalesOrderHeader soh
ON sod.SalesOrderID = soh.SalesOrderID
SELECT *
FROM Sales.SalesOrderDetail sod
INNER LOOP JOIN Sales.SalesOrderHeader soh
ON sod.SalesOrderID = soh.SalesOrderID
SELECT *
FROM Sales.SalesOrderDetail sod
INNER HASH JOIN Sales.SalesOrderHeader soh
ON sod.SalesOrderID = soh.SalesOrderID

SQL Server Denali – New Features

8/31/2011 – An update of new features can now be found here.

I’m hoping this is a collaborative post, because I do not know a lot about the capabilities of Denali, and the documentation looks pretty sparse.

What I do know however is that everything I have seen looks promising.  The SQL Server development team is top notch in my opinion.  The direction they are travelling is very promising for us as SQL Server professionals.

Onto the post.  So I downloaded the CTP version.  Which can be downloaded here

Installation

One note on the install.  You cannot install this on Windows XP or on Server 2003.  This is a little surprising because I can only imagine how many clients will not be able to upgrade.  The other surprising thing is that because many companies have not adopted Windows 7 yet on their desktops (and will likely never move to Vista), the developers will not be able to install SSMS (SQL Server Management Studio).  Because of this limitation, many people may find themselves calling this version SQL Server 2013.  As for the actual installation, it’s getting more cumbersome, and complicated, however the new features and added security are worth it.

SQL Server Management Studio

The first cool part is SSMS.  SSMS now boasts a “Powered by Visual Studio” text on the splash screen.  Digging into it deeper, you can see the changes.  The borders of the interface are a different color and the tabs that appear when you create a new query look different.  The color coding is nicer looking (table names are not black anymore), and the intellisense is quicker and responds without having to type in a schema name.

Code Snippets

Within SSMS is a new feature to add code snippets.  Tools -> Code Snippets Manager

code snippet manager

Within this dialog, you can add a new folder that contains code snippets.  The good news is that it appears folder does not have to be on your local computer, it can be on a shared drive so you can share code with your team.

Task List

This may be a small enhancement, however there is now a task list within SSMS.  Click on View -> Task List and you will find a place where you can add tasks for you to do and assign priorities to them.

Zoom (or Magnify)

Much like internet explorer (or Word) you can change the size of your display by altering the change the zoom percentage in the lower left corner.

TSQL

Two of the TSQL enhancements I have covered are the ability to Paginate result sets using ORDER BY.  And the ability to transform column names and datatypes returned from a stored procedure using the WITH RESULT SETS clause.

HADR (High Availability and Disaster Recovery)

This new model of HADR replaces the database mirroring setups in the past by using Windows Clustering technology.  Groups can be setup now that contain multiple databases.  These multiple databases (or group) can be failed over at once.

SQL Server Denali – Pagination using ORDER BY

SQL Server Denali (2011) now has additional features built into the ORDER BY Clause. This features makes pagination simpler and more efficient. Here is an example:

	SELECT *
	FROM HumanResources.Employee E
	ORDER BY E.EmployeeID ASC
		OFFSET 55 ROWS
		FETCH NEXT 30 ROWS ONLY;

Offset determines the start row to begin on, while the fetch next defines how many rows should be returned. Both parameters above can be variable driven. An example of this would be the following:

CREATE PROCEDURE dbo.spGetEmployees
(
	@StartRow int = 1,
	@RowCount int = 10
)
AS
BEGIN

	SELECT *
	FROM HumanResources.Employee E
	ORDER BY E.EmployeeID ASC
		OFFSET @StartRow ROWS
		FETCH NEXT @RowCount ROWS ONLY;

END

As you can see, this new syntax makes pagination much simpler than having to roll your own in previous versions.

SQL Server Denali – EXECUTE Statement WITH RESULT SETS

When I had originally written this article, I was very excited that this would be an answer to a long-standing wish that I have hope SQL Server would support. I wished I could call a stored procedure without knowing what was in the result set and dump it to a table (or temp table). Currently in order to trap the result set from executing a procedure we need to perform work arounds – all because we do not dynamically know the columns and data types the procedure is returning. If there was a way SQL could facilitate this without a work-around, it would take
SQL to a new level of abstraction and modularity.

The WITH RESULT SETS feature does allow us to change the column names and datatypes of the result set a stored procedure returns, however it does not allow us to define a subset of columns to return. Therefore, if a new column is added to the result set of a procedure that is called using WITH RESULT SETS, an error will occur.

The SQL Server Denali CTP can be downloaded here.

Let’s take a look at this new feature, first we’ll create a sample stored procedure:

CREATE PROCEDURE spSalesOverAmt
(
	@Amount money
)
AS
BEGIN

	SELECT
		SalesOrderID,
		OrderQty,
		UnitPrice
	FROM Sales.SalesOrderDetail
	WHERE UnitPrice > @Amount

END

Now let’s change the column names and the data types:

-- Execute the procedure with a parameter
EXEC spSalesOverAmt @Amount = 1200
WITH RESULT SETS
(
	(
		-- Specify the columns we want and
		-- the data types we want them to be
		SalesOrderID smallint,
		Units smallint,
		UnitAmount float
	)
);

How to Export a Table

Often times you have a need to export a table from SQL Server into another format.  The most common of these is Excel, CSV (Comma Separated Value), Access, or even to another SQL Server.  Luckily, there is one tool native to SQL Server that can export to all these formats, the SQL Server Import and Export wizard.

To start this wizard, simply right-click on the database that contains the table you want to export within Management Studio, then select Tasks -> Export Data.

Export Table SQL Server

 

What pops up next is the SQL Server Import/Export Wizard.  Select next to get past the splash screen then choose the data source.  This is going to be the same data source as the database you selected to export the table from.  Once you select the ServerName and Database name, select next.

Next you want to choose the destination.  This is where multiple export options come in.  You can choose any of the options that appear in the destination drop down box.  Once you select the export file type and file location, you will then be prompted to select the tables to output, along with additional options.  This will assist you in outputting many different export formats for your table.

How to Create a View

In order to create a view, use the “Create View” statement using the following convention:

CREATE VIEW dbo.vStoreAccountNum
AS
SELECT AccountNumber
FROM Sales.Customer
WHERE CustomerType = 'S'

Views are essentially stored SELECT Statements. They do not offer any speed advantage over regular SELECT statements. And they are not faster than a SELECT statement executed from within a stored procedure. They are mostly good for hiding complicated logic, or for providing an additional layer of abstraction from your source tables as a security measure. Along with a good indexing strategy, views can be used to simplify highly normalized tables.

The more joins within a view, and the more columns it returns back, the slower it is going to be. If you use views, try to make them only perform one function and only return back a finite number of columns. You don’t want views that return back a lot of columns when the majority of your queries only use one or two columns.

Here are some Do’s and Don’ts:

  • Do use views if you want Data Analysts to access common queries.
  • Do make sure all the join columns in a view contain indexes.
  • Do Not call views from within views.
  • Try not to return a lot of columns within a view if the view is referenced a lot.
  • Do Not use SELECT * within a view.  Specify the columns names explicitly.


Indexed (or Materialized Views)

A materialized view is a view that has been indexed and has essentially been transformed into a dynamically updated table.  This kind of view requires a special clause called SchemaBinding which disallows changes to occur on the base tables it references.  An indexed view requires a column that can be used as a unique index, which in turn acts as the clustered index for which all the data is aligned.

I personally try not to create indexed views because of the potential for maintenance nightmares.  When schemabinding is placed on a view, none of the tables it references can be changed unless the view is dropped.  Once the base tables are modified, the view needs to be recreated.  The other issue I have found with indexed views, is the optimizers unwillingness to utilize them.  Instead, most of the time in order to get the optimizer to use the indexed view, I would have to specify the WITH NOEXPAND hint.  If I did not, then the optimizer would use the base tables instead.  This is cumbersome because in the cases I’ve needed to use an indexed view, my goal was to rename the original table referenced by many procedures, and create an indexed view in it’s stead, only to find I would have to modify the procedures also to use the NOEXPAND hint.

Nevertheless, they do have uses, for the most part however, I prefer using staging tables that I can maintain on my own, and that get updated on set schedules.

If you are going to use a schemabound view, specify the following:

CREATE VIEW dbo.vStoreAccountNum
WITH SCHEMABINDING
AS
SELECT AccountNumber
FROM Sales.Customer
WHERE CustomerType = 'S'

Now that the view is schemabound, you need to create a unique clustered index on it.

CREATE UNIQUE CLUSTERED INDEX IDX_vStoreAccountNum ON dbo.vStoreAccountNum(AccountNumber)

Now the view is schema bound. Some notes on SchemaBound views.

  1. They cannot contain non deterministic functions EX: GETDATE(), DateDiff()
  2. You cannot use Count() instead use COUNT_BIG()
  3. You cannot use UNION
  4. You cannot use DISTINCT
  5. You cannot use SubQueries
  6. You cannot use Self-Joins
  7. You cannot use Complex Aggregate function

As you can see, make sure not to code yourself into a corner using Indexed Views.

SQL Replace

The SQL Replace function replaces a value within a string with another value. The final output is the original string with the replaced value. There are three parameters to the Replace function. The first is the original string. The next parameter is the string to replace. And the last parameter is the value to replace the string with.

Here is an example:

SELECT
	REPLACE
	(
		'original string',	-- Original String
		'string',			-- String to replace
		'replaced'			-- What to replace with
	)

The output from this is: “original replaced”.

One thing you need to watch out for is that if any of the parameters in the replace statement are NULL, then the string that is returned will also be NULL. So if passing parameters dynamically and there is a chance of NULL, make sure to use the ISNULL function and turn it into an empty string.

You can also sometimes get away with using REPLACE rather than concatenating strings. I’ve done this before when you have a templated piece of text that needs values injected into it.

DECLARE @ResponseMessage varchar(max);

SET @ResponseMessage = 'Dear [firstname], thank you for signing up for [subscription].';

SELECT @ResponseMessage =
	REPLACE
	(
		@ResponseMessage,
		'[firstname]',
		pc.FirstName
	)
FROM Person.Contact pc
WHERE ContactID = 1;

SELECT @ResponseMessage;

Though it may not seem like much of a big deal, when dealing with many lines of concatenated text, it comes out much cleaner.

How to Call a Stored Procedure

Although there are a few ways to call a stored procedure, only one way is actually recommended. The best way is to explicitly specify the stored procedure parameters in the Execute statement. You do this because if the order of the parameters were to change, then the procedure would error out (or even worse pass the incorrect values to the parameters).

Let’s create a stored procedure as an example:

CREATE PROCEDURE spReturnFullName
(
	@FirstName varchar(50),
	@LastName varchar(50),
	@Age int,
	@ClientName varchar(100) OUTPUT
)
AS
SET NOCOUNT ON
BEGIN

	SELECT
		@ClientName = @FirstName + @LastName

END

Now that we have our procedure, let’s call the procedure using EXECUTE. We can also use the EXEC statement as a replacement for Execute, it is merely a simplified way to call the same statment. So below we are specifically specifying the parameters of the procedure. Unfortunately, SQL Server allows procedures to be executed without specifying the parameter name. The way we are doing it below is not only self-documenting, but as mentioned, it will not allow the wrong values to be passed to a parameter.

DECLARE @ClientNameReturn varchar(100)

EXEC spReturnFullName
	@FirstName = 'Vincent',
	@LastName = 'Vega',
	@Age = 40,
	@ClientName = @ClientNameReturn OUTPUT

SELECT @ClientNameReturn

This procedure also has an output parameter, which is denoted by the OUTPUT reserved word.