How to assign a static port to a SQL Server named instance – and avoid a common pitfall

While Books Online clearly mentions the steps to Configure a Server to Listen on a Specific TCP Port we still see people missing out on one small but important detail in these steps: they forget to delete the entry (0 or some random port) for dynamic port. This firstly results in confusion and occasionally can result in connectivity problems as well. Let me explain how using an example from one of our lab setups.

As a first step, let’s see what the ‘administrator’ (in this case, yours truly Smile) had done:

image

As you can see, they have added the static port for ‘IPAll’ with a value of 1450. That part is fine. The problem though is they forgot to remove the entries for the dynamic ports (0 or some random port). That means that when they restarted SQL, the dynamic port setting is still valid. In fact if we query sys.tcp_endpoints, you will still see the engine thinks it is listening on dynamic port:

SELECT        name, protocol_desc, type_desc, state_desc, is_admin_endpoint, port, is_dynamic_port, ip_address 
FROM            sys.tcp_endpoints

gives us:

name protocol_desc type_desc state_desc is_admin_endpoint port is_dynamic_port
Dedicated Admin Connection TCP TSQL STARTED 1 0 1
TSQL Default TCP TCP TSQL STARTED 0 0 1

The important observation is that the engine reports that it is still using a dynamic port. It does not report the static port number 1450 which we selected in Configuration Manager. Let’s double-check in the errorlog to see if indeed the static port is being picked up at all. And lo and behold:

Server is listening on [ 'any' <ipv6> 1450]. 
Server is listening on [ 'any' <ipv4> 1450]. 
Server is listening on [ 'any' <ipv6> 49626]. 
Server is listening on [ 'any' <ipv4> 49626].

In our case, sqlservr.exe has a PID of 1240. Using the command netstat –ano, we can see what it is listening on.

 Proto  Local Address          Foreign Address        State           PID 
  TCP    0.0.0.0:1450           0.0.0.0:0              LISTENING       1240 
  TCP    0.0.0.0:49626          0.0.0.0:0              LISTENING       1240 
  TCP    127.0.0.1:49627        0.0.0.0:0              LISTENING       1240 
  TCP    192.168.1.101:1450     192.168.1.200:49386    ESTABLISHED     1240 
  TCP    192.168.1.101:1450     192.168.1.200:49396    ESTABLISHED     1240 
  TCP    [::]:1450              [::]:0                 LISTENING       1240 
  TCP    [::]:49626             [::]:0                 LISTENING       1240 
  TCP    [::1]:49627            [::]:0                 LISTENING       1240

So it is not only listening on the static port, but also on the dynamic port 49626. The DAC is listening on TCP port 49627. The values with a local address of [::] are the IPv6 ‘All’ address.

So depending on what got cached earlier in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSSQLServer\Client\SNI11.0\LastConnect on the client (you can find some detail about the LastConnect registry key and the caching mechanism in this KB article), the client might attempt to connect to the previous dynamic port (which is still valid based on our observation above.)

FYI, if we run NetMon, we can see that the SSRP response from the SQL Browser correctly gives back 1450 as the port for this named instance:

image

For clarity I’ve reproduced the response from SQL Browser (using the SSRP protocol) back to my client (SQLCMD.exe):

.^.ServerName;SOMESERVER;InstanceName;SOMEINSTANCE;IsClustered;No;Version;11.0.2100.60;tcp;1450;;

From the above it is clear that SQL Browser is correctly sending the static port assignment. But if you are like me, I feel uneasy till I fix the root cause, which is to delete the dynamic port assignment!

To summarize here is what we saw in this walkthrough:

  1. The official steps (captured in Books Online) to assign a static port for a named instance involve also deleting the value (0 or some random port) for the dynamic port.
  2. Failure to delete the dynamic port value in SQL Configuration Manager will cause SQL to listen on both the static as well as the dynamic ports.
  3. This means that clients will succeed to connect to the erstwhile dynamic port if they had that cached in the LastConnect client side registry key.
  4. For clients which do not have cached connection details, SQL Browser seems to pickup the static port and sends that back to the client.
  5. So follow the steps in the BOL article to the T and delete the dynamic port value right after you type in the static port value, and in any case before you restart the instance.

FYI the steps to fix a static port for the Dedicated Admin Connection (DAC) are in the KB article How to configure SQL Server to listen on a specific port under the section ‘Configuring an instance of SQL Server to use a static port’.

I hope you enjoyed reading this post as much as I enjoyed bringing it to you! Please take a minute to rate this post and leave some comments!

“Operating system error 1398” while backing up database to network share

I was facing this on a VM setup, where the service account of my SQL Server service is a domain account. The DC is also a VM. Here were the symptoms:

While backing up the database to the UNC share, I was getting an error:

"Operating system error 1398" (There is a time and/or date difference between the client and server)

First things first, I did not have any timesync issue. That would have affected Kerberos and everything else around it.

Secondly, doing a xp_cmdshell ‘dir \uncserverpath’ would result in the same error, so this was not unique to the backup code (obvious, but better to check.)

Interestingly, if I opened up a new CMD prompt ‘running as’ the SQL Service account, and execute the ‘dir’ command, then the network share contents were listed (i.e. no error message.)

So then it dawned to me that at the time the SQL Service was starting up, there was a race condition wherein the DC was not yet ‘fully’ started. This caused the usage of stale Kerberos tickets and in turn caused the above error.

Only way to fix this was to restart my SQL instance, because then the expired Kerberos ticket would be replaced with a freshly authenticated and timestamped one. Sad, but true.

Hope this helps someone out there!

Replication and Linked Servers

Problem

I recently hit upon an issue with trying to setup a linked server to an instance which was already a subscriber to a publication. When replication is setup, it actually creates a remote server for the subscriber. However that ‘remote server’ is not configured for data access. So if you try to use that server, you would end up with:

Msg 7411, Level 16, State 1, Line 1
Server ‘foosub’ is not configured for DATA ACCESS.

Also, any attempt to add a similarly named linked server would fail with the error message below:

EXEC master.dbo.sp_addlinkedserver @server = N’foosub’, @srvproduct=N’SQL Server’

Msg 15028, Level 16, State 1, Procedure sp_addlinkedserver, Line 82
The server ‘foosub’ already exists.

Troubleshooting

Next, I tried to add a linked server (using the SQL Native Client) but with a different name (MYSRV) but pointing to the right server (foosub). My initial attempt yielded the following error:

The linked server has been created but failed a connection test. Do you want to keep the linked server?

——————————
ADDITIONAL INFORMATION:

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

——————————

Named Pipes Provider: Could not open a connection to SQL Server [53].
OLE DB provider "SQLNCLI10" for linked server "MYSVR" returned message "Login timeout expired".
OLE DB provider "SQLNCLI10" for linked server "MYSVR" returned message "A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.". (Microsoft SQL Server, Error: 53)

Solution

The problem turned out that I had not used the right ‘provider’ string. Here is the script which finally worked for me:

EXEC master.dbo.sp_addlinkedserver @server = N’MYSVR’, @srvproduct=N’foosub’, @provider=N’SQLNCLI10′, @provstr=N’Server=foosub;Database=master;Trusted_Connection=yes’

EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname = N’MYSVR’, @locallogin = NULL , @useself = N’True’
GO

Or if you are more comfortable using the SQLOLEDB provider, here’s a sample:

EXEC master.dbo.sp_addlinkedserver @server = N’MYSRV’, @srvproduct=N’SQLOLEDB’, @provider=N’SQLOLEDB’, @datasrc=N’foosub’, @provstr=N’Data Source=foosub;Initial Catalog=master’, @catalog=N’master’

EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname = N’MYSRV’, @locallogin = NULL , @useself = N’True’
GO

You can also do this from the SSMS GUI by using ‘‘SQLNCI10’ or ‘SQLOLEDB’ as the Provider.

With this, I can subsequently access remote tables as such:

SELECT * FROM MYSVR.master.sys.tables

Hope this is useful! Please leave a comment if you find it useful.

My Favorite SQL Server Blogs

At our workshops and during other customer interactions, we are usually asked for links to good blogs and reading materials. Here is a list of my favorite SQL-related blogs, arranged in no specific order.

http://blogs.msdn.com/craigfr: Craig Freedman on Query Processing (QP)

http://blogs.msdn.com/sqlprogrammability: Plan cache, parameterization etc.

http://blogs.msdn.com/sqlserverstorageengine: Storage Engine

http://blogs.msdn.com/repltalk: Replication support team

http://blogs.msdn.com/psssql: CSS SQL Support team

http://www.sqlskills.com/blogs/PAUL: Paul Randal with lots of tips

http://blogs.msdn.com/sqlqueryprocessing/default.aspx: Query Processing again

http://blogs.msdn.com/sqlserverfaq: Other CSS SQL Support team members

http://blogs.msdn.com/davidlean: Good series of posts on SQL Spatial features

http://blogs.msdn.com/sql_pfe_blog: Our global SQL PFE team blog

http://blogs.msdn.com/robertbruckner: Good information on SSRS

http://www.sqlcat.com: The SQL Server Customer Advisory Team (SQLCAT)

http://blogs.msdn.com/sqlreleaseservices: Learn about the latest releases (Service Pack / Cumulative Update) from SQL product team

I will be updating this list as and when I remember / discover other useful blogs. I hope this is useful for the community!

Hack of the day: Shrink all log files in the instance

Before I proceed, I will add two disclaimers:

  1. This script is provided as-is for knowledge purposes. It is not a recommendation, or in any way intended for production usage. No warranty or guarantee is made about the correctness of the script. Use it at your own risk.
  2. We do not recommend using SHRINK operations on any production databases. This script is provided for demonstration purposes only and that too for test or development servers. For more details, please refer to the series of posts by Paul Randal at http://www.sqlskills.com/BLOGS/PAUL/category/Shrink.aspx

With that behind us, here’s what I would like to share. On my laptop I keep running large queries and many times those will grow the log file. Though my database is in simple recovery mode, a single long running transaction can cause this kind of growth.

So I would sometimes like to reclaim disk space (oh so precious on a laptop!) periodically. I would however not like to shrink the data file, just the log file. (Shrinking data files can cause fragmentation issues to occur.)

The problem is I normally operate on like 5-6 databases on my laptop. I would not want to use the GUI nor script this manually each time. So here is a simple script-generator, which will generate the necessary DBCC SHRINKFILE commands, which you can then execute as per your requirement.

drop table #f
go
create table #f (name sysname, fileid int, filename sysname, filegroup sysname null,
size sysname, maxsize sysname, growth sysname, usage sysname)
go
exec sp_MSforeachdb ‘declare @s varchar(8000); use ?;
truncate table #f; INSERT #f (name, fileid, filename, filegroup, size, maxsize, growth, usage )
exec sp_helpfile
select @s = ”use ?; DBCC SHRINKFILE (” + name + ” )” from #f where usage = ”log only”
print @s ‘

Really crude script, but it does save me some time once in a while.

SQL Server setup log locations and more

In order to efficiently troubleshoot problems involving failed setup, we need to use the setup logs. Once you know where the log files are located, you can start troubleshooting. What to look for in the log files is a separate topic in itself, but at least if you know where the log files are, then you can attempt to look for logical clues in the files. So, where are these log files located? Here’s a summary, listed by product and version. I took some efforts to compile the list of available rollup hotfixes and their relevant KB articles, hopefully that in itself is a very useful thing.

So, I hope the list and the notes below are very useful for you; please feel free to rate the article and provide comments as well!

Product Major Version 1

Product Level 2

Version post setup 3

Bulletins / KB Articles

Location of setup logs 4

SQL Server 2000 (8.0)

RTM

8.00.194

N/A

%WINDIR%sqlstp.log

SP4

8.00.2039

888799

%WINDIR%sqlsp.log

Post-SP4 GDR security rollup 7

8.00.2050

MS08-040; 948110

%WINDIR%SQL2000-KB948110-v8.00.2050-x86-ENU.log; SQLServerClient_Hotfix_KB948110_*.log

[Recommended] Post-SP4 QFE security rollup 8.a

8.00.2273

MS08-040;

948111

%WINDIR%SQL2000-KB948111-v8.00.2273-x86-ENU.log; SQLServerClient_Hotfix_KB948111_*.log 11

IA64 versions of the security rollups 13

Per respective packages mentioned above

%WINDIR%HotfixHotfix.log

%WINDIR%HotfixSQL64_Hotfix_KBnumber_.log

Clustered instance 14

N/A

RTM:

%WINDIR%Sqlclstr.log 5;

%WINDIR%sqlstpN.log from both nodes

 

Service Pack only:

%WINDIR%sqlspN.log from both nodes;

%WINDIR%remsetup.log from active node

 

Hotfix only:

Relevant hotfix files as described above from both nodes

SQL Server 2005 (9.0)

RTM

9.00.1399

N/A

%ProgramFiles%Microsoft SQL Server90SetupBootstrapLOGFiles

SP1

9.00.2047

913090

%WINDIR%Hotfix 10

SP2

9.00.3042

921896

%ProgramFiles%Microsoft SQL Server90Setup BootstrapLOGHotfix; files will have KB921896 as part of their name.

Post-SP2 GDR security rollup (engine only)

9.00.3068

MS08-040;

948109

Same as above; files will have KB948109 as part of their name.

Post-SP2 GDR security rollup (engine + Reporting Services)

9.00.3073

MS08-052; 954606

 

Same as above; files will have KB954606 as part of their name.

Post-SP2 QFE security rollup (engine only)

9.00.3233

MS08-040;

948108

Same as above; files will have KB948108 as part of their name.

Post-SP2 QFE security rollup (engine + Reporting Services)

9.00.3282

MS08-052; 954607

Same as above; files will have KB954607 as part of their name.

CU #11 for SP2 8

9.00.3301

958735

Same as above; files will have KB958735 as part of their name.

[Recommended] SP3

9.00.4035

955706

Same as above; files will have KB955706 as part of their name.

CU #1 for SP3 8

9.00.4207

959195

Same as above; files will have KB959195 as part of their name.

Clustered instance

N/A

%WINDIR%tasksSchedLgU.Txt 6; Setup bootstrap will copy all the files from remote node to active node.

SQL Server 2008

RTM

10.0.1600.22

N/A

%ProgramFiles%Microsoft SQL Server100Setup BootstrapLog 9

CU #2

10.00.1779

958186

 

Clustered instance

N/A

At the time of writing no special considerations are known.

 

Notes:

  1. This is just the common name which we all tend to use, the value in brackets is the major version number used by the product itself.
  2. To determine the product level use SELECT SERVERPROPERTY(‘ProductLevel’). However this will not reflect the GDR / QFE hotfix status. It just shows the latest service pack.
  3. To determine the exact SQL Server build number / version, use SELECT SERVERPROPERTY(‘ProductVersion’). (More information about other product levels and version numbers is summarized in this KB article.)
  4. Here %WINDIR% refers to the location where Windows is installed (typically C:WINDOWS) and %PROGRAMFILES% denotes location of the Program Files folder (typically C:PROGRAM FILES).
  5. This file is only generated for SQL 2000 clustered instance installation. Service Pack installation will update this file, but subsequent hotfix installers will not update this file.
  6. The SQL Server 2005 RTM and service pack installers will launch the remote node setup program using Task Scheduler service. This log file is used to check if the remote task was actually started or not. Check this file (in addition to the regular setup files) on each node of your SQL failover cluster.
  7. GDR vs. QFE releases can be a confusing topic. Luckily they are explained very well in the following blog links:
  8. In general, note that applying a Cumulative Update should be done only if you are facing specific issues fixed by that update; doing so will take you to the QFE tree and you should apply the appropriate QFE rollup (if and when available.) This note applies to all rollups and Cumulative Updates.
    • 8.a However for SQL 2000 SP4, we might recommend being on the QFE tree due to the fact that there will be no further service packs for that product.
  9. There are no separate folders for RTM and hotfix / Service Pack installers. Instead, look at the relevant sub-folder of this folder with the latest time stamp to get to the latest installation files. Historical logs will be maintained in the subfolder with the respective timestamps.
  10. SQL Server 2005 SP1 is unique in the SQL 2005 chain because it stores its logs under the WindowsHotfix folder. Under that there are subfolders for each component, such as SQL9, SQLTools9 etc.
  11. The exact filename is dependent on the name of the self-extracting package which was executed. The SQL 2000 QFE rollup generates many more log files than the GDR installer. This is expected because the GDR security rollup only contains security fixes. The QFE security rollup contains bug fixes and the requisite security fixes. See this KB article for details.
  12. For what to look for in the log files, consider this topic from SQL Books Online. An additional reference for troubleshooting Windows Installer logs is found here.
  13. The hotfix installer for SQL 2000 IA64 is described in this KB article.
  14. Cluster delta specific instructions only; regular log files still apply.

Windows, SQL Server and Multi-Core / Hyper-Threading

Updated 14 March 2019: the information in this blog post is NO LONGER RELEVANT. Official licensing information should be obtained through your regular Microsoft sales / licensing specialists.

Very often we get asked a question about ‘how many processors does my SQL Server 200x Edition really support’. This post hopes to summarize the considerations and actual limits.

SQL Licensing is per socket

First, let’s take a look at the Books Online topic ‘Maximum Number of Processors Supported by the Editions of SQL Server‘. It says: SQL Server is licensed per processor socket, and not per logical CPU basis.

So what does that really mean? Again from the above Books Online topic:

For example, the following is considered a single processor for purposes of this table:

  • A single-core, hyper-threaded processor with 2 logical CPUs per socket.
  • A dual-core processor with 2 logical CPUs.
  • A quad-core processor with 4 logical CPUs.

Windows Licensing is also per socket

Just like SQL Server, Windows Server licensing has been based around sockets. This document summarizes it nicely:

For Microsoft software with processor limits, each processor counts as a single processor regardless of the number of cores and/or threads that the processor contains. For example, Windows Server 2003 R2 Standard Edition can be used on a four-processor system, whether the processors in the system are single-core, hyperthreaded, or multicore.

So, what’s in my box?

A natural question for customers is, how to determine what is in my server hardware? Is it socket, or cores, or logical CPUs from HyperThreading?

Option 1: Use Intel’s utility

From our KB article on SQL and Hyperthreading, we recommend using Intel’s CPUCount utility to distinguish this for IA32 or Intel 64 class CPUs. The utility distinguishes between CPUs (sockets), cores, and logical CPUs (HyperThreading). The current article describing it is here and the sample code is here (both are from the Intel website.) If you build the samples using Visual Studio you can see the output below:

httpsmsdnshared.blob.core.windows.netmediaTNBlogsFSBlogFileStorageblogs_msdnarvindshWindowsLiveWriterWindowsSQLServerandMultiCoreHyperThreadi_A41Bimage_thumb_1.png

Option 2: Use SysInternals CoreInfo

Mark Russinovich has released CoreInfo which can also dump similar information. The only concern I have is that it uses the term ‘physical processor’ a little too freely, which can confuse someone. Here is sample output for a 1-socket, 2-core, no HT CPU:

httpsmsdnshared.blob.core.windows.netmediaTNBlogsFSBlogFileStorageblogs_msdnarvindshWindowsLiveWriterWindowsSQLServerandMultiCoreHyperThreadi_A41Bimage_thumb_2.png

Option 3: Windows Vista / Server 2008

MSInfo32 (a.k.a. System Information program) can be used to distinguish between sockets, cores and logical processors.

httpsmsdnshared.blob.core.windows.netmediaTNBlogsFSBlogFileStorageblogs_msdnarvindshWindowsLiveWriterWindowsSQLServerandMultiCoreHyperThreadi_A41Bimage_thumb_3.png

Option 4: Third Party Tools

A variety of 3rd party tools can report detailed CPU information, such as number of cores, logical processors and more.

Product Limitations

The Windows 2003 CPU (physical socket) limitations can be drawn from this page and an updated one for Windows 2008 at this page:

Windows Edition Number of physical sockets supported
Windows Server 2003 / 2008, Standard Edition 4
Windows Server 2003 / 2008, Enterprise Edition 8
Windows Server 2003 / 2008, Datacenter Edition 32 (32-bit) or 64 (64-bit)

SQL Server limitations from Books Online:

SQL Edition Number of physical sockets supported
SQL Server 2005 / 2008, Express Edition 1
SQL Server 2005 / 2008, Standard Edition 4
SQL Server 2005 / 2008, Enterprise Edition OS Maximum (as per the table above)
SQL Server 2000, Desktop Engine (MSDE) 2 on currently supported OS editions *
SQL Server 2000, Standard Edition 4 on currently supported OS
SQL Server 2000, Enterprise Edition Check link below *

* There are specific notes for older unsupported OS platforms such as Windows 98 and NT 4.0 which are described in this SQL 2000 Books Online entry.

Also note that SQL 2000 only introduced HyperThreading awareness in SP3 so that the number of logical processors are handled correctly when it comes to enforcing the licensing limits.

There is a related blog post from our CSS SQL brethren which I would recommend all to read.

Resources from TechEd sessions

Several attendees have asked for the resources section from our presentations at TechEd India. Enjoy!


Blogs
SQLCAT:
http://blogs.msdn.com/sqlcat
WINCAT:
http://blogs.msdn.com/wincat
Slava Oks’ blog: http://blogs.msdn.com/slavao


Product Feedback
http://lab.msdn.microsoft.com/productfeedback/


External Links:
NUMA FAQ:
http://lse.sourceforge.net/numa/faq


Books:
Inside Microsoft SQL Server 2005: Query Processing and Optimization (Volume 3)
The Guru’s Guide to SQL Server Architecture and Internals (SQL Server 2000)


SQL Server 2005 – Operations 
http://www.microsoft.com/technet/prodtechnol/sql/2005/library/operations.mspx


Whitepapers
Troubleshooting Performance Problems in SQL Server 2005
(http://www.microsoft.com/technet/prodtechnol/sql/2005/tsprfprb.mspx)
Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005(http://www.microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx)
Statistics Used by the Query Optimizer in SQL Server 2005 (http://www.microsoft.com/technet/prodtechnol/sql/2005/qrystats.mspx)
SQL Server 2000 I/O Basics (
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/sqlIObasics.mspx)


Webcasts
Troubleshooting Performance Problems In SQL Server 2005 (http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?EventID=1032275646&EventCategory=5&culture=en-US&CountryCode=US)
Performance Diagnosis in SQL Server 2005 (
http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?EventID=1032275586&EventCategory=5&culture=en-US&CountryCode=US)
Supportability features for SQL Server 2005
(http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?EventID=1032275593&EventCategory=5&culture=en-US&CountryCode=US)


PASS 2005 PSS Service Center Labs
http://www.microsoft.com/downloads/details.aspx?FamilyID=9b8098a7-e75f-462a-b296-e80199c9f323&displaylang=en

Database Snapshots

DB Snapshots are a very useful new feature in SQL Server 2005 Enterprise Edition. They provide us the ability to preserve a consistent picture of the database as it was at a point in time. Both metadata and data are captured in the snapshot.


 


The main uses of snapshots are for preserving views of the database for reporting purposes, and also to easily recover data affected by user error. Snapshots also provide a means to view the data in the mirror database.


 


DB Snapshots use “sparse files”, which means that the actual snapshot size on disk will be very small initially and would only grow by recording the the changes to the database using a copy-on-write scheme.


 


Another cool feature of snapshots for recovering from administrative or user error is the ability to revert a database back to a snapshot. This is potentially much quicker than restoring a full backup, as we only copy the changed pages from the snapshot back into the data files. But there are other implications of reverting to a snapshot – most important being the fact that we rebuild the log in such cases. To ensure recoverability of all transactions committed after the snapshot was taken, be sure to backup the log prior to using this feature.