• SQL to SQL: A Practical Guide from a DBA

    SQL to SQL: A Practical Guide from a DBA

    If you’re looking to connect, migrate, or convert between SQL databases, this post is aimed at you.

    Whether you’re working with SQL Server, MySQL, PostgreSQL, or others, I’m hoping to help explain some common tasks you might need to perform.

    Contents:
    > Connecting SQL Databases
    > Linked Servers: SQL Server to Another SQL Server
    > Copying Data Between SQL Systems
    > Converting SQL Queries Between Systems
    > Best Practices for SQL Operations

    Connecting SQL Databases

    Connecting two SQL systems can be done in several ways depending on your environment. Here are some of the most common options:

    SQL Server Management Studio (SSMS)
    SSMS, the go-to tool for connecting to and managing Microsoft SQL Server databases.

    ODBC (Open Database Connectivity)
    ODBC (Open Database Connectivity) allows you to connect SQL databases across platforms like MySQL and PostgreSQL. Tools like DBeaver or DataGrip can help manage these connections.

    Linked Servers: SQL Server to Another SQL Server

    If you’re working with SQL Server and need to connect to another SQL Server instance or even a different database, Linked Servers is a feature to consider. This allows you to query data across servers as if it were on the same server. Check out my other blog post on how to configure Linked Servers.

    Once set up, you can query the remote server like this:

    -- query sql linked server 
    SELECT *  
    FROM [LinkedServerName].SourceDB.dbo.SourceTable;
    
    
    
    
    
    Query Linked Server

    This method simplifies querying data from one SQL Server to another, even if they are on separate servers. It is not the preferred method though*

    Copy Data Between SQL Systems

    Copying data from SQL to SQL systems involves a few key steps:

    1. Backup the Source Database:
    Always start by creating a backup to safeguard your data.

    2. Extract Schema and Data:
    Use tools like SSMS, pgAdmin, or MySQL Workbench to export the schema and data from the source database.

    3. Prepare the Target Database:
    Set up the necessary schema and ensure compatibility between data types (e.g., VARCHAR vs. TEXT).

    4. Transfer the Data:
    Use tools like the SQL Server Import and Export Wizard or third-party software like SQLines.

    5. Validate the Transfer:
    Test the data in the target system to ensure accuracy.

    Converting SQL Queries Between Systems

    When moving from one SQL system to another, there are often slight differences in syntax. Here are some common conversions:

    SQL Server to PostgreSQL
    – Replace GETDATE() with CURRENT_TIMESTAMP.
    – Change IDENTITY columns to SERIAL or GENERATED in PostgreSQL.

    MySQL to SQL Server
    Replace LIMIT with TOP or use OFFSET FETCH for pagination.

    Tools to Help with Query Conversion

    ETL Tools: For larger, more complex migrations, consider using ETL tools like Microsoft SSIS or Azure Data Factory.
    SQLines: A free online tool that can convert SQL queries between most SQL systems.
    DBConvert: A paid solution for more complex SQL conversions.

    Best Practices for SQL Operations

    To ensure a smooth SQL operations, keep these best practices in mind:

    1. Always Backup Your Data: This cannot be stressed enough. Always create a full backup before making any changes.

    2. Check Compatibility: Review the data types, syntax, and features in the source and target databases to ensure compatibility.

    3. Test with a Small Dataset: Before transferring large amounts of data, test the process with a small sample to ensure everything works as expected.

    4. Use Reliable Tools: Tools like Talend, SQLines, or SSIS are excellent for handling large migrations or complex workflows.

    Whether you’re setting up linked servers, migrating data, or converting SQL queries, these strategies should help you work effectively with SQL databases. Let me know in the comments if you have any specific questions or need help.

  • How to Check and Manage SQL Server Services

    How to Check and Manage SQL Server Services

    As a SQL Server DBA, it’s important that we have quick and efficient ways to check SQL Services. We need to ensure our SQL Services are always available, and we often need to restart or make changes to them in SQL Server Configuration Manager. This includes the SQL Server Engine, Agent, and any other service you are relying on.

    There are many ways to check running SQL Services, including via PowerShell, SQL Server Configuration Manager, or by querying SQL DMVs. In this blog post, I’m showing how to check SQL Services on Windows, which includes the following:
    1. SQL Query to Check Services Information
    2. Checking Services in SQL Server Configuration Manager
    3. Other Methods to Check SQL Server Services

    1. SQL Query to Check Services Information

    Using a SQL query to check services information is the go to choice for a Database Admin. We need to have good eyes on this information, and if any of my critical SQL Servers were to go down, we need an alert.

    Run the following query to show a snapshot of SQL Server services, including the SQL Agent and key information:

    -- SQL Server Services Info
    SELECT servicename, process_id, startup_type_desc, status_desc, 
    last_startup_time, service_account, is_clustered, cluster_nodename, [filename], 
    instant_file_initialization_enabled -- New in SQL Server 2016 SP1
    FROM sys.dm_server_services WITH (NOLOCK) OPTION (RECOMPILE);
    
    get sql services information with query

    We’re querying the sys.dm_server_services DMV, which returns the following (and more):
    > servicename : The name of the SQL Server service.
    > process_id : The process ID for the service.
    > startup_type_desc : The startup type (e.g., Automatic, Manual).
    > status_desc : Current service status (e.g., Running, Stopped).
    > last_startup_time : Timestamp of the last service startup.
    > service_account : The account the service is running under.
    > is_clustered : Whether the service is clustered.
    > cluster_nodename : Name of the cluster node (if clustered).
    > filename : Path of the service executable.
    > instant_file_initialization_enabled : Whether instant file initialization is enabled (available from SQL Server 2016 SP1).

    2. Checking Services in SQL Server Configuration Manager

    SQL Server Configuration Manager is the best place and practice for managing and checking SQL Server services. It is the tool we should use for starting and stopping (and configuring) services, as apposed to doing via Services.msc.

    If you’re working in a high-availability environment with Always On Availability Groups (AGs), you’ll often be managing services through Failover Cluster Manager to ensure proper failover and service health.

    SQL Server Configuration Manager should be installed on your machine if it has SQL Server installed. If you can’t find it by searching for the app, it can be found in the following locations:

    SSMS 18
    C:\Program Files (x86)\Microsoft SQL Server Management Studio 18\Common7\IDE\Extensions\Application
    SSMS 19
    C:\Program Files (x86)\Microsoft SQL Server Management Studio 19\Common7\IDE\Extensions\Application
    SSMS 20
    C:\Program Files (x86)\Microsoft SQL Server Management Studio 20\Common7\IDE\Extensions\Application

    sql server config manager services

    You can restart and manage your MSSQL services within here.

    3. Other Methods to Check SQL Server Services

    Aside from using the SQL query and SQL Server Configuration Manager, there are some other ways we can view and manage SQL Server services, including:

    PowerShell
    We can use Get-Service, Start-Service, Stop-Service and Restart-Service cmdlets.

    Services.msc
    As mentioned within this post we can view and restart SQL Server services within Windows Services, but it’s not the recommended way (it’ll be fine, don’t worry).

    Hope all this was useful and what you were looking for. Feel free to checkout other links around for more random tips from a SQL Server DBA!

  • Checking if Query Store is Enabled in SQL Server

    Checking if Query Store is Enabled in SQL Server

    This post shares the simple steps and SQL queries to check if Query Store is enabled in SQL Server, and view its options for a database.

    The Query Store feature was introduced in SQL Server 2016 (13.x) and enabled by default in SQL Server 2022 (16.x), if created on SQL2022. It helps DBAs and developers identify and resolve query performance issues, and its easy to setup.


    1. Check if Query Store is Enabled

    To check if Query Store is enabled for all databases, query the sys.databases system table and review the is_query_store_on column. This is a table I query very often as a SQL Server Database Administrator to check database properties.

    -- Check if query store is enabled for all dbs
    SELECT is_query_store_on, *
    FROM sys.databases;
    check if query store is enabled

    Alternatively, in SSMS, you can right-click the database, select Properties, and go to the Query Store tab:

    sql server query store properties

    2. Check Query Store Options

    To view the Query Store options for a database, we can query the Query Store Catalog Views:

    -- View Query Store Options for a database  
    SELECT *  
    FROM sys.database_query_store_options WITH (NOLOCK) OPTION (RECOMPILE);  
    
    sql server query store options

    Query Store State Descriptions:
    > OFF (0): Not enabled.
    > READ_ONLY (1): Query Store is in read-only mode (quota exceeded).
    > READ_WRITE (2): Query Store is capturing all queries.
    > ERROR (3): Query Store is in an error state. To recover, disable and re-enable Query Store:

    ALTER DATABASE [YourDatabase] SET QUERY_STORE = OFF;  
    ALTER DATABASE [YourDatabase] SET QUERY_STORE = ON;  
    

    I say recover from, as it’s easy to enable and is enabled by default as of SQL Server 2022, as mentioned at the top of this post.

    When using Query Store, review Microsoft’s Best Practices for Monitoring Workloads with Query Store to ensure optimal performance. If you found this useful, check out my other posts for more SQL Server tips!

  • SQL Server Performance Troubleshooting Scripts

    When your SQL Server starts to experience performance issues, it’s important to have tools at your disposal to quickly identify and address the root causes.

    One such tool is having a performance troubleshooting scripts that provides some insights into SQL Server’s current state; including worker threads, running queries, and various performance metrics.

    In this blog post, I’m sharing scripts that will help check in on your SQL Server performance and problem processes. Diagnosing performance issues in SQL Server is complex, and every issue is unique to your own environment. The tips I share here are based on my experience in the field troubleshooting such issues. It’s aimed to help you see the issue at-hand, get you out of the danger zone, and live another day so we can investigate the root cause while things are operational.

    The following is included in this article:
    1. Get SQL Worker Threads
    2. Show Currently Running SQL Queries
    3. Additional Notes & Follow-up Tasks

    MSSQL Performance Troubleshooting Scripts

    1. Get SQL Worker Threads

    This part retrieves the current count of SQL worker threads, providing visibility into the server’s overall workload and resource utilisation.

    -- Get SQL worker threads
    SELECT SUM(current_workers_count) as [Current Worker Threads] FROM sys.dm_os_schedulers

    In an unhealthy scenario, thread starvation may occur, leading to increased wait times and degraded performance.

    During my current role and experience, if threads are above 600/700 things are nearing danger and a MSSQL Team on call alert may have been triggered. It means we have to check in on the SQL Server activity, look at processes and figure out what’s causing SQL to be throttled. More explained on this in the next part.

    Understanding good and normal thread counts takes understanding over time. Have a look at this Microsoft article on the recommendations with SQL Server thread configurations. If you want to get to know your ideal number(s) take your time with it, monitor/check current worker thread counts, and read the MS docs.

    2. Show Running SQL Queries

    Next up, I’m sharing a SQL script displays information about the queries currently running on the SQL Server instance. This includes details such as the hostname the connection is coming from, as well as usernames, session ID’s, program names, wait types, and more.

    By running this script we can see all the active queries at the time of execution, and using the information displayed within the columns we can easily get a picture of what’s happening. We’ll be able to see the longest running queries, and the number of rows returned is always a point of interest, if there’s 100’s of rows it might be harder to find a root cause.

    -- Show current sessions
    SELECT 
         s.host_name, 
         s.program_name, 
         r.blocking_session_id 'Blk by', 
         Coalesce(Quotename(Db_name(st.dbid)) + N'.' + Quotename(Object_schema_name(st.objectid,st.dbid)) + N'.' +
         Quotename(Object_name(st.objectid,st.dbid)), '') AS command_text, 
         s.session_id, 
         r.cpu_time, 
         r.logical_reads, 
         r.reads, 
         r.status, 
         r.wait_type, 
         wait_resource, 
         r.wait_time / (1000.0) 'Wait (in sec)', 
         r.writes, 
         r.total_elapsed_time / (1000.0) 'Elapsed (in sec)', 
         SUBSTRING(st.TEXT,(r.statement_start_offset / 2) + 1, 
         ((CASE r.statement_end_offset 
              WHEN -1 
              THEN Datalength(st.TEXT) 
              ELSE r.statement_end_offset 
              END - r.statement_start_offset) / 2) + 1) AS statement_text, 
         r.command, 
         s.login_name, 
         s.last_request_end_time, 
         s.login_time, 
         r.open_transaction_count 
    FROM  sys.dm_exec_sessions AS s 
    JOIN sys.dm_exec_requests AS r 
    ON r.session_id = s.session_id 
    CROSS APPLY sys.Dm_exec_sql_text(r.sql_handle) AS st 
    ORDER BY r.cpu_time DESC;
    
    sql server performance troubleshooting script

    If the script to show running queries returns a large number of rows, indicating a high level of activity, it suggests that the server is busy processing numerous concurrent queries, potentially leading to resource saturation and system instability. The reason you are seeing so many active sessions may also be due to the root issue, we should look for blocking SPIDs and find the lead blocker.

    SQL Server DBAs must monitor worker thread counts and query activity closely. Interpreting results effectively to proactively address performance issues is a key part of our day to day role to ensure optimal database operations.

    3. Additional Notes & Follow-Up Tasks

    Here are some suggestions on what we can be doing if troubleshooting a critical performance issue in SQL Server:

    Try optimise/recompile Stored Procedures
    Use sp_recompile to recompile stored procedures periodically, ensuring optimal execution plans. As indexes or statistics change, compiled procedures, triggers, and functions may become less efficient. Recompiling them can reoptimise queries for better performance.

    Identified a problem SQL query (Blocking/Locking/Resource Intense)?
    You might need to kill the SQL Session (SPID) for various reasons, including blocking, locking or resource heavy queries. My other post as linked here contains good info on terminating SQL Server sessions.

    Is your SQL Server part of Always On, Replication or Mirroring?
    The following scripts/articles may help identify if you have issues with High Availability SQL Features.
    1. Get Always On Availability Group (AAG) Latency
    2. Get Mirroring Status & Latency
    3. Get Replication Latency

    Reindexing/Updating Statistics
    Regularly maintain indexes and update statistics to ensure efficient query execution. This includes adding new indexes where required. Index optimisation should happen on regular schedules during non-busy periods.

    I hope these suggestions and explanations are helpful, cheers.

  • Checking SQL Server Mirroring Health & Latency

    In this blog post I’m sharing info on how to check the health and latency of your Database Mirroring in SQL Server.

    Monitoring Database Mirroring latency in SQL Server is needed for maintaining database availability and ensuring quick failover readiness. As a DBA we should be monitoring this if heavily relying on the Mirroring Feature for our High Availability (HA) setup.

    If Database Mirroring is lagging behind, I’ve added notes at the end to help with troubleshooting it. The intention of this post was more to help with helping identify if you have issues, check out my other post if you’re looking to jump right into the troubleshooting part.

    The following is included in this article:
    1. Use Database Mirroring Monitor.
    2. Use T-SQL to Get Mirroring Health & Latency.
    3. More Ways to Monitor Mirroring Health.
    4. Further Troubleshooting Mirroring Issues.

    I hope this is all useful and provide you good info on ways to check if mirroring is healthy!

    How to Check SQL Server Mirroring Status

    1. Use Database Mirroring Monitor

    In SQL Server Management Studio (SSMS), we have an easy way to view the state and health of Mirroring.

    To open Database Mirroring Monitor:
    -> Right-click the Mirroring database > select Tasks > Launch Database Mirroring Monitor.

    With the right permissions, you can use Database Mirroring Monitor to track the performance and data flow of mirrored databases, troubleshoot issues, and verify synchronization. You can register mirrored databases on each failover partner to cache details like the database name, partner server instances, and their last known roles (principal or mirror).

    Using this dashboard, we can quickly see if the mirroring state is unhealthy. It’ll show key mirroring issues, including: Unknown (monitor disconnected, cached data only), Synchronizing (mirror lagging behind principal), Suspended (principal not sending logs), and Disconnected (partners cannot connect).

    2. Use T-SQL to Get Mirroring Health & Latency

    We can check Mirroring sync status and Properties by running the script below on the Principal server.

    -- get health status of database mirroring
    SELECT 
       SERVERPROPERTY('ServerName') AS Principal, 
       m.mirroring_partner_instance AS Mirror,
       DB_NAME(m.database_id) AS DatabaseName,
       CAST(SUM(f.size * 8 / 1024.0 / 1024) AS DECIMAL(10,1)) AS DatabaseSizeGB,
       CASE m.mirroring_safety_level
          WHEN 1 THEN 'HIGH PERFORMANCE'
          WHEN 2 THEN 'HIGH SAFETY'
       END AS OperatingMode,
       RIGHT(m.mirroring_partner_name, CHARINDEX(':', REVERSE(m.mirroring_partner_name) + ':') - 1) AS Port,
       m.mirroring_state AS MirroringState,
       CASE 
          WHEN m.mirroring_state = 4 THEN 'SYNCHRONIZED'
          WHEN m.mirroring_state = 3 THEN 'PENDING FAILOVER'
          WHEN m.mirroring_state = 2 THEN 'SYNCHRONIZING'
          WHEN m.mirroring_state = 1 THEN 'DISCONNECTED'
          WHEN m.mirroring_state = 0 THEN 'SUSPENDED'
          ELSE 'UNKNOWN'
       END AS MirroringHealth,
       m.mirroring_witness_name AS WitnessName
    FROM sys.database_mirroring m
    JOIN sys.master_files f ON m.database_id = f.database_id
    WHERE m.mirroring_role_desc = 'PRINCIPAL'
      AND m.mirroring_state IS NOT NULL
    GROUP BY 
       m.mirroring_partner_instance, 
       m.database_id, 
       m.mirroring_safety_level, 
       m.mirroring_partner_name,
       m.mirroring_state, 
       m.mirroring_witness_name;
    

    The script provides database mirroring health metrics, including size, operating mode, state, and witness server details by querying the sys.database_mirroring system table, which tracks mirroring session information.

    We could add a JOIN on this table to sys.dm_os_performance_counters, which would allow us to calculate latency, giving us the estimated time for the mirror to catch up with the Principal. Since cntr_value is cumulative, we can determine log transfer rates by sampling at intervals, calculating the difference between samples, and dividing by elapsed time to measure mirroring performance.

    Instead of the above, we can also use system stored procedures, sp_dbmmonitorresults:

    USE msdb;
    GO
    EXEC sp_dbmmonitorresults databaseName, 2, 0; 
    

    Param 1: database name.
    Param 2: range of rows to retrieve, from the last row (0) to a specific time period (e.g., last two hours: 1) or a set number of rows (e.g., last 1,000: 8).
    Param3:
    0: uses the last two rows without refreshing the status. 1: Updates status if not refreshed in the last 15 seconds or if the user lacks sysadmin role.

    The MS Docs link above for this proc includes all the details on parameters, what’s returned, and what it means. Key metrics include role (Principal/Mirror), state (e.g., Synchronizing, Synchronized), log generation/sent/restored rates, transaction delays, and timestamps for monitoring sync status.

    3. More Ways to Monitor Mirroring Health

    While the methods above in this post are great for checking mirroring health and latency, there are other ways that can provide a more continuous approach to monitoring SQL Server mirroring, including:

    > Extended Events and Performance Counters: You can create custom sessions to monitor specific events related to database mirroring. This will offer real-time statistics on mirroring performance, including latency, log send/receive rates, and state changes. These methods are ideal for setting up continuous monitoring and alerts.
    > Third-Party Monitoring Tools: Various third-party monitoring tools can be used for more advanced monitoring setups. Tools like Redgate SQL Monitor or Idera SQL Diagnostic Manager provide easy-to-use dashboards and alerting mechanisms for mirroring latency, among other performance metrics.
    > Custom / In-house Monitoring: Build your own monitor. Collect the mirroring health data on a schedule and build a dashboard and/or alert. We can use the methods described above in this post for this.
    > SQL Server Profiler: Not the preferred choice, especially for continuous monitoring, but it’s there if wanted.

    4. Further Troubleshooting Mirroring Issues

    If you’re seeing high redo or send queue sizes, it could point to:
    Workload Spikes at Principal Mirror: Evaluate the workload on the principal server that might be overwhelming the mirror.
    Disk Performance: Ensure the mirror server can handle the redo operations efficiently.
    Network Issues: Check the bandwidth and reliability between the principal and mirror (or ask the network team).

    Other troubleshooting strategies include restarting Mirroring Endpoints, or SQL Service (starting with Secondary Mirror). For more information on steps to resolve SQL Server Database Mirroring issues, check out my other post: Troubleshooting Database Mirroring Issues in SQL Server