DBA Diaries

Thoughts and experiences of a DBA working with SQL Server and MySQL

Andy Hayes is a DBA working with SQL Server since version 7.0. He has a wonderful wife and two beautiful children. He loves database technology, playing cricket, and blogging. He is passionate about sharing his experiences as a DBA and learning more to further his understanding and knowledge. You can follow me on Twitter, check out my Facebook page or follow me on Google+

How to Find Buffer Pool Usage Per Database in SQL Server

Posted on May 10, 2016 Written by Andy Hayes 1 Comment

As a DBA it’s important to understand what the buffer pool is doing and which databases are using it the most.

Data in SQL Server is stored on disk in 8k pages. The buffer pool (Aka “buffer cache”) is a chunk of memory allocated to SQL Server. It’s size is determined by the minimum and maximum memory settings in SQL Server memory options:

sp_configure 'min server memory (MB)'
go
sp_configure 'max server memory (MB)';
name                                minimum     maximum     config_value run_value
----------------------------------- ----------- ----------- ------------ -----------
min server memory (MB)              0           2147483647  0            16

name                                minimum     maximum     config_value run_value
----------------------------------- ----------- ----------- ------------ -----------
max server memory (MB)              128         2147483647  2147483647   2147483647

Data is processed via the buffer pool. When an application requests some data for reading and those pages are not in the buffer pool, the server has to read them from disk. These are known as physical reads. If those pages have been read already into the buffer pool, these are known as logical reads and will typically return to the application more quickly than the physical reads – memory is faster than disk.

The buffer pool is also where modifications are made to pages (dirty pages) before those changes are persisted to storage.

If sufficient memory is not available, pages are aged out of the buffer and space is made available for the pages being read from disk when they are requested. Excessive “churn” of pages in the buffer pool could indicate that it is sized too small. The Page Life Expectancy counter can be used to help diagnose if this is a problem on your server.

The buffer pool is a critical area of the system that must be adequately sized for optimal performance.

SQL Server can tell you how many of those pages reside in the buffer pool. It can also tell you which databases those pages belong to. We can use sys.dm_os_buffer_descriptors to provide this information as it returns a row for each page found in the buffer pool at a database level.

SELECT 
  CASE WHEN database_id = 32767 THEN 'ResourceDB' ELSE DB_NAME(database_id) END AS DatabaseName,
  COUNT(*) AS cached_pages,
  (COUNT(*) * 8.0) / 1024 AS MBsInBufferPool
FROM
  sys.dm_os_buffer_descriptors
GROUP BY
  database_id
ORDER BY
  MBsInBufferPool DESC
GO
DatabaseName                   cached_pages MBsInBufferPool
------------------------------ ------------ ---------------------------------------
ResourceDB                     2968         23.187500
AdventureWorks2012             2614         20.421875
msdb                           438          3.421875
master                         390          3.046875
ReportServer                   332          2.593750
AdventureWorksDW2012           245          1.914062
tempdb                         225          1.757812
PerformanceDW                  205          1.601562
ReportServerTempDB             172          1.343750
Test                           155          1.210937
model                          66           0.515625

(17 row(s) affected)

The data tells us what is in the buffer pool at that moment in time. These results can change quickly depending on the activity of the server. Tracking this data over the course of a day for example would require some kind of job to periodically run this query and capture the output to file or logging table.

The data can be reset if the buffer pool is emptied either upon a SQL Server restart or by executing DBCC DROPCLEANBUFFERS which will clear out the unmodified (clean) pages from the buffer pool.

Emptying the buffer pool of clean pages by running DBCC DROPCLEANBUFFERS will increase load on the disks and decrease application performance until the cache fills again.

Filed Under: All Articles, SQL Server Performance Tagged With: performance, sql server

Revoking Access for All Users of a Database in SQL Server

Posted on May 9, 2016 Written by Andy Hayes Leave a Comment

I love Management Studio for SQL Server but sometimes it just doesn’t cut the mustard. I had a scenario recently where I needed to revoke access for all users from a database. I could delete them all but that would be more effort than necessary. I just wanted to deny access to a specific database.

To my knowledge, there is not a quick way to do this and so I had to write some T-SQL to do it.

You can revoke a database user using syntax REVOKE CONNECT, for example

REVOKE CONNECT FROM my_user;

What if you have many SQL Server users and you want to stop all of them accessing the database?

Identify them using this query from sysusers. In my case I am using an additional filter to pull out users with a specific prefix.

SELECT name FROM sysusers WHERE hasdbaccess = 1 AND name LIKE 'user{96f5cbaad583fb885ff5d18ecce5734bf1e8596949521e20c290401929518f75}';

In my case, this returns:

name
-----------
user_1
user_2
user_3

(3 row(s) affected)

To do this, I made a small modification to this query using some concatenation and square brackets in case of weird user names which contain hyphens:

SELECT 'REVOKE CONNECT FROM [' + name + '];' FROM sysusers WHERE hasdbaccess = 1 AND name LIKE 'user{96f5cbaad583fb885ff5d18ecce5734bf1e8596949521e20c290401929518f75}';

This returned and I know have a series of statements that I can run as one batch:

-------------------------------
REVOKE CONNECT FROM [user_1];
REVOKE CONNECT FROM [user_2];
REVOKE CONNECT FROM [user_3];

(3 row(s) affected)

Copy and paste these into a new query and execute them. Test that access has been revoked either by looking in Management Studio or querying sysusers for hasdbaccess = 1 again.

SELECT COUNT(*) FROM sysusers WHERE hasdbaccess = 1 AND name LIKE 'user{96f5cbaad583fb885ff5d18ecce5734bf1e8596949521e20c290401929518f75}';

If you’re looking in Management Studio at the users, then a red arrow indicates that the user has been revoked.

evoke access for all users in a sql server database

Filed Under: All Articles, SQL Server Administration Tagged With: sql server

How to Determine Which Port MySQL is Running On

Posted on May 9, 2016 Written by Andy Hayes Leave a Comment

MySQL defaults to port 3306 when installed but sometimes for another reason, it may be listening on a different port. In this post, I list some ways in which you can find out which port your MySQL instance is running on.

Using the MySQL configuration file to determine which port it is running on

If you are running linux, then this is an easy one liner. On my test machine, my config is stored in /etc/mysql:

cat /etc/mysql/my.cnf | grep port

On my test machine this returns

port            = 3306

If you are running Windows, then find your my.ini file and open it in a text editor such as notepad. Perform a search for the term – “port”

Using the MySQL client to determine the MySQL port

MySQL can tell you which port it is running on. Log into it and use the “show variables like ‘port{3a76cbff1b5ce5ace3bd34bbdd68f71285d81fb9af60114c7a6db80d3a8688de}'”;

[email protected]:~# mysql -uroot -p;
mysql> show global variables like 'port{96f5cbaad583fb885ff5d18ecce5734bf1e8596949521e20c290401929518f75}';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| port          | 3306  |
+---------------+-------+
1 row in set (0.00 sec)

If you are using a client such as MySQL Workbench, you use the above syntax or click on “Server Status” over on the left hand side, for example in Windows, it looks like this:

how to find mysql port using MySQL workbench

Using the netstat command to check which port MySQL is running on

If you’re wanting to know how to check if mysql is running then there are some useful one line commands that can be run.

This command is very useful as a mysql port check either on a unix or windows operation system – “netstat” which is short for “network statistics”

netstat -tln | grep mysql

In the results below (unix OS), the port is listed in this section as “0 0.0.0.0:3306”

tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      1115/mysqld

If running on windows, you could do

netstat -anob

It’s a little hard then to find the one which is relevant as the list can be quite large, scrolling past quickly. It’s possible to pipe it to make the output page or dump it to file and then look for mysqld.exe in the list.

Paging example

netstat -anob | more

Dumping to file example

netstat -anob > output.txt

In each case find the mysqld.exe lines where the status is LISTENING

Filed Under: All Articles, MySQL Administration Tagged With: mysql

MySQL Replication Events – Statement versus Row-Based Formats

Posted on May 8, 2016 Written by Andy Hayes Leave a Comment

In a recent post, I briefly touched upon mysql replication events. It is important to know the advantages and disadvantages of each type.

There are two types of replication event:

Statement based – these are the write operations. They written to the binary log as SQL statements.

Row based – these are the row changes and they are written to the binary log as complete copies of the rows.

It’s also possible to configure the server as MIXED which is a combination of both statement and row based replication. We’ll talk about that in a minute…

When setting up replication, in order to specify a particular replication event format, the binlog_format setting is configured in my.cnf (my.ini if using Windows)

Is there a best option ?

Let’s expand first on the differences between statement based and row based replication in MySQL.

Statement based

With this format, every SQL statement that could perform a modify operation is logged to the binary log. Those same statements are then executed on the slaves by the SQL thread.

If the application makes five update operations to a table on the master, those same five statements are executed on the slave against the same dataset and in the same order.

Row based

These are complete copies of the rows that are written to the binary log and subsequently applied to the slaves.

If an application modified the same row five times, then five copies of that row would be written to the binary log and applied to the slaves. A modification might only be an update to a single field on the row but in row based replication, this operation is replicated as the entire row and not a single field update.

What are the advantages of statement based replication in MySQL?

  • Binary logs are smaller so less disk space is needed for them
  • Less data is transferred between the master and slaves
  • Auditing changes made to a database could be done by looking in the binary logs using a tool like mysqlbinlog

What are the disadvantages of statement based replication in MySQL?

  • Non-deterministic queries can produce results on the slave than are different to what were returned on the master

This points deserves padding out as will cause data consistency problems.

There are many non-deterministic operations which are considered as unsafe when used with replication.

An example of a non-deterministic operation would be where the function UUID() is used to a universal unique identifier. This will return a different result upon each execution. Having this function inside an insert or update statement for example would produce different results on both the master and the slave because the SQL statements generated on the master and executed on the slave. This behaviour applies to not only ad-hoc SQL statements but also to stored procedures and UDF’s.

  • Greater numbers of row level locks may be necessary, for example if the operation was a INSERT…SELECT. This can introduce a performance penalty.
  • There is a lot of extra context work needed to ensure that update operations are executed in the correct order

What are the advantages of row based replication in MySQL?

  • The entire row is replicated. This avoids all of the issues with non-deterministic operations found in ad-hoc SQL, stored procedures and UDF’s
  • No context information is needed to help guarantee operations are executed in the correct order
  • Fewer row level locks are needed

What are the disadvantages of row based replication in MySQL?

  • More data can be generated, increasing disk space requirements, network traffic and I/O
  • The data is logged in binary format, making it harder to audit the changes to the database using the binary logs

Conclusion – statement based, row based replication or mixed replication

In my view it really depends 🙂 Data consistency should be the number one priority and this has to factor heavily in any decision.

Assess requirements based on the application. If you know for sure that the application is not using non-deterministic functions, then statement based replication would be sufficient but how can you guarantee this as a DBA?

As a DBA you might be responsible for introducing stored procedures and udf’s into production. This gives you a chance to review things before they are live and non-deterministic functions may raise a red flag. However ad-hoc SQL inside the application would be harder to review. In a statement based configuration, data consistency cannot be guaranteed between the master and the slave in this scenario.

So let’s go with row based – no wait, our application is write heavy, that’s going to cost more in hardware storage and potentially affect I/O performance!

Given the above, I would choose MIXED which switches between statement and row based replication depending on the operations performed. This helps keep binary logs in check whilst helping to guarantee data consistency between the master and the slave.

If I had to choose between statement and row based and storage, I/O etc was not an issue, I would stick to row based replication given the extra peace of mind it provides.

 

Filed Under: All Articles, MySQL Administration Tagged With: mysql, replication

  • « Previous Page
  • 1
  • …
  • 3
  • 4
  • 5
  • 6
  • 7
  • …
  • 22
  • Next Page »

Categories

  • All Articles (82)
  • Career Development (8)
  • MySQL Administration (18)
  • MySQL Performance (2)
  • SQL Server Administration (24)
  • SQL Server News (3)
  • SQL Server Performance (14)
  • SQL Server Security (3)
  • SQL Tips and Tricks (19)

Top 10 Popular Posts

  • Using sp_change_users_login to fix SQL Server orphaned users
  • How to shrink tempdb
  • MySQL SHOW USERS? – How to List All MySQL Users and Privileges
  • How to Transfer Logins to Another SQL Server or Instance
  • How to Delete Millions of Rows using T-SQL with Reduced Impact
  • T-SQL – How to Select Top N Rows for Each Group Using ROW_NUMBER()
  • New T-SQL features in SQL Server 2012 – OFFSET and FETCH
  • How to Kill All MySQL Processes For a Specific User
  • Using exec sp_who2 to help with SQL Server troubleshooting
  • How to move tempdb

Recent Posts

  • How to Setup MySQL Master Master Replication
  • How To Use SQL to Convert a STRING to an INT
  • How to set up MySQL Replication Tutorial
  • How to Use SQL CASE for Conditional Logic in Your SQL Queries
  • Using ISNULL in SQL Server to Replace NULL Values

Search

Connect

  • Twitter
  • Facebook
  • Google+
  • RSS

About

  • Cookie Policy
  • Disclaimer
  • About
Copyright © ‘2021’ DBA Diaries built on the Genesis Framework

This site uses cookies. We assume you are happy with cookies but click the link if you are not. Close