Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Intro to Deadlocks

•
•

This document discusses deadlocks in database systems. It explains that deadlocks occur when two or more competing actions get stuck waiting on each other to finish. It then provides an example of a deadlock between two dogs, Tony and Jake, fighting over bones. It demonstrates how to detect and debug deadlocks using MySQL status commands and log files. Finally, it offers best practices like defining proper indexes to avoid deadlocks and handling them through retry logic or manual locking.

1 of 11
Download to read offline
Intro to Deadlocks February 4th, 2013 Panupan Sriautharawong Monday, February 11, 13
What are Deadlocks? A situation that occurs in concurrent systems. Happens when two or more competing actions get stuck because they’re waiting on one another to finish. A bug or design flaw in most cases. Preventable! Monday, February 11, 13
InnoDB Storage Engine Monday, February 11, 13
InnoDB Storage Engine Transactions Are used to group of database operations ## Bank Transfer Example BEGIN; UPDATE account SET balance=balance+100 WHERE id=1; UPDATE account SET balance=balance-100 WHERE id=2; COMMIT; Are atomic, meaning they can’t broken into smaller pieces. Either everything goes through or nothing does. Monday, February 11, 13
InnoDB Storage Engine Shared vs Exclusive Locks A lock must be acquired (either implicitly or explicitly) before retrieving or modifying any data. There are two types of locks: - Shared (S) = Read only - Exclusive (X) = Read/Write Table vs Row-Level Locking Properly defined indexes prevent too many rows from unintentionally getting locked and also ensures optimal performance. Monday, February 11, 13
Deadlocked Doggies Jake Tony Tony starts with 5 bones Jake starts with 1 bone Action #1: Jake steals a bone from Tony Action #2: Tony, being a good doggie, gives Jake a bone so they’ll both have 3 Monday, February 11, 13
Deadlocked Doggies ## INITIAL SETUP CREATE TABLE `doggies` ( `id` INT NOT NULL AUTO_INCREMENT, Jake `name` VARCHAR(255) NOT NULL, `bones` INT NOT NULL, PRIMARY KEY (`id`) Tony ) ENGINE=InnoDB; INSERT INTO doggies (id, name, bones) VALUES (1, "Tony", 5); INSERT INTO doggies (id, name, bones) VALUES (2, "Jake", 1); ## ACTION #1 # Jake steals a bone BEGIN; UPDATE doggies SET bones=bones-1 WHERE id=1; # X lock acquired on 1 ## ACTION #2 # Tony gives Jake a bone BEGIN; UPDATE doggies SET bones=bones+1 WHERE id=2; # X lock acquired on 2 UPDATE doggies SET bones=bones-1 WHERE id=1;! # LOCK WAIT! COMMIT; ## ACTION #1 (continued) UPDATE doggies SET bones=bones+1 WHERE id=2;! # DEADLOCK! COMMIT; Monday, February 11, 13
How to Debug’em Inspecting MySQL Status mysql> SHOW ENGINE INNODB STATUS; mysql> SHOW PROCESSLIST; mysql> EXPLAIN SELECT ...; Grepping Rails Logs $ cat log/production.log | grep -B 40 Deadlock Monday, February 11, 13
------------------------ How to Debug’em LATEST DETECTED DEADLOCK ------------------------ 130204 13:10:32 *** (1) TRANSACTION: TRANSACTION 214FC, ACTIVE 12 sec starting index read mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s), undo log entries 1 MySQL thread id 182, OS thread handle 0x111c8c000, query id 120976 localhost 127.0.0.1 root Updating UPDATE doggies SET bones=bones-1 WHERE id=1 *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 0 page no 1130 n bits 72 index `PRIMARY` of table `deadlocks`.`doggies` trx id 214FC lock_mode X locks rec but not gap waiting Record lock, heap no 2 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 80000001; asc ;; 1: len 6; hex 0000000214fb; asc ;; 2: len 7; hex 7900000de72774; asc y 't;; 3: len 4; hex 546f6e79; asc Tony;; 4: len 4; hex 80000004; asc ;; *** (2) TRANSACTION: TRANSACTION 214FB, ACTIVE 44 sec starting index read mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s), undo log entries 1 MySQL thread id 184, OS thread handle 0x111e2b000, query id 120977 localhost 127.0.0.1 root Updating ## CONNECTION #1 # Jake steals a bone (cont..) UPDATE doggies SET bones=bones+1 WHERE id=2 *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 0 page no 1130 n bits 72 index `PRIMARY` of table `deadlocks`.`doggies` trx id 214FB lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 80000001; asc ;; 1: len 6; hex 0000000214fb; asc ;; 2: len 7; hex 7900000de72774; asc y 't;; 3: len 4; hex 546f6e79; asc Tony;; 4: len 4; hex 80000004; asc ;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 0 page no 1130 n bits 72 index `PRIMARY` of table `deadlocks`.`doggies` trx id 214FB lock_mode X locks rec but not gap waiting Record lock, heap no 3 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 80000002; asc ;; 1: len 6; hex 0000000214fc; asc ;; 2: len 7; hex 7a00000cf122d8; asc z " ;; 3: len 4; hex 4a616b65; asc Jake;; 4: len 4; hex 80000002; asc ;; *** WE ROLL BACK TRANSACTION (2) Monday, February 11, 13
Best Practices Define proper indexes and use EXPLAIN SELECT to verify. For example, make sure to define multi-column indexes across [id, type] on our polymorphic models. Most common method of deadlock exception handling is to retry but this should only be used a last resort. Easiest fix is to rearrange queries to make sure updates occur before selects (not always possible). You can also lock records manually at the beginning of a transaction when you know they’ll need updating. Monday, February 11, 13
Helpful Resources http://dev.mysql.com/doc/refman/5.0/en/ innodb-lock-modes.html http://rhnh.net/2010/07/01/debugging- deadlocks-in-rails http://www.chriscalender.com/?p=426 Monday, February 11, 13

More Related Content

Intro to Deadlocks

  • 1. Intro to Deadlocks February 4th, 2013 Panupan Sriautharawong Monday, February 11, 13
  • 2. What are Deadlocks? A situation that occurs in concurrent systems. Happens when two or more competing actions get stuck because they’re waiting on one another to finish. A bug or design flaw in most cases. Preventable! Monday, February 11, 13
  • 3. InnoDB Storage Engine Monday, February 11, 13
  • 4. InnoDB Storage Engine Transactions Are used to group of database operations ## Bank Transfer Example BEGIN; UPDATE account SET balance=balance+100 WHERE id=1; UPDATE account SET balance=balance-100 WHERE id=2; COMMIT; Are atomic, meaning they can’t broken into smaller pieces. Either everything goes through or nothing does. Monday, February 11, 13
  • 5. InnoDB Storage Engine Shared vs Exclusive Locks A lock must be acquired (either implicitly or explicitly) before retrieving or modifying any data. There are two types of locks: - Shared (S) = Read only - Exclusive (X) = Read/Write Table vs Row-Level Locking Properly defined indexes prevent too many rows from unintentionally getting locked and also ensures optimal performance. Monday, February 11, 13
  • 6. Deadlocked Doggies Jake Tony Tony starts with 5 bones Jake starts with 1 bone Action #1: Jake steals a bone from Tony Action #2: Tony, being a good doggie, gives Jake a bone so they’ll both have 3 Monday, February 11, 13
  • 7. Deadlocked Doggies ## INITIAL SETUP CREATE TABLE `doggies` ( `id` INT NOT NULL AUTO_INCREMENT, Jake `name` VARCHAR(255) NOT NULL, `bones` INT NOT NULL, PRIMARY KEY (`id`) Tony ) ENGINE=InnoDB; INSERT INTO doggies (id, name, bones) VALUES (1, "Tony", 5); INSERT INTO doggies (id, name, bones) VALUES (2, "Jake", 1); ## ACTION #1 # Jake steals a bone BEGIN; UPDATE doggies SET bones=bones-1 WHERE id=1; # X lock acquired on 1 ## ACTION #2 # Tony gives Jake a bone BEGIN; UPDATE doggies SET bones=bones+1 WHERE id=2; # X lock acquired on 2 UPDATE doggies SET bones=bones-1 WHERE id=1;! # LOCK WAIT! COMMIT; ## ACTION #1 (continued) UPDATE doggies SET bones=bones+1 WHERE id=2;! # DEADLOCK! COMMIT; Monday, February 11, 13
  • 8. How to Debug’em Inspecting MySQL Status mysql> SHOW ENGINE INNODB STATUS; mysql> SHOW PROCESSLIST; mysql> EXPLAIN SELECT ...; Grepping Rails Logs $ cat log/production.log | grep -B 40 Deadlock Monday, February 11, 13
  • 9. ------------------------ How to Debug’em LATEST DETECTED DEADLOCK ------------------------ 130204 13:10:32 *** (1) TRANSACTION: TRANSACTION 214FC, ACTIVE 12 sec starting index read mysql tables in use 1, locked 1 LOCK WAIT 3 lock struct(s), heap size 376, 2 row lock(s), undo log entries 1 MySQL thread id 182, OS thread handle 0x111c8c000, query id 120976 localhost 127.0.0.1 root Updating UPDATE doggies SET bones=bones-1 WHERE id=1 *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 0 page no 1130 n bits 72 index `PRIMARY` of table `deadlocks`.`doggies` trx id 214FC lock_mode X locks rec but not gap waiting Record lock, heap no 2 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 80000001; asc ;; 1: len 6; hex 0000000214fb; asc ;; 2: len 7; hex 7900000de72774; asc y 't;; 3: len 4; hex 546f6e79; asc Tony;; 4: len 4; hex 80000004; asc ;; *** (2) TRANSACTION: TRANSACTION 214FB, ACTIVE 44 sec starting index read mysql tables in use 1, locked 1 3 lock struct(s), heap size 376, 2 row lock(s), undo log entries 1 MySQL thread id 184, OS thread handle 0x111e2b000, query id 120977 localhost 127.0.0.1 root Updating ## CONNECTION #1 # Jake steals a bone (cont..) UPDATE doggies SET bones=bones+1 WHERE id=2 *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 0 page no 1130 n bits 72 index `PRIMARY` of table `deadlocks`.`doggies` trx id 214FB lock_mode X locks rec but not gap Record lock, heap no 2 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 80000001; asc ;; 1: len 6; hex 0000000214fb; asc ;; 2: len 7; hex 7900000de72774; asc y 't;; 3: len 4; hex 546f6e79; asc Tony;; 4: len 4; hex 80000004; asc ;; *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 0 page no 1130 n bits 72 index `PRIMARY` of table `deadlocks`.`doggies` trx id 214FB lock_mode X locks rec but not gap waiting Record lock, heap no 3 PHYSICAL RECORD: n_fields 5; compact format; info bits 0 0: len 4; hex 80000002; asc ;; 1: len 6; hex 0000000214fc; asc ;; 2: len 7; hex 7a00000cf122d8; asc z " ;; 3: len 4; hex 4a616b65; asc Jake;; 4: len 4; hex 80000002; asc ;; *** WE ROLL BACK TRANSACTION (2) Monday, February 11, 13
  • 10. Best Practices Define proper indexes and use EXPLAIN SELECT to verify. For example, make sure to define multi-column indexes across [id, type] on our polymorphic models. Most common method of deadlock exception handling is to retry but this should only be used a last resort. Easiest fix is to rearrange queries to make sure updates occur before selects (not always possible). You can also lock records manually at the beginning of a transaction when you know they’ll need updating. Monday, February 11, 13
  • 11. Helpful Resources http://dev.mysql.com/doc/refman/5.0/en/ innodb-lock-modes.html http://rhnh.net/2010/07/01/debugging- deadlocks-in-rails http://www.chriscalender.com/?p=426 Monday, February 11, 13