sql server 删除上千萬的資料

1.準備工作

當你對一張存放了上千萬行資料表進行删除時，首先要做的工作如下：

1.1 備份整個資料庫

當一張表裡所存放的資料行數達到上千萬行的時候，一般情況下資料庫可能都會有好幾十個G。這個時候，如果備份失敗。那可能是因為你限制了資料庫事務日志的增長上限。

--設定資料庫日志檔案增長方式為，無限制增長
    ALTER DATABASE court_juror
	MODIFY FILE(
		NAME = court_juror_log,
		MAXSIZE =UNLIMITED , -- 指定檔案将增長到磁盤充滿
		FILEGROWTH = 5mb    -- 指定檔案的自動增量
	);
           

1.2設定資料庫為簡單恢複模式

--将資料庫設定為簡單恢複模式
USE my_database ;  
ALTER DATABASE [model] SET RECOVERY SIMPLE ;
           

1.3檢查你要删除的這樣表的索引碎片情況，重新組織生成索引。

1.3.1重新生成或重新組織索引

自動重新組織或重新生成整個資料庫中平均碎片超過 10％ 的所有分區

-- Ensure a USE <databasename> statement has been executed first.  
SET NOCOUNT ON;  
DECLARE @objectid int;  
DECLARE @indexid int;  
DECLARE @partitioncount bigint;  
DECLARE @schemaname nvarchar(130);   
DECLARE @objectname nvarchar(130);   
DECLARE @indexname nvarchar(130);   
DECLARE @partitionnum bigint;  
DECLARE @partitions bigint;  
DECLARE @frag float;  
DECLARE @command nvarchar(4000);   
-- Conditionally select tables and indexes from the sys.dm_db_index_physical_stats function   
-- and convert object and index IDs to names.  
SELECT  
    object_id AS objectid,  
    index_id AS indexid,  
    partition_number AS partitionnum,  
    avg_fragmentation_in_percent AS frag  
INTO #work_to_do  
FROM sys.dm_db_index_physical_stats (DB_ID('資料庫'), NULL, NULL , NULL, 'LIMITED')  
WHERE avg_fragmentation_in_percent > 10.0 AND index_id > 0;  
  
-- Declare the cursor for the list of partitions to be processed.  
DECLARE partitions CURSOR FOR SELECT * FROM #work_to_do;  
  
-- Open the cursor.  
OPEN partitions;  
  
-- Loop through the partitions.  
WHILE (1=1)  
    BEGIN;  
        FETCH NEXT  
           FROM partitions  
           INTO @objectid, @indexid, @partitionnum, @frag;  
        IF @@FETCH_STATUS < 0 BREAK;  
        SELECT @objectname = QUOTENAME(o.name), @schemaname = QUOTENAME(s.name)  
        FROM sys.objects AS o  
        JOIN sys.schemas as s ON s.schema_id = o.schema_id  
        WHERE o.object_id = @objectid;  
        SELECT @indexname = QUOTENAME(name)  
        FROM sys.indexes  
        WHERE  object_id = @objectid AND index_id = @indexid;  
        SELECT @partitioncount = count (*)  
        FROM sys.partitions  
        WHERE object_id = @objectid AND index_id = @indexid;  
  
-- 30 is an arbitrary decision point at which to switch between reorganizing and rebuilding.  
        IF @frag < 30.0  
            SET @command = N'ALTER INDEX ' + @indexname + N' ON ' + @schemaname + N'.' + @objectname + N' REORGANIZE';  
        IF @frag >= 30.0  
            SET @command = N'ALTER INDEX ' + @indexname + N' ON ' + @schemaname + N'.' + @objectname + N' REBUILD';  
        IF @partitioncount > 1  
            SET @command = @command + N' PARTITION=' + CAST(@partitionnum AS nvarchar(10));  
        EXEC (@command);  
        PRINT N'Executed: ' + @command;  
    END;  
  
-- Close and deallocate the cursor.  
CLOSE partitions;  
DEALLOCATE partitions;  
  
-- Drop the temporary table.  
DROP TABLE #work_to_do;  
GO
           

1.3.2檢測行存儲索引中的碎片

DECLARE @db_id SMALLINT;  
DECLARE @object_id INT;  
--擷取資料id
SET @db_id = DB_ID(N'資料庫');  
--擷取表id
SET @object_id = OBJECT_ID(N'要檢測的表');  
-- 判斷資料庫和表是否存在
IF @db_id IS NULL  
BEGIN;  
    PRINT N'Invalid database';  
END;  
ELSE IF @object_id IS NULL  
BEGIN;  
    PRINT N'Invalid object';  
END;  
--查詢碎片情況
ELSE  
BEGIN;  
    SELECT avg_fragmentation_in_percent,fragment_count,avg_fragment_size_in_pages FROM sys.dm_db_index_physical_stats(@db_id, @object_id, NULL, NULL , 'LIMITED');  
END;  
           

2.開始删除

while 1=1
	begin
		delete top(100000) from my_table where pDate <'2020-1-1'
	end
           

在删除的時候，執行的的效率可能會特别的緩慢。通過，參考SQL server官方文檔我們知道執行效率慢是因為索引碎片的原因。

參考：

通過重新組織或重新生成索引來解決索引碎片問題

https://docs.microsoft.com/zh-cn/sql/relational-databases/indexes/reorganize-and-rebuild-indexes?view=sql-server-ver15

sys.dm_db_index_physical_stats (Transact-SQL)

https://docs.microsoft.com/zh-cn/sql/relational-databases/system-dynamic-management-views/sys-dm-db-index-physical-stats-transact-sql?view=sql-server-ver15#examples

檢視或更改資料庫的恢複模式 (SQL Server)

https://docs.microsoft.com/zh-cn/sql/relational-databases/backup-restore/view-or-change-the-recovery-model-of-a-database-sql-server?view=sql-server-ver15

恢複模式 (SQL Server)

https://docs.microsoft.com/zh-cn/sql/relational-databases/backup-restore/recovery-models-sql-server?view=sql-server-ver15