sql server 删除上千万的数据

1.准备工作

当你对一张存放了上千万行数据表进行删除时，首先要做的工作如下：

1.1 备份整个数据库

当一张表里所存放的数据行数达到上千万行的时候，一般情况下数据库可能都会有好几十个G。这个时候，如果备份失败。那可能是因为你限制了数据库事务日志的增长上限。

--设置数据库日志文件增长方式为，无限制增长
    ALTER DATABASE court_juror
	MODIFY FILE(
		NAME = court_juror_log,
		MAXSIZE =UNLIMITED , -- 指定文件将增长到磁盘充满
		FILEGROWTH = 5mb    -- 指定文件的自动增量
	);
           

1.2设置数据库为简单恢复模式

--将数据库设置为简单恢复模式
USE my_database ;  
ALTER DATABASE [model] SET RECOVERY SIMPLE ;
           

1.3检查你要删除的这样表的索引碎片情况，重新组织生成索引。

1.3.1重新生成或重新组织索引

自动重新组织或重新生成整个数据库中平均碎片超过 10％ 的所有分区

-- Ensure a USE <databasename> statement has been executed first.  
SET NOCOUNT ON;  
DECLARE @objectid int;  
DECLARE @indexid int;  
DECLARE @partitioncount bigint;  
DECLARE @schemaname nvarchar(130);   
DECLARE @objectname nvarchar(130);   
DECLARE @indexname nvarchar(130);   
DECLARE @partitionnum bigint;  
DECLARE @partitions bigint;  
DECLARE @frag float;  
DECLARE @command nvarchar(4000);   
-- Conditionally select tables and indexes from the sys.dm_db_index_physical_stats function   
-- and convert object and index IDs to names.  
SELECT  
    object_id AS objectid,  
    index_id AS indexid,  
    partition_number AS partitionnum,  
    avg_fragmentation_in_percent AS frag  
INTO #work_to_do  
FROM sys.dm_db_index_physical_stats (DB_ID('数据库'), NULL, NULL , NULL, 'LIMITED')  
WHERE avg_fragmentation_in_percent > 10.0 AND index_id > 0;  
  
-- Declare the cursor for the list of partitions to be processed.  
DECLARE partitions CURSOR FOR SELECT * FROM #work_to_do;  
  
-- Open the cursor.  
OPEN partitions;  
  
-- Loop through the partitions.  
WHILE (1=1)  
    BEGIN;  
        FETCH NEXT  
           FROM partitions  
           INTO @objectid, @indexid, @partitionnum, @frag;  
        IF @@FETCH_STATUS < 0 BREAK;  
        SELECT @objectname = QUOTENAME(o.name), @schemaname = QUOTENAME(s.name)  
        FROM sys.objects AS o  
        JOIN sys.schemas as s ON s.schema_id = o.schema_id  
        WHERE o.object_id = @objectid;  
        SELECT @indexname = QUOTENAME(name)  
        FROM sys.indexes  
        WHERE  object_id = @objectid AND index_id = @indexid;  
        SELECT @partitioncount = count (*)  
        FROM sys.partitions  
        WHERE object_id = @objectid AND index_id = @indexid;  
  
-- 30 is an arbitrary decision point at which to switch between reorganizing and rebuilding.  
        IF @frag < 30.0  
            SET @command = N'ALTER INDEX ' + @indexname + N' ON ' + @schemaname + N'.' + @objectname + N' REORGANIZE';  
        IF @frag >= 30.0  
            SET @command = N'ALTER INDEX ' + @indexname + N' ON ' + @schemaname + N'.' + @objectname + N' REBUILD';  
        IF @partitioncount > 1  
            SET @command = @command + N' PARTITION=' + CAST(@partitionnum AS nvarchar(10));  
        EXEC (@command);  
        PRINT N'Executed: ' + @command;  
    END;  
  
-- Close and deallocate the cursor.  
CLOSE partitions;  
DEALLOCATE partitions;  
  
-- Drop the temporary table.  
DROP TABLE #work_to_do;  
GO
           

1.3.2检测行存储索引中的碎片

DECLARE @db_id SMALLINT;  
DECLARE @object_id INT;  
--获取数据id
SET @db_id = DB_ID(N'数据库');  
--获取表id
SET @object_id = OBJECT_ID(N'要检测的表');  
-- 判断数据库和表是否存在
IF @db_id IS NULL  
BEGIN;  
    PRINT N'Invalid database';  
END;  
ELSE IF @object_id IS NULL  
BEGIN;  
    PRINT N'Invalid object';  
END;  
--查询碎片情况
ELSE  
BEGIN;  
    SELECT avg_fragmentation_in_percent,fragment_count,avg_fragment_size_in_pages FROM sys.dm_db_index_physical_stats(@db_id, @object_id, NULL, NULL , 'LIMITED');  
END;  
           

2.开始删除

while 1=1
	begin
		delete top(100000) from my_table where pDate <'2020-1-1'
	end
           

在删除的时候，执行的的效率可能会特别的缓慢。通过，参考SQL server官方文档我们知道执行效率慢是因为索引碎片的原因。

参考：

通过重新组织或重新生成索引来解决索引碎片问题

https://docs.microsoft.com/zh-cn/sql/relational-databases/indexes/reorganize-and-rebuild-indexes?view=sql-server-ver15

sys.dm_db_index_physical_stats (Transact-SQL)

https://docs.microsoft.com/zh-cn/sql/relational-databases/system-dynamic-management-views/sys-dm-db-index-physical-stats-transact-sql?view=sql-server-ver15#examples

查看或更改数据库的恢复模式 (SQL Server)

https://docs.microsoft.com/zh-cn/sql/relational-databases/backup-restore/view-or-change-the-recovery-model-of-a-database-sql-server?view=sql-server-ver15

恢复模式 (SQL Server)

https://docs.microsoft.com/zh-cn/sql/relational-databases/backup-restore/recovery-models-sql-server?view=sql-server-ver15