В пуле ресурсов "внутренний" недостаточно системной памяти
SQL Server 2008 Linked Server и Ad-hoc вставки вызывают быструю утечку памяти, которая в конечном итоге приводит к тому, что сервер становится не реагирующим и заканчивается следующей ошибкой:
Msg 701, Level 17, State 123, Server BRECK-PCSQLEXPRESS, Line 2
There is insufficient system memory in resource pool 'internal' to run this
query.
Location: qxcntxt.cpp:1052
Expression: cref == 0
SPID: 51
Process ID: 1880
сервер остается не реагирующим до перезапуска SQL Server.
программное обеспечение в использовании:
Windows Vista Ultimate 64 бит построить 6001 SP1
Microsoft SQL Server 2008 (SP1) - 10.0.2734.0 (X64) Сентябрь 11 2009 14:30: 58 Copyright (c) 1988-2008 Microsoft Corporation Express Edition с расширенными службами (64-разрядная версия) в Windows NT 6.0 (сборка 6001: пакет обновления 1)
SAOLEDB.11 драйвер из SQL Anywhere 11.0.1.2276
установка max server memory (MB) в 2048 не помогла.
добавление различных значений-g (например, -g256;) к параметрам запуска сервера не помогло.
использование DBCC FREESYSTEMCACHE ('ALL'), DBCC FREESESSIONCACHE и DBCC FREEPROCCACHE не помог.
установка пакета обновления 4 Cumnulative в пакет обновления 1 для SQL Server 2008 не помогла, хотя он содержал исправление симптома утечки памяти, связанного с использованием сервера.
разделение выбора ... ROW_NUMBER () OVER ... запрос от INSERT не помог. Эксперименты показали, что сложный выбор не вызвал утечку памяти, это сделала вставка.
изменение кода для использования ad-hoc " вставить в Синтаксис OPENROWSET " вместо связанного сервера не помог; приведенный ниже код показывает использование связанного сервера.
sysinternals.com утилита Process Explore показывает, что использование памяти было связано с sqlserver.exe, а не DLL, используемые драйвером SQL Anywhere OLEDB SAOLEDB.11.
обратите внимание, что SQL Anywhere версия связанного сервера (прокси-таблицы) работает нормально, чтобы "вытащить" 1,9 миллиона строк из таблицы SQL Server 2008 в базу данных SQL Anywhere 11 в одной транзакции. Логика, показанная здесь, - это попытка использовать функцию связанного сервера для" выталкивания " строк; то же направление, другой синтаксис.
код следует; 4G ОЗУ исчерпывается после трех или четырех исполнений EXECUTE copy_mss_t2:
EXEC sys.sp_configure
N'show advanced options',
N'1'
GO
RECONFIGURE WITH OVERRIDE
GO
EXEC sys.sp_configure
N'max server memory (MB)',
N'2048'
GO
RECONFIGURE WITH OVERRIDE
GO
EXEC sys.sp_configure
N'show advanced options',
N'0'
GO
RECONFIGURE WITH OVERRIDE
GO
EXEC master.dbo.sp_MSset_oledb_prop
N'SAOLEDB.11',
N'AllowInProcess',
1
GO
sp_addlinkedserver
@server = 'mem',
@srvproduct = 'SQL Anywhere OLE DB Provider',
@provider = 'SAOLEDB.11',
@datasrc = 'mem_PAVILION2'
GO
EXEC master.dbo.sp_serveroption
@server=N'mem',
@optname=N'rpc',
@optvalue=N'true'
GO
EXEC master.dbo.sp_serveroption
@server=N'mem',
@optname=N'rpc out',
@optvalue=N'true'
GO
sp_addlinkedsrvlogin
@rmtsrvname = 'mem',
@useself = 'false',
@locallogin = NULL,
@rmtuser = 'dba',
@rmtpassword = 'sql'
GO
CREATE PROCEDURE copy_mss_t2
@from_row BIGINT,
@to_row BIGINT,
@rows_copied_count BIGINT OUTPUT
AS
SELECT *
INTO #t
FROM ( SELECT *,
ROW_NUMBER()
OVER ( ORDER BY sample_set_number,
connection_number )
AS t2_row_number
FROM mss_t2 ) AS ordered_mss_t2
WHERE ordered_mss_t2.t2_row_number BETWEEN @from_row AND @to_row;
SELECT @rows_copied_count = COUNT(*)
FROM #t;
INSERT INTO mem..dba.sa_t2
SELECT sampling_id,
sample_set_number,
connection_number,
blocker_owner_table_name,
blocker_lock_type,
blocker_owner_name,
blocker_table_name,
blocker_reason,
blocker_row_identifier,
current_engine_version,
page_size,
ApproximateCPUTime,
BlockedOn,
BytesReceived,
BytesSent,
CacheHits,
CacheRead,
"Commit",
DiskRead,
DiskWrite,
FullCompare,
IndAdd,
IndLookup,
Isolation_level,
LastReqTime,
LastStatement,
LockCount,
LockName,
LockTableOID,
LoginTime,
LogWrite,
Name,
NodeAddress,
Prepares,
PrepStmt,
QueryLowMemoryStrategy,
QueryOptimized,
QueryReused,
ReqCountActive,
ReqCountBlockContention,
ReqCountBlockIO,
ReqCountBlockLock,
ReqCountUnscheduled,
ReqStatus,
ReqTimeActive,
ReqTimeBlockContention,
ReqTimeBlockIO,
ReqTimeBlockLock,
ReqTimeUnscheduled,
ReqType,
RequestsReceived,
Rlbk,
RollbackLogPages,
TempFilePages,
TransactionStartTime,
UncommitOp,
Userid,
previous_ApproximateCPUTime,
interval_ApproximateCPUTime,
previous_Commit,
interval_Commit,
previous_Rlbk,
interval_Rlbk
FROM #t;
GO
DECLARE @rows_copied_count BIGINT
EXECUTE copy_mss_t2 1110001, 1120000, @rows_copied_count OUTPUT
SELECT @rows_copied_count
GO
EXECUTE create_linked_server
GO
DECLARE @rows_copied_count BIGINT
EXECUTE copy_mss_t2 1120001, 1130000, @rows_copied_count OUTPUT
SELECT @rows_copied_count
GO
EXECUTE create_linked_server
GO
вот исходная таблица SQL Server, содержащая около 1G данных в 1,9 миллиона строк:
CREATE TABLE mss_t2 (
sampling_id BIGINT NOT NULL,
sample_set_number BIGINT NOT NULL,
connection_number BIGINT NOT NULL,
blocker_owner_table_name VARCHAR ( 257 ) NULL,
blocker_lock_type VARCHAR ( 32 ) NULL,
blocker_owner_name VARCHAR ( 128 ) NULL,
blocker_table_name VARCHAR ( 128 ) NULL,
blocker_reason TEXT NULL,
blocker_row_identifier VARCHAR ( 32 ) NULL,
current_engine_version TEXT NOT NULL,
page_size INTEGER NOT NULL,
ApproximateCPUTime DECIMAL ( 30, 6 ) NULL,
BlockedOn BIGINT NULL,
BytesReceived BIGINT NULL,
BytesSent BIGINT NULL,
CacheHits BIGINT NULL,
CacheRead BIGINT NULL,
"Commit" BIGINT NULL,
DiskRead BIGINT NULL,
DiskWrite BIGINT NULL,
FullCompare BIGINT NULL,
IndAdd BIGINT NULL,
IndLookup BIGINT NULL,
Isolation_level BIGINT NULL,
LastReqTime TEXT NOT NULL DEFAULT '1900-01-01',
LastStatement TEXT NULL,
LockCount BIGINT NULL,
LockName BIGINT NULL,
LockTableOID BIGINT NULL,
LoginTime TEXT NOT NULL DEFAULT '1900-01-01',
LogWrite BIGINT NULL,
Name VARCHAR ( 128 ) NULL,
NodeAddress TEXT NULL,
Prepares BIGINT NULL,
PrepStmt BIGINT NULL,
QueryLowMemoryStrategy BIGINT NULL,
QueryOptimized BIGINT NULL,
QueryReused BIGINT NULL,
ReqCountActive BIGINT NULL,
ReqCountBlockContention BIGINT NULL,
ReqCountBlockIO BIGINT NULL,
ReqCountBlockLock BIGINT NULL,
ReqCountUnscheduled BIGINT NULL,
ReqStatus TEXT NULL,
ReqTimeActive DECIMAL ( 30, 6 ) NULL,
ReqTimeBlockContention DECIMAL ( 30, 6 ) NULL,
ReqTimeBlockIO DECIMAL ( 30, 6 ) NULL,
ReqTimeBlockLock DECIMAL ( 30, 6 ) NULL,
ReqTimeUnscheduled DECIMAL ( 30, 6 ) NULL,
ReqType TEXT NULL,
RequestsReceived BIGINT NULL,
Rlbk BIGINT NULL,
RollbackLogPages BIGINT NULL,
TempFilePages BIGINT NULL,
TransactionStartTime TEXT NOT NULL DEFAULT '1900-01-01',
UncommitOp BIGINT NULL,
Userid VARCHAR ( 128 ) NULL,
previous_ApproximateCPUTime DECIMAL ( 30, 6 ) NOT NULL DEFAULT 0.0,
interval_ApproximateCPUTime AS ( COALESCE ( "ApproximateCPUTime", 0 ) - previous_ApproximateCPUTime ),
previous_Commit BIGINT NOT NULL DEFAULT 0,
interval_Commit AS ( COALESCE ( "Commit", 0 ) - previous_Commit ),
previous_Rlbk BIGINT NOT NULL DEFAULT 0,
interval_Rlbk AS ( COALESCE ( Rlbk, 0 ) - previous_Rlbk ) )
вот целевая таблица в SQL Anywhere 11:
CREATE TABLE sa_t2 (
sampling_id BIGINT NOT NULL,
sample_set_number BIGINT NOT NULL,
connection_number BIGINT NOT NULL,
blocker_owner_table_name VARCHAR ( 257 ) NULL,
blocker_lock_type VARCHAR ( 32 ) NULL,
blocker_owner_name VARCHAR ( 128 ) NULL,
blocker_table_name VARCHAR ( 128 ) NULL,
blocker_reason TEXT NULL,
blocker_row_identifier VARCHAR ( 32 ) NULL,
current_engine_version TEXT NOT NULL,
page_size INTEGER NOT NULL,
ApproximateCPUTime DECIMAL ( 30, 6 ) NULL,
BlockedOn BIGINT NULL,
BytesReceived BIGINT NULL,
BytesSent BIGINT NULL,
CacheHits BIGINT NULL,
CacheRead BIGINT NULL,
"Commit" BIGINT NULL,
DiskRead BIGINT NULL,
DiskWrite BIGINT NULL,
FullCompare BIGINT NULL,
IndAdd BIGINT NULL,
IndLookup BIGINT NULL,
Isolation_level BIGINT NULL,
LastReqTime TEXT NOT NULL DEFAULT '1900-01-01',
LastStatement TEXT NULL,
LockCount BIGINT NULL,
LockName BIGINT NULL,
LockTableOID BIGINT NULL,
LoginTime TEXT NOT NULL DEFAULT '1900-01-01',
LogWrite BIGINT NULL,
Name VARCHAR ( 128 ) NULL,
NodeAddress TEXT NULL,
Prepares BIGINT NULL,
PrepStmt BIGINT NULL,
QueryLowMemoryStrategy BIGINT NULL,
QueryOptimized BIGINT NULL,
QueryReused BIGINT NULL,
ReqCountActive BIGINT NULL,
ReqCountBlockContention BIGINT NULL,
ReqCountBlockIO BIGINT NULL,
ReqCountBlockLock BIGINT NULL,
ReqCountUnscheduled BIGINT NULL,
ReqStatus TEXT NULL,
ReqTimeActive DECIMAL ( 30, 6 ) NULL,
ReqTimeBlockContention DECIMAL ( 30, 6 ) NULL,
ReqTimeBlockIO DECIMAL ( 30, 6 ) NULL,
ReqTimeBlockLock DECIMAL ( 30, 6 ) NULL,
ReqTimeUnscheduled DECIMAL ( 30, 6 ) NULL,
ReqType TEXT NULL,
RequestsReceived BIGINT NULL,
Rlbk BIGINT NULL,
RollbackLogPages BIGINT NULL,
TempFilePages BIGINT NULL,
TransactionStartTime TEXT NOT NULL DEFAULT '1900-01-01',
UncommitOp BIGINT NULL,
Userid VARCHAR ( 128 ) NULL,
previous_ApproximateCPUTime DECIMAL ( 30, 6 ) NOT NULL DEFAULT 0.0,
interval_ApproximateCPUTime DECIMAL ( 30, 6 ) NOT NULL COMPUTE ( COALESCE ( "ApproximateCPUTime", 0 ) - previous_ApproximateCPUTime ),
previous_Commit BIGINT NOT NULL DEFAULT 0,
interval_Commit BIGINT NOT NULL COMPUTE ( COALESCE ( "Commit", 0 ) - previous_Commit ),
previous_Rlbk BIGINT NOT NULL DEFAULT 0,
interval_Rlbk BIGINT NOT NULL COMPUTE ( COALESCE ( Rlbk, 0 ) - previous_Rlbk ),
PRIMARY KEY ( sample_set_number, connection_number ) );
5 ответов
разве вам не нужно очистить таблицу temp #t после каждой итерации? т. е. добавить TRUNCATE TABLE #t в конце процедуры? Я думаю, что temp table #t существует до окончания сеанса, а не до завершения хранимой процедуры. SELECT INTO просто добавляет к существующей #t, но не заменить его.
другое дело было бы использовать постоянную таблицу, а не что-то, хранящееся в tempdb #tables.
проблема заключается в использовании связанного сервера через SQL Anywhere 11.0.1 поставщика SAOLEDB.11 для вставки данных в целевой столбец, объявленный как больше VARCHAR (8000 ). Вот упрощенное воспроизводимое:
-- Overview: SQL Server 2008 suffers from a fatal memory leak
-- if an attempt is made to use a linked server and the
-- SAOLEDB.11 provicer to copy data from SQL Server
-- to a SQL Anywhere 11.0.1 table that contains a single column
-- larger than VARCHAR ( 8000 ); i.e, a VARCHAR ( 8000 ) target
-- column is OK but VARCHAR ( 8001 ) is not. The actual string
-- length is not an issue, nor is the fact that the SQL Server
-- column is declared as TEXT. The memory leak is faster if
-- there is more than one target column larger than VARCHAR ( 8000 ).
-- The server computer freezes and must be rebooted.
-- Msg 701, Level 17, State 123, Server BRECK-PC\SQLEXPRESS, Line 2
-- There is insufficient system memory in resource pool 'internal' to run this
-- query.
-- Location: qxcntxt.cpp:1052
-- Expression: cref == 0
-- SPID: 52
-- Process ID: 2044
---------------------------------------------------------
-- SQL ANYWHERE 11 on target computer
---------------------------------------------------------
-- Target:
-- HP Pavilion laptop, 4GHz Pentium 4, 2G RAM
-- Windows XP SP2
-- SQL Anywhere 11.0.1.2276
---------------------------------------------------------
-- Windows commands used to create and start in-memory database
/*
"%SQLANY11%\bin32\dbinit.exe"^
mem.db
"%SQLANY11%\bin32\dbspawn.exe" -f^
"%SQLANY11%\bin32\dbsrv11.exe"^
-im nw^
-o dbsrv11_log.txt^
mem.db
"%SQLANY11%\bin32\dbisql.com"^
-c "ENG=mem;DBN=mem;UID=dba;PWD=sql;CON=mem-1"
*/
---------------------------------------------------------
-- Create target table with one single "long" column.
BEGIN
DROP TABLE sa_target;
EXCEPTION WHEN OTHERS THEN
END;
CREATE TABLE sa_target (
primary_key INTEGER NOT NULL PRIMARY KEY,
string_column_1 VARCHAR ( 8001 ) NOT NULL );
---------------------------------------------------------
--- SQL SERVER 2008 on source (server) computer
---------------------------------------------------------
-- Server:
-- Desktop with Intel Core 2 Quad Q9450 2.66Ghz 4G RAM
-- Windows Vista Ultimate 64 bit build 6001 SP1
-- SQL Server 2008 Express 64 Service Pack 1 with cumulative update package 4:
-- Microsoft SQL Server 2008 (SP1) - 10.0.2734.0 (X64) Sep 11 2009 14:30:58
-- Copyright (c) 1988-2008 Microsoft Corporation Express Edition with
-- Advanced Services (64-bit) on Windows NT 6.0 <X64>
-- (Build 6001: Service Pack 1)
-- SAOLEDB.11 driver from SQL Anywhere 11.0.1.2276
---------------------------------------------------------
-- Windows command used to start osql.exe
/*
"c:\Program Files\Microsoft SQL Server0\Tools\Binn\osql.exe"^
-d main^
-I^
-l 10^
-P j68Fje9#fyu489^
-S BRECK-PC\SQLEXPRESS^
-U sa
*/
---------------------------------------------------------
-- Create database.
USE master
GO
BEGIN TRY
DROP DATABASE main;
END TRY
BEGIN CATCH
END CATCH;
GO
CREATE DATABASE main
ON PRIMARY
( NAME = main_dat,
FILENAME = 'E:\data\main\main.mdf',
SIZE = 2GB,
FILEGROWTH = 200MB )
LOG ON
( NAME = 'main_log',
FILENAME = 'E:\data\main\main.ldf',
SIZE = 2GB,
FILEGROWTH = 200MB )
GO
----------------------------------------------------------------------------
-- Configure SAOLEDB.11 provider.
USE master
go
-- SAOLEDB.11 provider dlls registered via these Windows commands:
-- regsvr32 dboledb11.dll
-- regsvr32 dboledba11.dll
EXEC master.dbo.sp_MSset_oledb_prop N'SAOLEDB.11', N'AllowInProcess', 1
GO
-- If the following statement produces this message, it probably means
-- that 'DisallowAdHocAccess' is already set to zero for SAOLEDB.11:
--
-- RegDeleteValue() returned error 2, 'The system cannot find the file specified.'
EXEC master.dbo.sp_MSset_oledb_prop N'SAOLEDB.11', N'DisallowAdHocAccess', 0
GO
----------------------------------------------------------------------------
-- THIS SECTION WAS NOT RUN.
-- Set up for OPENROWSET usage.
-- NOT REQUIRED for required for Linked Server usage.
USE master
GO
sp_configure 'show advanced options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
----------------------------------------------------------------------------
-- Set up Linked Server usage.
USE main
GO
BEGIN TRY
EXEC sp_droplinkedsrvlogin
@rmtsrvname = 'mem',
@locallogin = NULL
END TRY
BEGIN CATCH
END CATCH
GO
BEGIN TRY
EXEC sp_dropserver
@server = 'mem'
END TRY
BEGIN CATCH
END CATCH
GO
EXEC sp_addlinkedserver
@server = 'mem',
@srvproduct = 'SQL Anywhere OLE DB Provider',
@provider = 'SAOLEDB.11',
@datasrc = 'mem_PAVILION2'
GO
EXEC master.dbo.sp_serveroption
@server=N'mem',
@optname=N'rpc',
@optvalue=N'true'
GO
EXEC master.dbo.sp_serveroption
@server=N'mem',
@optname=N'rpc out',
@optvalue=N'true'
GO
EXEC sp_addlinkedsrvlogin
@rmtsrvname = 'mem',
@useself = 'false',
@locallogin = NULL,
@rmtuser = 'dba',
@rmtpassword = 'sql'
GO
----------------------------------------------------------------------------
-- Create and fill source table with 1 million rows.
USE main
GO
BEGIN TRY
DROP TABLE mss_source;
END TRY
BEGIN CATCH
END CATCH
GO
CREATE TABLE mss_source (
primary_key INTEGER NOT NULL PRIMARY KEY,
string_column_1 TEXT NOT NULL )
GO
BEGIN
DECLARE @primary_key INTEGER
SELECT @primary_key = 1
BEGIN TRANSACTION
WHILE @primary_key <= 1000000
BEGIN
INSERT mss_source VALUES (
@primary_key,
REPLICATE ( 'Some test data. ', 2 ) )
SELECT @primary_key = @primary_key + 1
END
COMMIT
END
GO
SELECT COUNT(*) FROM mss_source
GO
-- 1000000
---------------------------------------------------------
-- Copy data to target table.
SELECT CURRENT_TIMESTAMP
GO
INSERT INTO mem..dba.sa_target
SELECT *
FROM mss_source
GO
SELECT CURRENT_TIMESTAMP
GO
---------------------------------------------------------
-- Test 1 - Code as shown above.
-- FAILED
--
-- Started at 2009-10-12 10:06:33.393
--
-- A slow server memory leak began immediately.
-- The initial target insert rate was about 2000 rows per second.
-- Server RAM usage reached 3.82 GB, Physical Memory: 95%, Page File 16236M / 16288M
-- The server display became frozen.
-- The server became unresponsive to mouse input.
-- The target insert rate dropped below 1000 rows per second, but inserts continued.
-- The copy process reached 937,817 rows inserted on the target.
-- This dialog box appeared on the server: "Your computer is low on memory"
-- Eventually, the process failed, and this message appeared in the osql.exe window:
--
-- Msg 701, Level 17, State 123, Server BRECK-PC\SQLEXPRESS, Line 2
-- There is insufficient system memory in resource pool 'internal' to run this
-- query.
-- Location: qxcntxt.cpp:1052
-- Expression: cref == 0
-- SPID: 52
-- Process ID: 2044
--
-- Failed at 2009-10-12 10:22:21.817
-- The server disk I/O light remained lit without interruption.
-- The server required a hard reboot.
---------------------------------------------------------
-- Test 2 - Code as shown above, except for VARCHAR ( 8000 ).
-- SUCCESSFUL
BEGIN
DROP TABLE sa_target;
EXCEPTION WHEN OTHERS THEN
END;
CREATE TABLE sa_target (
primary_key INTEGER NOT NULL PRIMARY KEY,
string_column_1 VARCHAR ( 8000 ) NOT NULL );
-- Started at 2009-10-12 10:41:46.427
-- There was some slight initial growth in RAM usage on the server.
-- Server RAM usage stabilized at 1.40 GB, Physical Memory: 35%, Page File 1560M / 8352M
-- The target insert rate remained about 2000 rows per second throughout.
-- Finished OK at 2009-10-12 10:50:52.240
---------------------------------------------------------
-- Test 3 - Repeat Test 2
-- SUCCESSFUL
-- Started at 2009-10-12 10:53:38.350
-- No further RAM usage growth on the server.
-- Finished OK at 2009-10-12 11:02:10.457
вместо использования временных таблиц вы можете попробовать использовать таблицы переменных?
например.
DECLARE @ResultTable TABLE (TableId INT PRIMARY KEY, ... etc)
INSERT INTO @ResultTable
SELECT TableId, ....
FROM mss_t2
... etc. ...
У меня была аналогичная проблема, мой код содержит использование простой таблицы #temp в цикле, что вызвало это, и Я заменил постоянной таблицей.
кажется, работает.
спасибо Навин!--1-->