Here’s a new Knowledge Base article we published. This one talks about an issue where MSDPM.EXE crashes during Garbage Collection causing DPMDB growth in DPM 2007.
=====
Symptoms
Every day at midnight, System Center Data Protection Manager 2007 (DPM) starts a couple of maintenance tasks which are referred to as Garbage Collection. During Garbage Collection, a couple things happen such as:
- Expired Recovery Points are removed
- Entries from the DPMDB database that are older than 33 days are removed
A problem that can occur is that during Garbage Collection, the DPM service (MSDPM.EXE) crashes and the Garbage Collection job never completes. As result, stale data within DPMDB is never completely removed and you end up with a large and ever growing DPMDB database.
Cause
This can occur if SQL encounters a deadlock and in order to resolve it, one of the store procedures involved in the deadlock is killed by the SQL Engine.
You can find something similar to the following in MSDPMCurr.errlog if this is occurring:
NOTE If DPM was installed in its default location, this file will be in C:\Program Files\Microsoft DPM\DPM
Attempt 1 failed with exception Microsoft.Internal.EnterpriseStorage.Dls.DB.NonFatalDbException: exception ---> System.Data.SqlClient.SqlException: Transaction (Process ID 63) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.Internal.EnterpriseStorage.Dls.DB.SqlRetryCommand.ExecuteNonQuery()
--- End of inner exception stack trace ---
*** Mojito error was: DatabaseNonFatalError; 0; None
--- SqlException details -----------------
System.Data.SqlClient.SqlException: Transaction (Process ID 63) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.Internal.EnterpriseStorage.Dls.DB.SqlRetryCommand.ExecuteNonQuery()
Error = 1205
Index #0
Source: .Net SqlClient Data Provider
Number: 1205
State: 52
Class: 13
Server: <Server_Name>
Message: Transaction (Process ID 63) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Procedure: prc_PRM_GarbageCollect
Line: 714
--- Original Command -----------------
dbo.prc_DLS_GarbageCollect
--- Caller StackTrace -----------------
FileName:; Method:ExecuteNonQuery(); lineNo:0; ilOffset:8.
FileName:; Method:CleanupDlsDatabase(); lineNo:0; ilOffset:119.
FileName:; Method:OnStart(); lineNo:0; ilOffset:260.
FileName:; Method:Start(); lineNo:0; ilOffset:2.
FileName:; Method:Execute(); lineNo:0; ilOffset:8.
FileName:; Method:ChangeState(); lineNo:0; ilOffset:62.
FileName:; Method:Process(); lineNo:0; ilOffset:338.
FileName:; Method:Function(); lineNo:0; ilOffset:16.
FileName:; Method:Run(); lineNo:0; ilOffset:95.
FileName:; Method:PerformWaitCallbackInternal(); lineNo:0; ilOffset:28.
FileName:; Method:PerformWaitCallback(); lineNo:0; ilOffset:40.
Resolution
This is a known issue in System Center Data Protection Manager 2007. To have this issue addressed, please contact Microsoft Support (http://support.microsoft.com/contactus/).
More Information
This issue is addressed in System Center Data Protection Manager 2010 and later.
=====
For the most current version of this article please see the following:
J.C. Hornbeck | Knowledge Engineer | Management and Security Division
Get the latest System Center news on Facebook and Twitter:
App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/
The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/