Support Tip: Scheduled backup to tape runs on a wrong date on DPM 2007, 2010 and 2012

Hi there DPM administrators, Wilson Souza here from DPM Support team. As you know, Data Protection Manager offers many ways to protect server workloads  such as Exchange, SQL, SharePoint, Hyper-V, System State, Bare metal, Files, Shares etc. Protection can be done in the following ways:

Disk to Disk (D-D) – When the protected data goes from a disk on the protected server to a volume that sits on your DPM Server

Disk to Tape (D-T)– When the protected data goes from a disk on the protected server to a tape device managed by the DPM server.

Disk to Disk to Tape (D-D-T) – The combination  of the two options above where protected data goes from a disk on the protected server to volume that sits on the DPM server and then we copy that data from the DPM volume to a tape device.

 

When using tapes, we have a range of options available as to when we want the backup to go to tape. This can be daily, weekly, monthly, quarterly, yearly or any other combination that you see fit to your SLA.

 

DPM delegates the schedule backup control to SQL agent and when it is time for the tape backup to run, SQL agent triggers a DPM engine job to start the backup.

 

So let’s say that one day you get into the office in the morning to check how backups are being created and notice that a backup that was supposed to run 2 months from now just got completed last night, and you didn’t get a warning stating that DPM would run that backup on an unexpected date. The purposes of this blog is to explain the issue and provide a workaround.

 

NOTE: This issue does not affect daily, weekly, or monthly tape backup schedules, it primarily affects tape backups that are in multiples of months, like quarterly, semi-annually, yearly etc. This issue is presented on DPM 2007, 2010 and 2012 but this workaround is not applicable to DPM 2007.

 

 

EXPLAINING THE ISSUE

 

 

 

Assume that today is 10/07/2011 and we created a new protection group and set long term protection (weekly, quarterly and yearly). At the end of the new protection group wizard, DPM will create the necessary scheduled jobs and send them over to SQL Agent.

As illustrated, this is the quarterly backup as seem from SQL Agent.

 

Note: Quarterly backups should run only on Jan/Apr/Jul/Oct

 

 

 

SQL Agent shows that this job should run two days after the job was created

 

 

 

On the 10/09/11, the backup to tape ran as expected. For the quarterly schedule definition we are now expecting this backup to run on 01/09/2012.

 

Almost every action done on a Protection Group (manually: add/remove protected members, modify disk allocation or simply completing a modify protection group wizard without making any change or automatically: SQL and SharePoint auto protection, Disk auto grow) will cause all scheduled jobs from that group to be deleted and recreated. This is where scheduled jobs have a potential of running on a wrong date.

 

By deleting/creating new  schedules, DPM will use the original XML to generate the new scheduled job. The ScheduleXml will use the original Start Date which could now be in the past. Below is the snipped of ScheduleXML.

 

<?xml version="1.0" encoding="utf-16"?>

<Schedule xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" ScheduleID="9b0c036b-5c2d-49b8-a374-3842ba6cfb96" JobDefID="c5241cb4-8dc2-4574-b758-2e7b7db0ca70" xmlns="http://schemas.microsoft.com/2003/dls/Scheduler.xsd">

  <Recurrence>

    <Monthly StartDt="2011-10-09" EndDt="9999-12-31" Interval="3" MonthDayList="9" />

    <Time StartTm="20:00:00" EndTm="20:00:00" />

  </Recurrence>

</Schedule>

 

 

 

Now we fast forward to November the 6^th and the protection group was modified to add a new data source. The original scheduled job above will be removed and a new one created in its place.

Note that the start date below is unchanged.

 

 

 

Highlighted are the new schedules created by the modify protection group operation (one for weekly, quarterly and yearly). The quarterly new scheduled job is the second from the last line (Next Run = 11/09/2011 8:00:00 PM)

 

 

 

SQL agent sees that this schedule  was set to run for the first time on 10/09/2011.  Now it is almost a month later and Last Run column shows that this job never ran. To resolve that, SQL Agent will set the Next run time for this new job for the first available date. As we are on the 6^th, and the 9^th is three days from now, SQL Agent will schedule this job to run on the 9^th.

 

So instead of the quarterly backup being run in January as expected, it will now run 2 months earlier. In addition, this job won’t show up as scheduled in the DPM UI (there is an explanation for that but we will cover it on another blog). You will only see a reference to this job when it is running, completed or failed.

 

WORKAROUND

 

To work around this issue, copy the script below to SQL Server Management Studio and execute it. This new stored procedure will check if the start date is in the past, and if it is it will calculate the next run time and set it accordingly. From the SQL Agent standpoint, the wrong schedule can only happen when the scheduled start date is set to be in the past which will always be the case once the original schedule day is past.

Script:

=====

 

USE [DPMDB]

GO

/****** Object:  StoredProcedure [dbo].[prc_IM_UserSchedule_Update]    Script Date: 11/10/2011 19:11:01 ******

******* Edited by........: Wilson Souza     

******* Version..........: 2.2

******* Date Created.....: 11/10/11

******* Date Last Change.: 04/04/11

*******                                     THIS SCRIPT IS FOR DPM 2010/2012 RTM   

******* Change Log for V 2.2

*******           Addressed issue if selected DAY instead any day of the week.

******* Change log for V 2.0

*******           Now using XML variable to retrieve data instead of searching string on schedule variable

*******           Now addresses issues for Weekly schedules. Not only Months

*******           Now addresses issues when user select First, Second, Third, Four or Last day of the month.

*/

 

SET ANSI_NULLS ON

GO

SET QUOTED_IDENTIFIER ON

GO

 

--

-- Update one row in UserSchedule table by ScheduleID.

-- If this ScheduleID doesn't exist,

--  add a new row with this ScheduleID.

--

ALTER PROCEDURE [dbo].[prc_IM_UserSchedule_Update]

(

    @ScheduleID GUID,

    @ProtectedGroupID GUID,

    @JobType tinyint,

    --------------- Change Start ---------------

    -- @Schedule ntext,

    @Schedule nvarchar(max),

    --------------- Change end ---------------

    @Immediacy bit,

    @TimeOffset int,

    @MaxDuration bigint,

    @ScheduleListId GUID

)

AS

 

DECLARE @error int,

        @rowcount int,

       

       

    --------------- Change Start ---------------

     

                  @xml                                xml,

                  @CurrentDate                        date,

                  @ForLOfTheMonth                     date, -- First or Last Day of the Month

                  @count                              int,

                  @count1                             int,

 

                  @Monthly_StartDt                    date,

                  @Monthly_Interval                   int,

                  @Monthly_MonthDayList               int, -- This might not be needed

 

                  @MonthlyRelative_StartDt            date,

                  @MonthlyRelative_Interval           int,

                  @MonthlyRelative_RelativeWeekDay    nvarchar(3),

                  @MonthlyRelative_RelativeInterval   nvarchar(6),

                 

                  @Weekly_StartDt                     date,

                  @Weekly_Interval                    int,

                  @Weekly_WeekDayList                 nvarchar(20) -- This might not be needed

 

 

set @xml = CONVERT(xml,SUBSTRING(@schedule,42,LEN(@schedule)-41))

set @CurrentDate = GETDATE()

 

select @Weekly_StartDt        = @xml.value ('(//*[local-name()="Weekly"]/@StartDt)[1]', 'date')

select @Weekly_Interval       = @xml.value ('(//*[local-name()="Weekly"]/@Interval)[1]', 'int')

select @Weekly_WeekDayList    = @xml.value ('(//*[local-name()="Weekly"]/@WeekDayList)[1]', 'nvarchar(20)') -- This might not be needed

 

select @Monthly_StartDt       = @xml.value ('(//*[local-name()="Monthly"]/@StartDt)[1]', 'date')

select @Monthly_Interval      = @xml.value ('(//*[local-name()="Monthly"]/@Interval)[1]', 'int')

select @Monthly_MonthDayList  = @xml.value ('(//*[local-name()="Monthly"]/@MonthDayList)[1]', 'int') -- This might not be needed

 

select @MonthlyRelative_StartDt           = @xml.value ('(//*[local-name()="MonthlyRelative"]/@StartDt)[1]', 'date')

select @MonthlyRelative_Interval          = @xml.value ('(//*[local-name()="MonthlyRelative"]/@Interval)[1]', 'int')

select @MonthlyRelative_RelativeWeekDay   = @xml.value ('(//*[local-name()="MonthlyRelative"]/@RelativeWeekDay)[1]', 'nvarchar(3)')

select @MonthlyRelative_RelativeInterval  = @xml.value ('(//*[local-name()="MonthlyRelative"]/@RelativeInterval)[1]', 'nvarchar(6)')

 

If @Monthly_StartDt is NOT NULL

      while @Monthly_StartDt < @Currentdate

            Set @Monthly_StartDt = DATEADD(MONTH,@Monthly_Interval,@Monthly_StartDt)

           

if @Weekly_StartDt is NOT NULL

      if @Weekly_Interval > 1

            while @Weekly_StartDt < @CurrentDate

                  set @Weekly_StartDt = DATEADD(DAY,@Weekly_Interval * 7,@Weekly_StartDt)      

     

If @MonthlyRelative_StartDt is NOT NULL

Begin

      set @ForLOfTheMonth = DATEADD(dd,-(DAY(DATEADD(mm,1,@Currentdate))-1)-(DAY(@Currentdate)-DAY(DATEADD(mm,1,@Currentdate))),@Currentdate)

      if @MonthlyRelative_RelativeInterval = 'Last'

      begin

            set @ForLOfTheMonth = DATEADD(Month,1,@ForLOfTheMonth)

            set @ForLOfTheMonth = DATEADD(dd,-(DAY(DATEADD(mm,1,@ForLOfTheMonth))-1)-(DAY(@ForLOfTheMonth)-DAY(DATEADD(mm,1,@ForLOfTheMonth))),@ForLOfTheMonth)

            set @ForLOfTheMonth = DATEADD(day,-1,@ForLOfTheMonth)

      end

      while @MonthlyRelative_StartDt < @CurrentDate

      begin

            while @MonthlyRelative_StartDt < @ForLOfTheMonth

                  Set @MonthlyRelative_StartDt = DATEADD(MONTH,@MonthlyRelative_Interval,@MonthlyRelative_StartDt)

            if @MonthlyRelative_RelativeInterval = 'Last' 

            Begin

                  set @MonthlyRelative_StartDt = DATEADD(Month,1,@ForLOfTheMonth)

                  set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

                  set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)      

            End

            else

                  set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

            if @MonthlyRelative_RelativeInterval = 'First' or @MonthlyRelative_RelativeInterval = 'Last'

                  set @count = 1

            if @MonthlyRelative_RelativeInterval = 'Second'

                  set @count = 2

            if @MonthlyRelative_RelativeInterval = 'Third'

                  set @count = 3

            if @MonthlyRelative_RelativeInterval = 'Fourth'

                  set @count = 4

            set @count1 = @count

            if @MonthlyRelative_RelativeWeekDay = 'Day'

                  if @count <> 1

                  Begin

                        set @MonthlyRelative_StartDt = DATEADD(dd,@count-1,@MonthlyRelative_StartDt)

                        set @count = 0

                  End

                  Else

                        set @count = 0         

            while @count <> 0

            begin

                  if substring(DATENAME(dw,@MonthlyRelative_StartDt),1,2) = @MonthlyRelative_RelativeWeekDay

                        set @count = @count - 1

                  if @count <> 0

                        if @MonthlyRelative_RelativeInterval <> 'Last'

                              set @MonthlyRelative_StartDt = DATEADD(day,1,@MonthlyRelative_StartDt)

                        else

                              set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)

            end

            if @MonthlyRelative_StartDt < @CurrentDate

            begin

                  set @MonthlyRelative_StartDt = DATEADD(MONTH,@MonthlyRelative_Interval,@MonthlyRelative_StartDt)

                  set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

                  if @MonthlyRelative_RelativeInterval = 'Last'

                  begin

                        set @MonthlyRelative_StartDt = DATEADD(Month,1,@MonthlyRelative_StartDt)

                        set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

                        set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)

                  end

                  set @count = @count1

                  if @MonthlyRelative_RelativeWeekDay = 'Day'

                        if @count <> 1

                        Begin

                              set @MonthlyRelative_StartDt = DATEADD(dd,@count-1,@MonthlyRelative_StartDt)

                              set @count = 0

                        End

                        Else

                              set @count = 0         

                  while @count <> 0

                  begin

                        if substring(DATENAME(dw,@MonthlyRelative_StartDt),1,2) = @MonthlyRelative_RelativeWeekDay

                              set @count = @count - 1

                        if @count <> 0

                              if @MonthlyRelative_RelativeInterval <> 'Last'

                                    set @MonthlyRelative_StartDt = DATEADD(day,1,@MonthlyRelative_StartDt)

                              else

                                    set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)

                  end  

            end  

      end

End

 

if @Monthly_StartDt is NOT NULL

      set @xml.modify ('replace value of (//*[local-name()="Monthly"]/@StartDt)[1] with sql:variable("@Monthly_StartDt")')

If @MonthlyRelative_StartDt is NOT NULL

      set @xml.modify ('replace value of (//*[local-name()="MonthlyRelative"]/@StartDt)[1] with sql:variable("@MonthlyRelative_StartDt")')

if @Weekly_StartDt is NOT NULL

      set @xml.modify ('replace value of (//*[local-name()="Weekly"]/@StartDt)[1] with sql:variable("@Weekly_StartDt")')

 

set @Schedule = '<?xml version="1.0" encoding="utf-16"?>  ' + CONVERT(nvarchar(max),@xml)

 

 

      --------------- Change end ---------------

 

SET @rowcount = 0

SET @error = 0

 

SET NOCOUNT ON

 

UPDATE  dbo.tbl_IM_UserSchedule

SET     ProtectedGroupID   = @ProtectedGroupID,

        JobType            = @JobType,

        Schedule           = @Schedule,

        Immediacy          = @Immediacy,

        TimeOffset         = @TimeOffset,

        MaxDuration        = @MaxDuration,

        ScheduleListId     = @ScheduleListId

WHERE ScheduleID = @ScheduleID

 

SELECT @error = @@ERROR, @rowcount = @@ROWCOUNT

 

IF (@error = 0 AND @rowcount = 0)

BEGIN

    INSERT INTO dbo.tbl_IM_UserSchedule

    (

        ScheduleID,

        ProtectedGroupID,

        JobType,

        Schedule,

        Immediacy,

        TimeOffset,

        MaxDuration,

        ScheduleListId

    )

    values

    (

        @ScheduleID,

        @ProtectedGroupID,

        @JobType,

        @Schedule,

        @Immediacy,

        @TimeOffset,

        @MaxDuration,

        @ScheduleListId

    )

 

    SET @error = @@ERROR

END

 

SET NOCOUNT OFF

 

RETURN @error

=====

Wilson Souza | Senior Support Escalation Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/