Quantcast
Channel: The Official System Center Data Protection Manager Team Blog
Viewing all 293 articles
Browse latest View live

New Style

$
0
0

Whoa!  What’s with the new style?!  The System Center engineering team blogs are now part of the Server & Tools blog network which is a select few top tier blogs from the Server and Tools division.  This network is part of a larger network that represent the best blogs on some of the most important topics across both the TechNet and MSDN blogs. 

Random factoid: Did you know that taken together, the System Center Engineering Team blogs are #5 on all of TechNet by page views only trailing such venerable blogs as the Exchange blog and the Hey Scripting Guys blog.

We are excited to be a part of this blog network.  This will provide some design consistency across our blogs in the network and make it easier to discover and navigate to other great blogs in the Server & Tools network.

You can find all our System Center blogs under the System Center category:

 

image

Note: the System Center Configuration Manager engineering team blog is also going to be included soon.

I also need to go back through and put all of our sidebar and social media content back.  I’ll do that sometime next week.

The design is probably going to evolve a bit so if you have some feedback please let us know!

Hint:  This is the root page for the blog network.  You can see all the categories and drill into them from here:

http://blogs.technet.com/b/serverandtools/


Announcing Availability of System Center 2012 SP1 Community Technology Preview 2 (CTP2)

$
0
0

As I write this post flying back from TechEd, North America (check out Day 1 keynote), it never ceases to amaze me how technology has really changed our lives!!! Speaking of which, what an honor that all of you voted System Center 2012 as best Microsoft Product at TechEd. Thank you for your support! We here have worked incredibly hard over the past few years to develop System Center to meet your needs and having this award means a lot to us. Below is a picture of the award; not the best picture I admit. It’s going to find a home in building 44 at the Microsoft Campus.

clip_image002

And now for what we all have been waiting for... YES, it is here!! We are super excited to announce the availability of CTP2 of System Center 2012 SP1. With the recent release of Windows Server 2012, Release Candidate, this release of System Center is targeted at supporting that. While at TechEd, the buzz about Windows Server 2012 was evident and we are thrilled to be able to release CTP2 which supports this incredible release of Windows Server. Given the plethora of capabilities in Windows Server 2012, this release of System Center has significant investments to provide you a management solution based on a rich platform.

While at TechEd, we had a session which talks about what’s coming in System Center 2012 SP1. Given the breadth of capabilities in SP1, I was able to cover only a few key things, but it’s a good place to start your journey with CTP2.  The recording was not up as I write this, but should be available by early next week. In addition there were deeper dive sessions on each of the System Center components which you should check out at the TechEd site.

This release contains updates to all the System Center components. See below for more information and stay tuned for additional posts describing the component updates.

Upgrade: CTP1 cannot be upgraded to CTP2 and CTP2 will not be upgradable to Beta.

Production Use: This release is not intended for production deployments.  It’s targeted at giving you an early preview of some of what’s coming in this SP1 release. 

Scenarios: We specifically focused on a key set of scenarios documented here for this release.

FEEDBACK, FEEDBACK, FEEDBACK – Please give us your feedback on use of Windows Server 2012, Release Candidate and CTP2 of System Center 2012 SP1 by visiting http://connect.Microsoft.com/SC

What’s New

All components now support Windows Server 2012 RC and SQL Server 2012.

Component

What’s New

More Info

App Controller

· Ability to migrate a VM from VMM to Azure

· Support for using Service Provider Foundation to create and operate VMs in VMM

· Azure IaaS enhancements: Ability to deploy VMs from an image or disk, start and stop VMs, and add VMs to a service

More Info
Configuration Manager

· Support for Windows 8, including deploying Windows 8 applications and the ability to detect 3G and 4G network connections to prevent delivering software at a time when data charges may apply.

· Additional operating support to extend manageability to Mac OS X and Unix/Linux servers.

 

Data Protection Manager

· Improved backup performance of Windows Server 2012 Hyper-V over CSV 2.0 deployments

· Protect Hyper-V over remote SMB share

· Protect Windows 8 de-duplicated volumes

· VM Live Migration: Uninterrupted data protection

More Info

Operations Manager

APM enhancements, including:

· Support for IIS8

· Monitoring of WCF, MVC and .NET NT services

· Azure SDK support

More Info

Orchestrator & Service Provider Foundation

· Supports existing System Center and 3rd-Party Integration Packs

· Service Provider Foundation, which provides a rich set of web services that manage VMM:

o Create, change, and operate VMs

o Manage VMM Self-service User Roles

o Manage multiple VMM stamps and aggregate results from multiple stamps

o Integration with App Controller to use hosted capacity

More Info

Service Manager

· Ability to apply price sheets to VMM Clouds

· Create VM chargeback reports

· Ability to pivot by Cost Center, VMM Clouds, and Price sheets

More Info

Server App-V

· Support for applications that create scheduled tasks during the packaging process

· Ability to create virtual application packages from applications installed natively on a remote server

 

Virtual Machine Manager

· Improved support for network virtualization

· Ability to convert VHD to VHDX and to use VHDX as a base operating system image

· Support for the Windows Standards-Based Storage Management Service, thin provisioning of logical units, and discovery of SAS storage

· You can now create Add-Ins that extend the VMM console.

More Info

There is a lot that still has to come after CPT2!!  Enjoy!!!

Vijay Tewari

Group Program Manager, System Center

 

Download the CTP2 release here:

Download

Location

Installation packages for all components

http://go.microsoft.com/fwlink/?LinkId=254659

Documentation for all components

http://go.microsoft.com/fwlink/?LinkId=254803

Announcing the Availability of System Center 2012 CTP2 – Data Protection Manager

$
0
0

As Vijay announced, the System Center 2012 SP1 CTP2 is now available for download! We’d like to provide some additional information about what’s shipping in CTP2 for the DPM component:

Efficient data protection of VMs deployed on Hyper-V over CSV:

Windows 2012 CSV 2.0 enhanced its CSV capabilities that will make the backup more efficient.  DPM has enabled Expressfull backup feature for Hyper-V CSV.  This takes out the big pain point of customer going through CC mode for all backups.  DPM protection performance has improved by 90% compared to CSV 1.0 deployments on Win2K8R2.  The performance is further improved by allowing parallel backups, taking owner-nonowner  node dependencies.  All these features are possible without customer deploying expensive hardware providers.

Efficient data protection of VMs deployed on Hyper-V over SMB:

Windows 2012 Hyper-V can now store its data not only on local storage or CSV but also can store its data on a remote SMB file share.  Using this, customers can benefit of live migration of VMs from standalone/cluster to another standalone/cluster without storage migration using Windows 2012 platform.  Customers also get great benefit of storage consolidation and cheaper cluster solutions.  DPM will now be able to protect VMs deployed on this configuration.  DPM continue to do the efficient backup of VM even after live migration when the source and target Hyper-V platforms are using same remote SMB file share which can be deployed on standalone file server or scale out SMB cluster.  DPM can do protection seamlessly in both scenarios.

VM Live Migration – Uninterrupted & Efficient VM backup:

Windows 2012 allows customers to do live migration inter cluster, intra cluster, cluster to standalone, standalone to cluster with or without storage migration.  DPM,  tightly integrated with VMM, is now able to detect and continue backup even if the VM is live migrated.  DPM now has intelligence to detect that VM has gone to other machine and is able to protect from there.  This technology combined with Hyper-V over Remote SMB share can now help customers continue to have efficient backups even when the VM live migration happens across hosts that share remote SMB.  Another great feature that provides flexibility and power to customer.

Efficient Dedupe protection:

Windows 2012 has introduced Dedupe functionality that will help customers reduce their storage consumption.  This is great for customers who have huge file servers that are mainly archival  that do not have much churning.  Thanks to DPM’s new support, DPM is now able to protect the “Windows 2012” Deduped file systems efficiently.  DPM is not only intelligent to detect that the file system is Deudpe enabled but also transfer the data on the wire efficiently and store it efficiently.  All of this is achieved without rerunning the Dedupe logic on the DPM server side.

Reminders:

· CTP1 cannot be upgraded to CTP2 and CTP2 will not be upgradable to Beta.

· CTP2 is not supported for production use.

· Procedures not covered in the documentation might not work.

Download: 

Location

Description

http://go.microsoft.com/fwlink/?LinkId=254659

The entire installation of System Center 2012 Service Pack 1 CTP2.

http://go.microsoft.com/fwlink/?LinkId=254803

Documentation for all the components.

-Neela Syam Kolli

System Center 2012 Self-Study Guide ( Data Protection Manager and Endpoint Protection)

$
0
0

NewDocsIntoHeadMicrosoft’s very own Scott Rachui put together another great self-study guide for System Center 2012, this time focusing exclusively on Data Protection Manager and Endpoint Protection. If you’re wanting a head start in mastering these technologies then these are a great place to start.

=====

In part 4 of this series, I turn to Data Protection Manager and I revisit Endpoint Protection. For those of you who might have seen earlier one of my earlier study guides, you will see that I have already put out a study guide for Forefront Endpoint Protection. In this post, I will focus exclusively on Endpoint Protection as it relates to System Center 2012.

As I did in Part 2 of this series, I want to start with a brief explanation of these two components of System Center 2012. I do this with the lesser-known components of the System Center suite for those who may not know what these tools do. Hopefully these brief explanations are helpful in setting the stage.

Data Protection Manager -"enables disk-based and tape-based data protection and recovery for servers such as SQL Server, Exchange Server, SharePoint, virtual servers, file servers, and support for Windows desktops and laptops. DPM can also centrally manage system state and Bare Metal Recovery (BMR)."

Endpoint Protection - "Microsoft System Center 2012 Endpoint Protection (previously known as Forefront Endpoint Protection 2010) allows you to consolidate desktop security and management in a single solution. "

You can continue reading Scott’s article here.

J.C. Hornbeck | System Center & Security Knowledge Engineer

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

DPM Certificate Troubleshooting–Part 1: General Troubleshooting

$
0
0

toolsignThe spirit of this document is to provide you with a quick guide to troubleshooting System Center 2012 Data Protection Manager (DPM) Certificate authentication issues. This document assumes that you are already familiar with DPM 2012 and have a healthy certificate infrastructure. This document also assumes that you have setup certificates in accordance with the following blog post:

How to use certificates to authenticate computers in workgroups or untrusted domains with Data Protection Manager:
http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

We will cover briefly at a high level some of the common caveats that you will come across when using certificates with DPM protection and how to assess what may be the root cause. I have included screenshots of some of the DPM GUI errors, event log errors and some log snippets. Admittedly the log reading is not very intuitive so I just highlighted the relevant portions.

Services

1. Make sure the DPMRA service can be started.

2. Make sure the DPM CPWrapper Service can be started. I can’t stress this enough. Upon my testing I performed various actions to simulate a failure.

a.) Removed the DPM cert
b.) Removed the client cert
c.)Removed the DPM reg key on the DPM server
d.) Removed the Member Server regkey on the DPM server
e.) Removed the DPM regkey on the Member server
f.) Removed the Member Server regkey on the Member Server.

After each failure I would either place the cert or regkey back and almost each time I would have to restart the DPM CPWrapper Service. In light of this you should make it a very common practice to restart the DPM CPWrapper Service during your troubleshooting.

3. Make sure the Cryptographic Services are started


Ports

1.) Remember that DPM certificate use relies on port 6076 for Certificate protection. You may have to adjust any intermediate firewall settings to allow for this port to be opened for certificate based protection. You can use the netstat command to verify if port 6076 is listening for communication from both the ends.

Type in: netstat –ano
or
netstat –ano |findstr 6076

a = Displays all connections and listening ports

n = Displays addresses and port numbers in numerical form.

o = Displays the owning process ID associated with each connection.

|findstr 6076 will show only the associations for that port.

Using the following: netstat –ano |findstr 6076

We see that the port 6076 is listening:

image

You could also use TCPView found at http://technet.microsoft.com/en-us/sysinternals/bb897437.aspx for a GUI interface to show what services are listening to which ports.

Example:

image

The use of certificates for authentication DOES NOT remove the need for other ports for types of domain communication such as name resolution, Kerberos and LDAP. These ports will still be needed for proper DNS or NetBIOS name resolution and AD authentication.

http://technet.microsoft.com/en-us/library/ff399341.aspx

Firewall

If on the target server you have the firewall turned off and you go through the setdpmserver command, the command will NOT create the necessary firewall rule for TCP port 6076. If you leave the firewall turned off , then there will be no issue. If at a later date you turn the firewall back on, your communication will fail because this rule is NOT created. To correct this you can do one of three things:

a.) Leave the integrated firewall off
b.) Manually create the rule yourself
c.) Re-run the setdpmserver command.

Verification that the ports have been added.

image

Firewall Rule Created

image

This firewall rule created specifies a local port of 6076 and a remote port of “all ports”.

image

4.) If testing with DPM beta If the firewall is turned off on the client you will get the following error:

image

You will have to have the firewall turned on. Again, this has been fixed in RTM for DPM 2012.

Certificate

By default, with web enrollment the certificate is saved in the Current User store but needs to exported with the private key and imported into the Local computer store. Again, this is if you are using web enrollment. If certs are configured for “enroll” then it can be specified to be placed in the local computer store.

The thumbprint in the command syntax (AttachProductionServerWithCertificate and the Set-DPMCredentials commands) needs to NOT have spaces when specifying it in the commands.

An example would be as follows.
Here is the DPM server cert:

image

Notice the spaces in the thumbprint.

When we use the certificate thumbprint to generate the bin file we need to remove the spaces.

image

Notice the Set-DPMCredentials command syntax used in this case.

Set-DPMCredentials –DPMServerName DPM2012.contoso.com –Type Certificate –Action Configure –OutputFilePath C:\Temp -Thumbprint 493f27f35b2105804afbd49bb5a59bf2e380e00

This is the thumbprint for the DPM server certificate without the spaces.

The certificate must specify certain parameters:

X.509 V3 certificates
Enhance Key Usage should have client authentication and server authentication.
Key length should be at least 1024 bits.
Key type should be exchange.
Certificate can NOT be self signed.
Subject name of the certificate and root certificate should not be empty.
Certificates shouldn’t be of Cryptography API Next Generation (CNG) Keys. DPM doesn’t support
certificates with CNG Keys.
The revocation servers of the associated Certificate Authorities are online and accessible by both the
protected server and DPM server.
The certificate has an associated private key

You can use the following command to verify the certificate parameters of the certs in use on a server.
certutil –store –v my

C:\>certutil -store -v my

================ Certificate 1 ================

X509 Certificate: <<<<<Denotes x.509>>>>>

Version: 3   <<<<<Denotes V3>>>>>

Serial Number: 5da52bdc000226d4c235

Signature Algorithm:

    Algorithm ObjectId: 1.2.840.113549.1.1.5 sha1RSA

    Algorithm Parameters:

    05 00

Issuer:

    CN=Corp NAP CA 1

 

 NotBefore: 9/14/2011 7:31 AM

 NotAfter: 9/17/2011 7:31 AM

 

……….

 

Public Key Length: 2048 bits   <<<<<Denotes length>>>>>

Public Key: UnusedBits = 0

    0000  30 82 01 0a 02 82 01 01  00 97 3c 11 94 27 58 47

    0010  4a 51 55 60 a5 b6 32 8a  4e 4b 59 1d 56 1f ac 53

……….

 

    Application Policies

        [1]Application Certificate Policy:

             Policy Identifier=Server Authentication <<<<<Denotes server>>>>>

        [2]Application Certificate Policy:

             Policy Identifier=Client Authentication  <<<<<Denotes client>>>>>

 

……….

  CERT_KEY_PROV_INFO_PROP_ID(2):

    Key Container = {57CE5453-2951-4AE2-A036-E685FC52AB83}

  Unique container name: bed058e40c5ed733d5da8a6655583c3d_d5520479-582f-4563-8c84-e153a68e8fe2

    Provider = Microsoft Enhanced Cryptographic Provider v1.0  <<<<<Denotes provider - must be cryptographic provider and NOT Key Storage Provider >>>>>

    ProviderType = 1

    Flags = 60

    KeySpec = 1 -- AT_KEYEXCHANGE <<<<<Denotes type is Exchange>>>>>

 

……….

 

Private key is NOT exportable

Encryption test passed

This output has been trimmed down and the key points have been bolded above.

If the certificate is invalid then when you run the command you may see an error like this.

Example Error
***********

C:\Program Files\Microsoft Data Protection Manager\DPM\bin>SetDPMServer -dpmCredential CertificateConfiguration_DPM2012.contoso.com.bin -Outputfilepath c:\temp

-Thumbprint 4301114a1d05b44bc834f34f04f4cb4333433bac

Error(Id= 33234), Details : The certificate provided with thumbprint 4301114a1d05b44bc834f34f04f4cb4333433bac on the personal machine store of machine MemberServerTest does not correspond to the requirements of DPM. The following requirements are not met for the certificate.

The certificate is not trusted on the local machine.

Please make sure certificate fulfills the following requirements:

1) The certificate is trusted on the local machine and has not expired.
2) The revocation servers of the associated Certificate Authorities are online.
3) The certificate has an associated private key with a valid exchange algorithm.
4) The certificate's public key length is greater than or equal to 1024 bits.
5) The certificate should have both Server and Client Authentication if EnhancedKey Usage is enabled.
6) The subject of the certificate and its root CA should not be empty.
7) DPM does not support certificates with Cryptography API Next Generation (CNG)keys.

For more details see help.
SetDpmServer failed with errorcode =0x809909b4, error says: (null)

Note the 33234 error which equates to an invalid cert. Most likely the cert used does NOT meet our requirements. Again you can use the command certutil –store –v my to verify our certs in use.

Troubleshooting the Attach-ProductionServerWithCertificate and the SetDPMServer commands

Attach-ProductionServerWithCertificate

1.) On the DPM server upon the attach attempt If you get the following error:

image

You will need to place the client bin file on the DPM server system32 directory OR specify the full path of the bin file. In the example above we specified:

DPMServerName: DPM2012
PSCredential: CertificateConfiguration_MemberServer.Contoso.com.bin <----This is not the full path so it will, by default, search the system32 directory.

If we placed the cert in a folder named C:\Cert then we would specifiy:

DPMServerName: DPM2012
PSCredential: C:\Cert\CertificateConfiguration_MemberServer.Contoso.com.bin <--This is a full path to the certificate we wish to use.

2.) On the DPM server the Attach-ProductionServerWithCertificate on the DPM server creates a registry key for the protected computer with the certificate.

HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\2.0\Certificates\<Protected ComputerName>

image

Note the Certificate name and port number.

If the Attach-ProductionServerWithCertificate fails, then the following needs to be looked at:

a.) There is a network issue between the DPM Server and Protected Computer. You can use the telnet command to verify if port 6076 is open for communication from both the ends.

b.) Certificate used for DPM server is not trusted on the Protected Computer. In Certificates MMC verify that ROOT CA Certificate is present in the Trusted Root Certification Authorities. Go to the Workstation and check the DPMRACurr.errlog files for failures.

SetDPMServer

1.) Specifying a Wrong Bin file on the Target server

In this case the SetDPMServer command was used to setup protection. We purposely used the wrong bin file for the DPM server to simulate an error.

image

2.) On the DPM server, when running the Set-DPMCredential the following registry key is created:

HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\2.0\Certificates\<DPMServerName>

Note the Certificate name, port number and thumbprint specified.

The Set-DPMCredentail command also enables the DPM CPWrapper Service and configures it to use the certificate.

3.) Failures while running this are logged in the DPM management Shell, MSDPM*.errlog and the CAP12 event viewer logs.

Sample Errors

Error in DPM Management Console:
***************************
Set-DPMCredentials : Unable to find certificate with the thumbprint 8d8bddbc15d73f3c20c3faf3faab9b69075e582c on the personal machine store of machine DPM2012.contoso.com. (ID: 33231)

Error in MSDPMCurr.errlog
************************
ConfigureCertificates.cs(400) NORMAL Getting certificate for thumbPrint : 8d8bddbc15d73f3c20c3faf3faab9b69075e582c

CertificatesHelper.cs(51) NORMAL Looking for Certificate with thumbprint: 8d8bddbc15d73f3c20c3faf3faab9b69075e582c in store: My at location: LocalMachine

CertificatesHelper.cs(88) NORMAL Could not find Certificate with thumbPrint: 8d8bddbc15d73f3c20c3faf3faab9b69075e582c in store :My at location :LocalMachine

ConfigureCertificates.cs(133) WARNING Getting certificate for thumbPrint : 8d8bddbc15d73f3c20c3faf3faab9b69075e582c failed

ConfigureCertificates.cs(256) WARNING Failed to configure the dpm credentials with exception: Microsoft.Internal.EnterpriseStorage.Dls.Utils.DlsException: Getting certificate for thumbPrint : 8d8bddbc15d73f3c20c3faf3faab9b69075e582c failed

ConfigureCertificates.cs(256) WARNING at Microsoft.Internal.EnterpriseStorage.Dls.CertificateHelper.ConfigureCertificates.GetCertificateByThumbPrint(String thumbPrint)

ConfigureCertificates.cs(256) WARNING at Microsoft.Internal.EnterpriseStorage.Dls.CertificateHelper.ConfigureCertificates.ConfigureDPMCredentials(String certificateThumbPrint, String authCAThumbprint, String outputFilePath, Boolean generateFileOnly)

Conclusion

This concludes Part 1 of DPM Certificate Based Authentication. Part 2 will entail troubleshooting missing or corrupt registry keys and their symptom and Part 3 will go over missing or invalid certificates.

Shane Brasher | Senior Support Escalation Engineer

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

DPM Certificate Troubleshooting–Part 2: Registry

$
0
0

toolsign[5]

Hello, Shane Brasher here again. This article picks up from where DPM Certificate Troubleshooting–Part 1: General Troubleshooting left off. We are going to jump right in and look at a few failed scenarios when your DPM certificate related registry keys are missing or corrupt.

When you have everything setup and working, the certs are in the right store and the proper command syntax has been used, there are specific registry entries placed on both the DPM server and the member server being protected. In this next section we will go over what errors you can expect to see when either the certificates or the registry keys are missing.

Theme: “Certs check, registry check, DPMCPWrapperService restart check. Rinse. Repeat.”
This troubleshooting theme should be strictly adhered to during your certificate based authentication troubleshooting. In the scenarios below, after I would remove a reg key or cert, in order to get things back into a working state I would have to repeat those steps. This is so important that it warrants repeating. When troubleshooting DPM certificate based authentication:

a.)Check the registry keys on both the DPM server and protected server.
b.) Check the certificate in use
c.) Restart the DPM CPWrapper service.

Registry Entries

We will look at the following scenarios:

DPM Server
Missing DPM cert reg key (it’s own key)
Missing Member cert reg key

Member Server
Missing DPM cert reg key
Missing Member server cert reg key (it’s own key)

We will note the following:
Error in the DPM gui
Error in the DPM alerts event log
Error in the MSDPMCurr.errlog
Errors in the DPMRACurr.errlog
Errors in the DPM CPWrapper log

After running both the SetDPMserver command on the member server or the Attach-ProductionServerWithCertificate command on the DPM server, registry entries are placed on the servers to associate the certificate with the DPM server and the protected server.

The default location is HKLM\Software\Microsoft\MicrosoftTDataProtectionManager\Agent\2.0\Certificates\<DPMServerName> or <ProtectedServerName>

DPM SERVER Registry Keys

DPM Server side registry keys

image

DPM Server Missing DPM cert reg key (its’ own reg key)

In this example we will look at the errors in the:
a.) DPM management tab
b.) DPM Alerts Event log
c.) MSDPMCurr.errlog

After the Set-DPMCredentail command is run, if the registry key on the DPM Server for the DPM server itself is missing or deleted for some reason then you can expect the following error in the DPM GUI:

Reg Key:
HKLM\Software\Microsoft\MicrosoftTDataProtectionManager\Agent\2.0\Certificates\<DPMServerName>


DPM Management Agent Status
****************************

image

Although this error suggests to check the CPWrapper service on the member server, which is not a bad idea, the issue in this case is with the DPM server itself. Remember this error was produced by removing the DPM certificate registry key.

Usually the 33304 indicates an issue with the DPM CPWrapper Service. In this case the related registry keys that bind to that service. (There is a list of the common causes for this error discussed in Part 3 of this series.) If this DPM registry key is missing then you may also see the DPM CPWrapper service in a “starting” state if not then a restart of this service may fail when attempting to bind the service with that missing registry key. You would also see a crash log generated in the following directory: C:\Program Files\Microsoft System Center 2012\DPM\DPM\Temp directory. The crash log name itself will be such as: DPMCPWrapperServiceCurr.errlog.2012-04-30_19-25-50

If the DPM registry key is missing then a consistency check and\or a recovery point on a protected datasource using certificate authentication will fail with the following errors.

DPM Alerts Event Log Error
*************************
You may get one if not all of the alerts listed below.

DPM Alerts Event Log: Event ID 3122 Warning

image

DPM Alerts Event Log: Event ID 3115 Warning

image

DPM Alerts Event Log: Event ID 3170 Critical

image

MSDPMCurr.errlog
******************

WARNING Failed: Hr: = [0x80990940] pDpmCmdProcObject->SubmitRequest failed on server MEMBERSERVER.Contoso.com, hrOriginal = 0x80990940, No further retry

WARNING CCommandProcessor::SendOutboundCommandUsingCertificate failed for Server: MEMBERSERVER.Contoso.com

4b0d-8401-d9773b85e7ab" xmlns="http://schemas.microsoft.com/2003/dls/StatusMessages.xsd">

WARNING <ErrorInfo ErrorCode="33304" DetailedCode="-2137454272" DetailedSource="2" ExceptionDetails="" xmlns="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd">

C2797F36-E616-4D5C-AC68-D9DA2216CE2D WARNING <Parameter Name="exceptionmessage" Value="The CPWrapper WCF Service encountered an unknown communication error" />

Solution: In this case to where the registry key is missing for the DPM Server itself on the DPM Server, the following needs to be done.

1.) Restore the key via registry backup, if no backup is available for this key and\or you do not feel comfortable with this measure then proceed to the next step.

2.) Verify that a valid certificate is in place on the DPM server. Once done, rerun the Set-DPMCredentials command to recreate that key. This being done taking care to use the proper syntax and correct thumbprint. Please reference the resource link below. Once done make sure the DPM reg key is present.
Example:

image

Resource: http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

DPM Server Missing the Member Server Reg key

After Attach-ProductionServerWithCertificate command is run on the DPM server, if the registry key on the DPM server for the protected server is missing or corrupted for some reason then you can expect to see the following errors listed below.

In this example we will be noting the errors in the:
a.) DPM monitoring tab
b.) DPM management tab
c.) DPM events alerts tab
d.) MSDPMCurr.errlog

Reg Key:
HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\2.0\Certificates\<Protected ComputerName>


DPM Management Tab--Agent Status
*******************************

image

DPM Monitoring Tab
*******************

Agent refresh error

image

DPM Alerts Event Logs—Event 3122

image

DPM Monitoring Tab--Protected server Consistency Check failure

image

MSDPMCurr.errlog
================

2C1A3335-C179-4D87-A993-CBD5B8B8A7C1 WARNING Failed: Hr: = [0x80070005]

0C9C 0F78 04/30 15:17:21.846 68 RornTaskDef.cs(488) 2C1A3335-C179-4D87-A993-CBD5B8B8A7C1 NORMAL RORN TaskDef: Task 2c1a3335-c179-4d87-a993-cbd5b8b8a7c1 stopped with error code 302

0C9C 0F78 04/30 15:17:21.846 02 EventManager.cs(98) 2C1A3335-C179-4D87-A993-CBD5B8B8A7C1 NORMAL Publishing event from AgentJobs.cs(747): JobProgress, [JobID=9470259c-538c-4e3d-8dc6-aff5bcee9d3c]

0C9C 0F78 04/30 15:17:21.847 07 AgentJobs.cs(751) 2C1A3335-C179-4D87-A993-CBD5B8B8A7C1 NORMAL refresh failed with error AMAgentAccessDenied; -2147024891; WindowsHResult

0C9C 0F78 04/30 15:17:21.847 01 TaskExecutor.cs(843) 2C1A3335-C179-4D87-A993-CBD5B8B8A7C1 FATAL Task stopped (state=Failed, error=AMAgentAccessDenied; -2147024891; WindowsHResult), search "Task Diagnostic Information" for details.

Solution: In the case of the DPM server missing the proper reg key for the protected member server, the following needs to be done:

1.) Restore the key via registry backup, if no backup is available for this key and\or you do not feel comfortable with this measure then proceed to the next step.

2.) Make sure you have the proper .bin file generated by the member server. Once done, then run the Attach-ProductionServerWithCertificate.ps1 command specifying the correct .bin file. Please reference the resource link below. Once done verify the member server registry key is present.

Example:

image

Resource: http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

Protected Server Side Registry Keys
Now we will focus on the protected server. We will experiment with both removing the Protected server registry key and the DPM registry key off of the Protected server. Once done we will take note of the common errors shown as a result.

We look at the following:
a.) DPM management tab
b.) DPM events alerts tab
c.) MSDPMCurr.errlog
d.) DPMRACurr.errlog

Member Server with Reg Key for itself missing.

If after running the setdpmserver –dpmCredential command, on the protected server if the registry key for itself is missing or deleted you may see the following errors below:

Reg Key:

HKLM\Software\Microsoft\MicrosoftDataProtectionManager\Agent\2.0\Certificates\<protectedServerName>

image

DPM Management Tab-Agent Status

image

DPM Alert Event Log-Event ID 3122

image

MemberServer Event Log –Event ID 85

image

MemberServer DPMRA.currerrlog
==========================

schannelutils.cpp(129) 7F9E668E-2A1D-4D55-A498-D7FA318B6068 WARNING Failed: Hr: = [0x80070002] : Error trying to open RegKey [HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\2.0\Certificates\MemberServer.Contoso.com]

0EF075F8-504F-48E4-9BAF-85418F0DBD68 WARNING Logging event for error: 33304, detailed: 0x30bf80

Note: Error 33304 has numerous causes listed at the later on in Part 3 of this series. This is the same indication as we saw when we removed the DPM registry key. In this case it is the member server missing its own registry key.

DPM MSDPMCurr.errlog
===============

034C 0FD4 04/30 15:55:09.481 07 AMUtil_expanded.cs(3590) 92CBF7B2-BA70-4ACF-B0DA-16FE40E43376 WARNING CheckTimeoutMessage: code[0x20000102], detailedCode[0x8099090e], errMgs[Internal error code: 0x8099090E]

TaskInstance.cs(798) 92CBF7B2-BA70-4ACF-B0DA-16FE40E43376 WARNING &lt;q1:ErrorInfo ErrorCode=”316” DetailedCode=”-2137454322” DetailedSource=”2” ExceptionDetails=”” xmlns:q1=”http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd”&gt;

TaskInstance.cs(798) 92CBF7B2-BA70-4ACF-B0DA-16FE40E43376 WARNING &lt;q1:Parameter Name=”servername” Value=”MEMBERSERVER.Contoso.com” /&gt;

RornTaskDef.cs(488) 92CBF7B2-BA70-4ACF-B0DA-16FE40E43376 NORMAL RORN TaskDef: Task 92cbf7b2-ba70-4acf-b0da-16fe40e43376 stopped with error code 316

92CBF7B2-BA70-4ACF-B0DA-16FE40E43376 FATAL Task stopped (state=Failed, error=AMAgentNotResponding; -2137454322; WindowsHResult),

Solution: If member server itself is missing it’s own registry key the we will need to perform the following:

1.) Restore the key via registry backup, if no backup is available for this key and\or you do not feel comfortable with this measure then proceed to the next step.

2.) Make sure we have the proper certificate in the computer\personal store on the member server.

3.) Make sure we have the correct .bin file created from the DPM server when you ran the Set-DPMCredentials on the DPM server.

4.) Run the SetDPMServer command on the member server taking care to make sure the correct DPM .bin file is specified along with the correct member server thumbprint from the certificate. Please reference the resource listed below.

Resource: http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

MemberServer with Missing DPM reg Key

If after running the setdpmserver –dpmCredential command, on the protected server if the registry key for the DPM Server is missing or deleted you may see the following errors:

Reg Key:

HKLM\Software\Microsoft\MicrosoftDataProtectionManager\Agent\2.0\Certificates\<DPMServerName>

DPM Management Tab- Agent Status

image

DPM Monitoring Tab

image

DPM Alerts Event Log—Event ID 3122

image



Log Name: DPM Alerts
Source: DPM-EM
Date: 4/28/2012 6:34:44 AM
Event ID: 3122
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: DPM2012.Contoso.com
Description:The DPM protection agent on MEMBERSERVER.Contoso.com could not be contacted. Subsequent protection activities for this computer may fail if the connection is not established. The attempted contact failed for the following reason: (ID: 3122)

The DPM CPWrapper Service authorization failed on the MEMBERSERVER.Contoso.com computer. Exception Message = Access is denied.. (ID: 33303)

Note the 3303 error which indicates that the client was not authorized by the service.

DPM Alerts Event Log—Event ID 3170

image

MSDPMCurr.errlog
****************

0DF4 0634 04/30 15:30:28.570 01 TaskInstance.cs(798) 51A021F9-8D5E-477B-94D4-73A986EEBDC0 WARNING &lt;q1:ErrorInfo ErrorCode="33303" DetailedCode="-2146233087" DetailedSource="2" ExceptionDetails="" xmlns:q1="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd"&gt;

0DF4 0634 04/30 15:30:28.570 01 TaskInstance.cs(798) 51A021F9-8D5E-477B-94D4-73A986EEBDC0 WARNING &lt;q1:Parameter Name="servername" Value="MEMBERSERVER.Contoso.com" /&gt;

0DF4 0634 04/30 15:30:28.570 01 TaskInstance.cs(798) 51A021F9-8D5E-477B-94D4-73A986EEBDC0 WARNING &lt;q1:Parameter Name="exceptionmessage" Value="Access is denied." /&gt;

0DF4 0634 04/30 15:30:28.573 01 TaskExecutor.cs(843) 51A021F9-8D5E-477B-94D4-73A986EEBDC0 FATAL Task stopped (state=Failed, error=WCFServiceAuthorizationFailed; -2146233087; WindowsHResult), search "Task Diagnostic Information" for details.

Note: the 33303 error which indicates that the client was not authorized by the service.

Solution: This will be the same steps done for the member server missing it’s own registry entry.
Those steps will recreate both the DPM server and member server registry key.

Conclusion: As a precautionary measure, a proactive step of backing up the server side and DPM side registry keys are suggested. This can be done via System state or BMR backup but should you not want to rollback to a previous system state or BMR snapshot, then just backing up those keys would work. In addition should you wish to backup the individual keys the restore would be much quicker vs. a system state and BMR restore.

This concludes Part 2 of DPM Certificate Troubleshooting. Part 3 will cover troubleshooting missing or invalid certificates.

Shane Brasher | Senior Support Escalation Engineer

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

Update Rollup 2 for System Center 2012 is now available for download

$
0
0

KBThis rollup includes updates for App Controller, Data Protection Manager (DPM), Operations Manager (OpsMgr), Orchestrator, Service Manager (SCSM) and Virtual Machine Manager (VMM). Download links, installation instructions and the list of issues fixed for each component are documented in the following KB:

KB2706783 - Description of Update Rollup 2 for System Center 2012 (http://support.microsoft.com/kb/2706783)

J.C. Hornbeck | System Center & Security Knowledge Engineer

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

DPM Certificate Troubleshooting–Part 3: Certificates

$
0
0

toolsign[5][2]

Hello, Shane Brasher here once again, following up DPM Certificate Troubleshooting–Part 1: General Troubleshooting and DPM Certificate Troubleshooting–Part 2: Registry with Part 3 of “DPM Certificate Authentication Troubleshooting”.

In this session we will go over some common symptoms you may see if the certificate is missing or is invalid. This is assuming that after you have installed the certificate, run all the proper commands and even have protection group set up, then later something has happened to the certificate itself.

Member Server with its certificate missing
This error is what you will likely see if AFTER cert protection is setup is done and then the cert is missing or corrupt.

DPM Management Tab-Agent status

image

MemberServer Application Alerts—Event ID 85

image

MemberServer DPMRACurr.errlog
****************************

5BD3AD20-B2AF-4D1F-95B6-B73212768440 WARNING Failed: Hr: = [0x80092004] : Error locating certificate with thumbprint 2ba53e0056bdde64a7fca789c62abd72a3f57610

5BD3AD20-B2AF-4D1F-95B6-B73212768440 WARNING Failed: Hr: = [0x80092004] : Encountered Failure: : lVal : CertificateUtil::GetCertificateContext(hCertStore, ssThumbprint, &pCertContext)

WARNING Failed: Hr: = [0x80092004] : Error locating certificate with thumbprint 2ba53e0056bdde64a7fca789c62abd72a3f57610

WARNING Failed: Hr: = [0x80092004] : Encountered Failure: : lVal : CertificateUtil::GetCertificateContext(hCertStore, ssThumbprint, &pCertContext)

WARNING OuterException of type System.InvalidOperationException from Method = GetCertificateFromStoreCore

WARNING Exception Message = Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine', FindType 'FindByThumbprint', FindValue '2ba53e0056bdde64a7fca789c62abd72a3f57610'.

Note: The highlighted portion shows that there is an issue with finding the thumbprint for the certificate.

Member Server DPM CPWrapper Log—Cert is missing and the CP Wrapper Service restarted.
******************************

WARNING Exception Message = Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine', FindType 'FindByThumbprint', FindValue '2ba53e0056bdde64a7fca789c62abd72a3f57610'.

WARNING Exception Stack = at System.ServiceModel.Security.SecurityUtils.GetCertificateFromStoreCore(StoreName storeName, StoreLocation storeLocation, X509FindType findType, Object findValue, EndpointAddress target, Boolean throwIfMultipleOrNoMatch)

WARNING Caught unhandled exception : System.InvalidOperationException: Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine', FindType 'FindByThumbprint', FindValue '2ba53e0056bdde64a7fca789c62abd72a3f57610'.

CRITICAL Exception Message = Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine', FindType 'FindByThumbprint', FindValue '2ba53e0056bdde64a7fca789c62abd72a3f57610'. of type System.InvalidOperationException, process will terminate after generating dump

Also if the Protected server cert if removed upon a reboot or the restart of the DPM CPWrapper service you may see the following error.

image

Solution: If the member server has it’s certificate missing then the following will need to be done.

1.) If the cert is backed up to a safe location, import the certificate into the proper computer\personal store and restart the DPMCPWrapper service. If you do not have a backup of the certificate then proceed to the next step.

2.) Request a new certificate making sure to specify the correct cert attributes and that it is placed into the computer\personal store.

3.) Re-run the SetDPMServer commands to recreate the memberserver bin file. Copy the bin file to the DPM server. Once done re-run the Attach-ProductionServerWithCertificate.ps1 on the DPM server. Please reference the resource link below.

Resource: http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

Important: There may be times to where you may still have to reboot both the member server and the DPM server.

DPM Server With Missing Cert
This scenario will go over symptoms when the DPM server certificate is missing.
If the DPM server is missing its certificate then you will see this in the DPM gui on the agent refresh.

DPM Monitoring tab
******************

image

Note the 3301 error which means the certificate is invalid.

DPM Alerts Event Log
******************

image

Note: The 33301 equates to the certificate is invalid.

MSDPMCurr.errlog snippet
***********************
cmdprocforcertificate.cpp(331) [000000001A7F4F50] WARNING CCommandProcessor::SendOutboundCommandUsingCertificate failed for Server: MemberServer.Contoso.com

WARNING ConfigureProtection.OnFailure.AADeactivationBlock.RAForRead.PT : RADeleteWorkItem, StatusReason = Timeout (StatusCode = -2146233079, ErrorCode = WCFClientCertificateInvalid, workitem = a1e5773c-a587-4788-a7fb-622f6bf7341e)

5A0AC966-C3A0-4D24-95FF-E96FD0DE04DA WARNING CheckTimeoutMessage: code[0x00008215], detailedCode[0x80131509], errMgs[Unknown error (0x80131509) (0x80131509)]

5A0AC966-C3A0-4D24-95FF-E96FD0DE04DA WARNING <ErrorInfo ErrorCode="33301" DetailedCode="-2146233079" DetailedSource="2" ExceptionDetails="" xmlns="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd">

5A0AC966-C3A0-4D24-95FF-E96FD0DE04DA WARNING <Parameter Name="machinename" Value="DPM2012Backup.Contoso.com" />

5A0AC966-C3A0-4D24-95FF-E96FD0DE04DA WARNING <Parameter Name="exceptionmessage" Value="Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine', FindType 'FindByThumbprint', FindValue '994b424d93fc08e4fe88c787298c7617ee095cda'." />

DPMCPWrapperServiceCurr.errlog
=============================
This may be seen upon restarting the DPMCPWrapper service if the cert is missing.

everettexception.cpp(761) CRITICAL Exception Message = Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine', FindType 'FindByThumbprint', FindValue 'c8ccf847ae8d319691feea1d6f796f0d67fdc7c4'. of type System.InvalidOperationException, process will terminate after generating dump

Note the message in regards to generating a dump. This dump (crash log) will be located in the following directory: %Program Files%\Microsoft System Center 2012\DPM\DPM\Temp. The dump file will have a name of “DPMCPWrapperServiceCurr.errlog.2012-07-11_18_06_16.Crash

Solution: If the DPM Certificate is missing, then please follow the steps below.

1.)If the cert is backed up to a safe location, import the certificate into the proper computer\personal store and restart the DPMCPWrapper service. If you do not have a backup of the certificate then proceed to the next step.

2.)Request a new certificate for the DPM server making sure to specify the correct cert attributes and that it is placed into the computer\personal store.

3.) Re-run the SetDPMCredentils commands to recreate the DPM bin file. Copy the bin file to the member server. Once done re-run the SetDPMServer command on the member server to generate this bin file. Copy the member server bin file to the DPMserver.

4.) On the DPM server re-run the Attach-ProductionServerWithCertificate.ps1 command.

Please reference the resource link below.

Resource: http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

Important: This is considered a very bad situation. As if you have many servers you are protecting via certificate based authentication and the DPM cert is missing, it will be like starting all over again. You will have to:

a.) Generate the DPM bin file
b.) Copy it to each server that you were protecting via cert authentication.
c.) run the setdpmserver command
d.) take each server bin file to the DPM server.
e.) on that DPM server run the attach command.

This will have to be done for each server that you are protecting with certificate authentication. Naturally if you are protecting 100 servers via cert then this can be very labor intensive.

As a precautionary measure I strongly suggest that you export your DPM and member server certificates and save them in a safe location.

Expired Certificate

MemberServer Cert Expired

If the certificate has expired on the protected server then you will see the following errors.

DPM Management Tab-Agent Status

image

DPM Monitoring Tab
=================

image

DPMRACurr.errlog
================

415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD FATAL <Status xmlns="http://schemas.microsoft.com/2003/dls/StatusMessages.xsd" StatusCode="-2146893016" Reason="Error" CommandID="RAReadDatasetDelta" CommandInstanceID="80b85883-9822-4a64-bea0-1c661101dbe5" GuidWorkItem="856c0da1-fad7-46ba-a215-db95b90de630" TETaskInstanceID="415bf1bd-04ef-486c-a8d0-0c6a8e8e0bbd"><ErrorInfo xmlns="http://schemas.microsoft.com/2003/dls/GenericAgentStatus.xsd" ErrorCode="536872925" DetailedCode="-2146893016" DetailedSource="2"><Parameter Name="AgentTargetServer" Value="MemberServer.Contoso.com"/></ErrorInfo><RAStatus><RAReadDatasetDelta xmlns="http://schemas.microsoft.com/2003/dls/ArchiveAgent/StatusMessages.xsd" BytesTransferred="0" NumberOfFilesTransferred="0" NumberOfFilesFailed="0" DataCorruptionDetected="false"/></RAStatus></Status>

415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING Failed: Hr: = [0x80090328] : Encountered Failure: : lVal : hr

415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING Logging event for error: 33302, detailed: 0xa61590

415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING Failed: Hr: = [0x00008216] CCmdProcEvent::GetEventId: unexpected errorCode: detailed hr: 0x00a61540

Note: The error codes -2146893016 and 0x80090328 basically translates to
SEC_E_CERT_EXPIRED
# The received certificate has expired.

The error code 33302 is the service authentication failed.

DPM SERVER with Cert Expired

If the certificate has expired on the DPM server the you will see an error like this.

image

DPMRCurr.errlog

ExceptionPolicy.cs(169) WARNING InnerException of type System.IdentityModel.Tokens.SecurityTokenValidationException from Method = Build

02F8 094C 05/02 17:32:29.282 04 ExceptionPolicy.cs(174) WARNING Exception Message = The X.509 certificate CN=DPM2012.Contoso.com chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. A required certificate is not within its validity period when verifying against the current system clock or the timestamp in the signed file.

02F8 094C 05/02 17:32:29.282 04 ExceptionPolicy.cs(174) WARNING Exception Stack = at System.IdentityModel.Selectors.X509CertificateChain.Build(X509Certificate2 certificate)

02F8 094C 05/02 17:32:29.282 04 cmdprocforcertificate.cpp(232) [0000000000B6FB90] 415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING Failed: Hr: = [0x80990941] pDpmCmdProcObject->SubmitResponse failed on server DPM2012.Contoso.com, hrOriginal = 0x80131501, No further retry

02F8 094C 05/02 17:32:29.282 04 cmdprocforcertificate.cpp(331) [0000000000B6FB90] 415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING CCommandProcessor::SendOutboundCommandUsingCertificate failed for Server: DPM2012.Contoso.com

02F8 094C 05/02 17:32:29.282 04 cmdproc.cpp(2631) [0000000000B6FB90] 415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING Logging event for error: 33302, detailed: 0xa61590

02F8 094C 05/02 17:32:29.282 04 events.cpp(89) [0000000000A2FF90] 415BF1BD-04EF-486C-A8D0-0C6A8E8E0BBD WARNING Failed: Hr: = [0x00008216] CCmdProcEvent::GetEventId: unexpected errorCode: detailed hr: 0x00a61540

DPM DPMCPWrapperServiceCurr.errlog

CertificatesHelper.cs(498) NORMAL Certificate with subject: CN=DPM2012.Contoso.com and thumbprint: 02E436145567778DED5E95138343AE1F19163ED1 is not valid

0AD0 0C3C 05/02 18:07:28.110 09 CertificatesHelper.cs(503) WARNING Flags = NotTimeValid, Info = A required certificate is not within its validity period when verifying against the current system clock or the timestamp in the signed file.

0AD0 0C3C 05/02 18:07:28.110 09 CertificatesHelper.cs(166) WARNING The certificate with subject: CN=DPM2012.Contoso.com is not trusted

DPM CPWrarpper Error logs codes

For Client related errors (33300 – 33302) refer to dpmra*.errlog or msdpm*.errlog.

For service side errors (33303 – 33304) look up failures in dpmcpwrapperservice*.errlog

For PKI related issues, the Crypto API Event log is a very useful way to figure out what went wrong during certificate validation. This event log is available from Windows Vista onwards.

Configuration Step Errors
33231 : Certificate not found in Personal Store of “LocalMachine” StoreLocation.
33232 : Exception trying to locate a certificate.
33233 : Exception encountered trying to validate certificate.
33234 : The certificate is invalid.
33235 : Error trying to add a firewall rule.
33236 : Error trying to configure DpmCPWrapperService.
33237 : The generic fall back error.
33241 : No .NET 3.5 SP1 detected on the machine (seen only by SetDpmServer.exe).

Errors during business continuity
33300: Configuration errors in the WCF Client config file. (dpmra.exe.config or msdpm.exe.config).
33301: Client certificate is invalid.
33302: The service authentication failed.
33303: The client was not authorized by the service.
33304: The WCF Service is in a bad state. Some possible reasons can be:
a.) Service not running on the remote peer.
b.) Crash in the WCF Service.
c.) WCF unresponsive to client requests leading to Timeouts.
d.) Generic communication failures.
e.) Authentication failure of the client on the service side.
f.) Missing Registry keys

Conclusion: It is imperative that your Certificate infrastructure is extremely solid with a good connection to the CRL for both the DPM server and the member server. In addition there needs to be a stable link between the DPM server and the member server. Once the certs are in place they should be left alone and not need to be altered in any manner. Of course its best to be prepared with a contingency plan should things go awry. As mentioned earlier, its suggested to export your certificates for safe keeping should you have to recover from a missing certificate.

Appendix A

CAPI2 Event Logging

If you are facing repeated authentication failures, refer CAPI2 event viewer logs on both DPM and protected computer. This is not enabled by default. To enable it navigate to:
Event Viewer\Applications and Service Logs\Microsoft\CAPI2
Then right click on “Operational” and select “Enable Log”.

image

Once done reproduce the problem.

Example:

image

Going into the details of the properties we can see:

image

This tells us the CRL server cannot be reached.

4.) Make sure the DPM CPWrapper Service is started and set to “Automatic”. If it is not, then restart the service and test your Attach then OR your connectivity via cert usage.

Additional Resources

Microsoft Root Certificate Program : http://technet.microsoft.com/en-us/library/cc751157.aspx

How to use certificates to authenticate computers in workgroups or untrusted domains with Data Protection Manager : http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

Shane Brasher | Senior Support Escalation Engineer

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/


Support Tip: Moving a DPM machine from a domain to workgroup causes Consistency Checks to fail

$
0
0

toolsign5

Consider the following scenario:

In your environment you have a Windows domain controller, a computer running System Center 2012 Data Protection Manager (DPM) and one member server. You are using Certificate Based Authentication (CBA) for the domain member and the domain member name is MemberServer.Contoso.Com.

Note: The proper steps are followed to setup CBA as per the following:

http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

You have successfully created a PG with successful backup.

You later move the server from the domain as a member and place it into a workgroup. The server name changes from "MemberServer.Contoso.com" to just "MemberServer". This is important to note.

From this point forward, a Consistency Check (CC) will fail with the following error:

DPM Alert Event: (ID: 3170) DPM failed to communicate with the protection agent on MemberServer.Contoso.com because the computer is unreachable.

If you perform an attach-productionserverwithcertificate.ps1 command you will see the following error:

DPM Alert Event: (3122) The DPM protection agent on memberserver could not be contacted. Subsequent protection activities for this computer may fail if the connection is not established. The attempted contact failed for the following reason: (ID:3122) The DPM CPWrapper Service authorization failed on the MemberServer computer. Exception Message= Access is denied. (ID: 33303)

Cause

The general thought is that if you have the certificate in place and it's valid and it can resolve the CRL then all should work if you move the server into a workgroup. This is actually incorrect. The thumbprint in use by the memberserver is used to create a bin file and to make registry entries on both the protected server and the DPM server.

Registry Key created on both DPM server and Protected server is MemberServer.Contoso.com.
Bin file used to create the registry entries is: CertificateConfiguration_MemberServer.Contoso.com.bin.

When DPM performs an authorization check, it checks the registry for MemberServer.Contoso.com, notes that its there and makes a CC attempt. The problem is that this server does not exist anymore. Remember, we removed it from the domain. As such, any CC attempts for that server will fail.
When you attempt to perform an attach-productionserverwithcertificate.ps1 command, this also fails because we do not have a new bin file created for MemberServer. Remember that since we removed the server from the domain the server name has changed. The DPM server has no associated bin file or registry entry for "MemberServer" but rather "MemberServer.Contoso.com".

Resolution

1.) Re-run the SetDPMserver command on the protected server. This will create:

a.) A bin file named CertificateConfiguration_MemberServer.bin
b.) The associated registry keys on the protected server.

2.) Take the CertificateConfiguration_MemberServer.bin file to the DPM server and re-run Attach-ProductionServerWithCertificate.ps1, specifying the newly created bin file from the protected server. This will create:

a.) The associated registry key on the DPM server for MemberServer.

You can now create a new Protection Group (PG) for "MemberServer" and continue with your backups.

NOTE: For the old PG, you will not be able to associate it with this server. You can delete the PG and retain data to disk. See the following for more information.

http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx

Shane Brasher | Senior Support Escalation Engineer

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

Cloud and Datacenter Management System Center Update Rollup Improvements

$
0
0

Greetings,

Hi, Carmen Summers here from the Cloud and Data Center Management System Center team.  Yesterday System Center released Update Rollup 2 for System Center 2012 posted on all System Center Blogs.  This is the second Update Rollup System Center 2012 has released since General Availability.  In the past year System Center has developed a new model for Sustained Engineering that centralized Virtual Machine Manager, App Controller, Orchestrator, Service Manager, Operations Manager, and Data Protection Manager into a single operational model.  This new operating model will deliver consistency for release of Update Rollups for Cloud and Data Center Management System Center components.

The words above accurately describe the efficiency we want to achieve, but at the end of the day we are doing this for you, our customers, based upon feedback that we needed to make update delivery for System Center more discoverable, more accessible, and more consistent.  Some of the new operational efficiencies that will benefit you, our customers, are:

  • Quarterly Update Rollup releases
  • Update Rollups are the primary way CDM System Center will release updates publicly
  • Each Update Rollup will supersede the last making is easier track if you have everything you need installed
  • Update delivery on Microsoft Update to enable ease of detection and installation via MU or WSUS (Please note, that this is a work in progress)
  • A single master KB article that describes all fixes to enable ease of finding all information in a single location

We are always striving to make our operating model more efficient.  If you have any suggestions or feedback I would love to hear it.

Carmen Summers
Senior Program Manager | Cloud & Datacenter Management System Center

Important note on DPM 2012 and the Windows Server 2012 Hyper-V replica role

$
0
0

InformationHello, Shane Brasher here, and I wanted to take a minute today to talk to you about a Windows Server 2012 Hyper-V / System Center 2012 Data Protection Manager (DPM) SP1 supportability issue. First let’s briefly discuss an exciting new feature of Windows Server 2012 Hyper-V. There is a new functionality added to the Hyper-V role that enables you to implement Business Continuity and Disaster Recovery. This functionality is called Hyper-V replication or Hyper-V Replica. This new feature allows you to have a Hyper-V Primary server that replicates it’s virtual machines to another server hosting the Hyper-V replica role. Any changes made on the Primary Hyper-V are replicated over to the Hyper-V Replica server every 5 minutes, thus if the Primary Hyper-V server should fail then the Hyper-V Replica can take over the workload. More information can be found at the link below:

Hyper-V Replica Overview : http://technet.microsoft.com/en-us/library/jj134172.aspx

The important thing to note about this is that while the DPM agent can be installed on both servers with no issues and you can backup the Primary DPM server as usual with no problems, on the Hyper-V Replica server you can enumerate the virtual machines and “may” even be able to back them up successfully, however backing up or restoring the Hyper-V replica is not supported.

Due to the inner workings of the Hyper-V replication architecture which may be in progress during the time of a DPM backup, there can be no guarantees of a successful backup or restore of virtual machines that reside on the Hyper-V Replica server. You will still be able to backup other types of data on the Hyper-V Replica such as flat files and system state for example.

A common question that is often asked is “if the Replica is a complete backup of the Primary Hyper-V server virtual machines, then why would I want to back that up again with DPM if redundancy is already built-in.” Well, then answer is you wouldn’t need to backup the Hyper-V Replica but you still may want to backup the Primary Hyper-V server for many reasons.

First, you may have a mandatory retention range set via service level agreement that you may want to adhere to. Second, you may choose to perform a restore from a previous point in time for example 2 weeks ago. Third, you may choose to perform a restore from a previous point in time to another Hyper-V server for testing without interrupting productivity. Forth, it’s possible there may be some catastrophic corruption on one of the virtual machines. For example, perhaps you have a virtual machine that has a virus and the virtual machine along with the virus is replicated over to the Hyper-V Replica.

In conclusion the key points are this:

Backing up or restoring virtual machines from a Windows Server 2012 Hyper-V Replica is not supported. If you need an extra layer of redundancy for your Windows Server 2012 Hyper-V virtual machines, you can achieve this by backing up the Hyper-V Primary server.

Shane Brasher | Senior Support Escalation Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

System Center 2012 Service Pack 1 Beta Now Available for Download

$
0
0

downloadThe Beta of System Center 2012 Service Pack 1 (“SP1”) enables System Center customers to jointly evaluate System Center 2012 with Windows Server 2012 and Windows 8. The Beta is for evaluation purposes only and not to be used in production as described in the EULAs associated with the product. No license keys are required to do this evaluation. The Beta includes updates and enhancements to the following System Center 2012 components:

  • Virtual Machine Manager
    • Improved Support for Network Virtualization
    • Extend the VMM console with Add-ins
    • Support for Windows Standards-Based Storage Management Service, thin provisioning of logical units and discovery of SAS storage
    • Ability to convert VHD to VHDX, use VHDX as base Operating System image
  • Configuration Manager
    • Deployment and management of Windows 8 and Windows Server 2012
    • Distribution point for Windows Azure to help reduce infrastructure costs
    • Automation of administrative tasks through PowerShell support
    • Management of Mac OS X clients and Linux and UNIX servers
    • Real-time administrative actions for Endpoint Protection related tasks
  • Data Protection Manager
    • Improved backup performance of Hyper-V over CSV 2.0
    • Online Backup support with Windows Azure Online Backup service
    • Protection for Hyper-V over remote SMB share
    • Protection for Windows Server 2012 de-duplicated volumes
    • Uninterrupted protection for VM live migration
  • App Controller
    • Service Provider Foundation API to create and operate Virtual Machines
    • Support for Azure VM; migrate VHDs from VMM to Windows Azure, manage from on-premise System Center
  • Operations Manager
    • Support for IIS 8
    • Monitoring of WCF, MVC and .NET NT services
    • Azure SDK support
  • Orchestrator
    • Support for Integration Packs, including 3rd party
    • Manage VMM self-service User Roles
    • Manage multiple VMM ‘stamps’ (scale units), aggregate results from multiple stamps
    • Integration with App Controller to consume Hosted clouds
  • Service Manager
    • Apply price sheets to VMM clouds
    • Create chargeback reports
    • Pivot by cost center, VMM clouds, Pricesheets
  • Server App-V
    • Support for applications that create scheduled tasks during packaging
    • Create virtual application packages from applications installed remotely on native server

For all the details and a download link please see the following: http://www.microsoft.com/en-us/download/details.aspx?id=34607

J.C. Hornbeck | Knowledge Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity- support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

Help Documentation for Using Windows Azure Online Backup with System Center 2012 SP1 – Data Protection Manager

$
0
0

imageSystem Center 2012 SP1 Data Protection Manager leverages Windows Azure Online Backup to allow users to store their data online. This document includes documentation to assist users with deploying and using this feature of System Center 2012 SP1 Beta DPM.

This Document will describe how to deploy and use System Center 2012 SP1 Data Protection Manager (DPM) to store backup data in the cloud using the Windows Azure Online Backup service. With System Center 2012 SP1, DPM can now support online backup using the Windows Azure Online Backup service. To use this functionality, customers will need to subscribe to the Windows Azure Online Backup service. Following are the key benefits of the new online backup capabilities in DPM:

  • Reduced TCO: This service with Azure based public cloud storage will reduce total TCO for customers by providing scalability, elasticity and simplified storage management.
  • Peace of mind: Windows Azure based backup service helps provide a reliable, secure, robust offsite backup & restore solution that is highly available.
  • Simplicity: The online backup workflows are seamlessly integrated into the existing DPM backup, recovery and monitoring workflows

For all the details please see the following:

http://www.microsoft.com/en-us/download/details.aspx?id=34608

J.C. Hornbeck | Knowledge Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

Support Tip: Scheduled backup to tape runs on a wrong date on DPM 2007, 2010 and 2012

$
0
0

toolsignHi there DPM administrators, Wilson Souza here from DPM Support team. As you know, Data Protection Manager offers many ways to protect server workloads  such as Exchange, SQL, SharePoint, Hyper-V, System State, Bare metal, Files, Shares etc. Protection can be done in the following ways:

Disk to Disk (D-D) – When the protected data goes from a disk on the protected server to a volume that sits on your DPM Server

Disk to Tape (D-T)– When the protected data goes from a disk on the protected server to a tape device managed by the DPM server.

Disk to Disk to Tape (D-D-T) – The combination  of the two options above where protected data goes from a disk on the protected server to volume that sits on the DPM server and then we copy that data from the DPM volume to a tape device.

 

When using tapes, we have a range of options available as to when we want the backup to go to tape. This can be daily, weekly, monthly, quarterly, yearly or any other combination that you see fit to your SLA.

 

DPM delegates the schedule backup control to SQL agent and when it is time for the tape backup to run, SQL agent triggers a DPM engine job to start the backup.

 

So let’s say that one day you get into the office in the morning to check how backups are being created and notice that a backup that was supposed to run 2 months from now just got completed last night, and you didn’t get a warning stating that DPM would run that backup on an unexpected date. The purposes of this blog is to explain the issue and provide a workaround.

 

NOTE: This issue does not affect daily, weekly, or monthly tape backup schedules, it primarily affects tape backups that are in multiples of months, like quarterly, semi-annually, yearly etc. This issue is presented on DPM 2007, 2010 and 2012 but this workaround is not applicable to DPM 2007.

 

 

EXPLAINING THE ISSUE

 

  clip_image001

 

Assume that today is 10/07/2011 and we created a new protection group and set long term protection (weekly, quarterly and yearly). At the end of the new protection group wizard, DPM will create the necessary scheduled jobs and send them over to SQL Agent.

As illustrated, this is the quarterly backup as seem from SQL Agent.

 

Note: Quarterly backups should run only on Jan/Apr/Jul/Oct

 

 clip_image002

 

SQL Agent shows that this job should run two days after the job was created

 

  clip_image003

 

On the 10/09/11, the backup to tape ran as expected. For the quarterly schedule definition we are now expecting this backup to run on 01/09/2012.

 

Almost every action done on a Protection Group (manually: add/remove protected members, modify disk allocation or simply completing a modify protection group wizard without making any change or automatically: SQL and SharePoint auto protection, Disk auto grow) will cause all scheduled jobs from that group to be deleted and recreated. This is where scheduled jobs have a potential of running on a wrong date.

 

By deleting/creating new  schedules, DPM will use the original XML to generate the new scheduled job. The ScheduleXml will use the original Start Date which could now be in the past. Below is the snipped of ScheduleXML.

 

<?xml version="1.0" encoding="utf-16"?>

<Schedule xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" ScheduleID="9b0c036b-5c2d-49b8-a374-3842ba6cfb96" JobDefID="c5241cb4-8dc2-4574-b758-2e7b7db0ca70" xmlns="http://schemas.microsoft.com/2003/dls/Scheduler.xsd">

  <Recurrence>

    <Monthly StartDt="2011-10-09" EndDt="9999-12-31" Interval="3" MonthDayList="9" />

    <Time StartTm="20:00:00" EndTm="20:00:00" />

  </Recurrence>

</Schedule>

 

  clip_image004

 

Now we fast forward to November the 6th and the protection group was modified to add a new data source. The original scheduled job above will be removed and a new one created in its place.

Note that the start date below is unchanged.

 

 clip_image005

 

Highlighted are the new schedules created by the modify protection group operation (one for weekly, quarterly and yearly). The quarterly new scheduled job is the second from the last line (Next Run = 11/09/2011 8:00:00 PM)

 

 clip_image006

 

SQL agent sees that this schedule  was set to run for the first time on 10/09/2011.  Now it is almost a month later and Last Run column shows that this job never ran. To resolve that, SQL Agent will set the Next run time for this new job for the first available date. As we are on the 6th, and the 9th is three days from now, SQL Agent will schedule this job to run on the 9th.

 

So instead of the quarterly backup being run in January as expected, it will now run 2 months earlier. In addition, this job won’t show up as scheduled in the DPM UI (there is an explanation for that but we will cover it on another blog). You will only see a reference to this job when it is running, completed or failed.

 

WORKAROUND

 

To work around this issue, copy the script below to SQL Server Management Studio and execute it. This new stored procedure will check if the start date is in the past, and if it is it will calculate the next run time and set it accordingly. From the SQL Agent standpoint, the wrong schedule can only happen when the scheduled start date is set to be in the past which will always be the case once the original schedule day is past.

Script:

=====

 

USE [DPMDB]

GO

/****** Object:  StoredProcedure [dbo].[prc_IM_UserSchedule_Update]    Script Date: 11/10/2011 19:11:01 ******

******* Edited by........: Wilson Souza     

******* Version..........: 2.2

******* Date Created.....: 11/10/11

******* Date Last Change.: 04/04/11

*******                                     THIS SCRIPT IS FOR DPM 2010/2012 RTM   

******* Change Log for V 2.2

*******           Addressed issue if selected DAY instead any day of the week.

******* Change log for V 2.0

*******           Now using XML variable to retrieve data instead of searching string on schedule variable

*******           Now addresses issues for Weekly schedules. Not only Months

*******           Now addresses issues when user select First, Second, Third, Four or Last day of the month.

*/

 

SET ANSI_NULLS ON

GO

SET QUOTED_IDENTIFIER ON

GO

 

--

-- Update one row in UserSchedule table by ScheduleID.

-- If this ScheduleID doesn't exist,

--  add a new row with this ScheduleID.

--

ALTER PROCEDURE [dbo].[prc_IM_UserSchedule_Update]

(

    @ScheduleID GUID,

    @ProtectedGroupID GUID,

    @JobType tinyint,

    --------------- Change Start ---------------

    -- @Schedule ntext,

    @Schedule nvarchar(max),

    --------------- Change end ---------------

    @Immediacy bit,

    @TimeOffset int,

    @MaxDuration bigint,

    @ScheduleListId GUID

)

AS

 

DECLARE @error int,

        @rowcount int,

       

       

    --------------- Change Start ---------------

     

                  @xml                                xml,

                  @CurrentDate                        date,

                  @ForLOfTheMonth                     date, -- First or Last Day of the Month

                  @count                              int,

                  @count1                             int,

 

                  @Monthly_StartDt                    date,

                  @Monthly_Interval                   int,

                  @Monthly_MonthDayList               int, -- This might not be needed

 

                  @MonthlyRelative_StartDt            date,

                  @MonthlyRelative_Interval           int,

                  @MonthlyRelative_RelativeWeekDay    nvarchar(3),

                  @MonthlyRelative_RelativeInterval   nvarchar(6),

                 

                  @Weekly_StartDt                     date,

                  @Weekly_Interval                    int,

                  @Weekly_WeekDayList                 nvarchar(20) -- This might not be needed

 

 

set @xml = CONVERT(xml,SUBSTRING(@schedule,42,LEN(@schedule)-41))

set @CurrentDate = GETDATE()

 

select @Weekly_StartDt        = @xml.value ('(//*[local-name()="Weekly"]/@StartDt)[1]', 'date')

select @Weekly_Interval       = @xml.value ('(//*[local-name()="Weekly"]/@Interval)[1]', 'int')

select @Weekly_WeekDayList    = @xml.value ('(//*[local-name()="Weekly"]/@WeekDayList)[1]', 'nvarchar(20)') -- This might not be needed

 

select @Monthly_StartDt       = @xml.value ('(//*[local-name()="Monthly"]/@StartDt)[1]', 'date')

select @Monthly_Interval      = @xml.value ('(//*[local-name()="Monthly"]/@Interval)[1]', 'int')

select @Monthly_MonthDayList  = @xml.value ('(//*[local-name()="Monthly"]/@MonthDayList)[1]', 'int') -- This might not be needed

 

select @MonthlyRelative_StartDt           = @xml.value ('(//*[local-name()="MonthlyRelative"]/@StartDt)[1]', 'date')

select @MonthlyRelative_Interval          = @xml.value ('(//*[local-name()="MonthlyRelative"]/@Interval)[1]', 'int')

select @MonthlyRelative_RelativeWeekDay   = @xml.value ('(//*[local-name()="MonthlyRelative"]/@RelativeWeekDay)[1]', 'nvarchar(3)')

select @MonthlyRelative_RelativeInterval  = @xml.value ('(//*[local-name()="MonthlyRelative"]/@RelativeInterval)[1]', 'nvarchar(6)')

 

If @Monthly_StartDt is NOT NULL

      while @Monthly_StartDt < @Currentdate

            Set @Monthly_StartDt = DATEADD(MONTH,@Monthly_Interval,@Monthly_StartDt)

           

if @Weekly_StartDt is NOT NULL

      if @Weekly_Interval > 1

            while @Weekly_StartDt < @CurrentDate

                  set @Weekly_StartDt = DATEADD(DAY,@Weekly_Interval * 7,@Weekly_StartDt)      

     

If @MonthlyRelative_StartDt is NOT NULL

Begin

      set @ForLOfTheMonth = DATEADD(dd,-(DAY(DATEADD(mm,1,@Currentdate))-1)-(DAY(@Currentdate)-DAY(DATEADD(mm,1,@Currentdate))),@Currentdate)

      if @MonthlyRelative_RelativeInterval = 'Last'

      begin

            set @ForLOfTheMonth = DATEADD(Month,1,@ForLOfTheMonth)

            set @ForLOfTheMonth = DATEADD(dd,-(DAY(DATEADD(mm,1,@ForLOfTheMonth))-1)-(DAY(@ForLOfTheMonth)-DAY(DATEADD(mm,1,@ForLOfTheMonth))),@ForLOfTheMonth)

            set @ForLOfTheMonth = DATEADD(day,-1,@ForLOfTheMonth)

      end

      while @MonthlyRelative_StartDt < @CurrentDate

      begin

            while @MonthlyRelative_StartDt < @ForLOfTheMonth

                  Set @MonthlyRelative_StartDt = DATEADD(MONTH,@MonthlyRelative_Interval,@MonthlyRelative_StartDt)

            if @MonthlyRelative_RelativeInterval = 'Last' 

            Begin

                  set @MonthlyRelative_StartDt = DATEADD(Month,1,@ForLOfTheMonth)

                  set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

                  set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)      

            End

            else

                  set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

            if @MonthlyRelative_RelativeInterval = 'First' or @MonthlyRelative_RelativeInterval = 'Last'

                  set @count = 1

            if @MonthlyRelative_RelativeInterval = 'Second'

                  set @count = 2

            if @MonthlyRelative_RelativeInterval = 'Third'

                  set @count = 3

            if @MonthlyRelative_RelativeInterval = 'Fourth'

                  set @count = 4

            set @count1 = @count

            if @MonthlyRelative_RelativeWeekDay = 'Day'

                  if @count <> 1

                  Begin

                        set @MonthlyRelative_StartDt = DATEADD(dd,@count-1,@MonthlyRelative_StartDt)

                        set @count = 0

                  End

                  Else

                        set @count = 0         

            while @count <> 0

            begin

                  if substring(DATENAME(dw,@MonthlyRelative_StartDt),1,2) = @MonthlyRelative_RelativeWeekDay

                        set @count = @count - 1

                  if @count <> 0

                        if @MonthlyRelative_RelativeInterval <> 'Last'

                              set @MonthlyRelative_StartDt = DATEADD(day,1,@MonthlyRelative_StartDt)

                        else

                              set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)

            end

            if @MonthlyRelative_StartDt < @CurrentDate

            begin

                  set @MonthlyRelative_StartDt = DATEADD(MONTH,@MonthlyRelative_Interval,@MonthlyRelative_StartDt)

                  set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

                  if @MonthlyRelative_RelativeInterval = 'Last'

                  begin

                        set @MonthlyRelative_StartDt = DATEADD(Month,1,@MonthlyRelative_StartDt)

                        set @MonthlyRelative_StartDt = DATEADD(dd,-(DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))-1)-(DAY(@MonthlyRelative_StartDt)-DAY(DATEADD(mm,1,@MonthlyRelative_StartDt))),@MonthlyRelative_StartDt)

                        set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)

                  end

                  set @count = @count1

                  if @MonthlyRelative_RelativeWeekDay = 'Day'

                        if @count <> 1

                        Begin

                              set @MonthlyRelative_StartDt = DATEADD(dd,@count-1,@MonthlyRelative_StartDt)

                              set @count = 0

                        End

                        Else

                              set @count = 0         

                  while @count <> 0

                  begin

                        if substring(DATENAME(dw,@MonthlyRelative_StartDt),1,2) = @MonthlyRelative_RelativeWeekDay

                              set @count = @count - 1

                        if @count <> 0

                              if @MonthlyRelative_RelativeInterval <> 'Last'

                                    set @MonthlyRelative_StartDt = DATEADD(day,1,@MonthlyRelative_StartDt)

                              else

                                    set @MonthlyRelative_StartDt = DATEADD(day,-1,@MonthlyRelative_StartDt)

                  end  

            end  

      end

End

 

if @Monthly_StartDt is NOT NULL

      set @xml.modify ('replace value of (//*[local-name()="Monthly"]/@StartDt)[1] with sql:variable("@Monthly_StartDt")')

If @MonthlyRelative_StartDt is NOT NULL

      set @xml.modify ('replace value of (//*[local-name()="MonthlyRelative"]/@StartDt)[1] with sql:variable("@MonthlyRelative_StartDt")')

if @Weekly_StartDt is NOT NULL

      set @xml.modify ('replace value of (//*[local-name()="Weekly"]/@StartDt)[1] with sql:variable("@Weekly_StartDt")')

 

set @Schedule = '<?xml version="1.0" encoding="utf-16"?>  ' + CONVERT(nvarchar(max),@xml)

 

 

      --------------- Change end ---------------

 

SET @rowcount = 0

SET @error = 0

 

SET NOCOUNT ON

 

UPDATE  dbo.tbl_IM_UserSchedule

SET     ProtectedGroupID   = @ProtectedGroupID,

        JobType            = @JobType,

        Schedule           = @Schedule,

        Immediacy          = @Immediacy,

        TimeOffset         = @TimeOffset,

        MaxDuration        = @MaxDuration,

        ScheduleListId     = @ScheduleListId

WHERE ScheduleID = @ScheduleID

 

SELECT @error = @@ERROR, @rowcount = @@ROWCOUNT

 

IF (@error = 0 AND @rowcount = 0)

BEGIN

    INSERT INTO dbo.tbl_IM_UserSchedule

    (

        ScheduleID,

        ProtectedGroupID,

        JobType,

        Schedule,

        Immediacy,

        TimeOffset,

        MaxDuration,

        ScheduleListId

    )

    values

    (

        @ScheduleID,

        @ProtectedGroupID,

        @JobType,

        @Schedule,

        @Immediacy,

        @TimeOffset,

        @MaxDuration,

        @ScheduleListId

    )

 

    SET @error = @@ERROR

END

 

SET NOCOUNT OFF

 

RETURN @error

=====

Wilson Souza | Senior Support Escalation Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

MMS 2013 Dates Announced!


System Center 2012 Update Rollup 3 (UR3) Released!

$
0
0

We are pleased to announce that System Center 2012 Update Rollup 3 (UR3) has been released.  Keeping with our currently schedule of releasing update rollups quarterly, Update Rollup 3 provides updates for issues that have been reported to Microsoft.

This update contains updates for Service Manager, Data Protection Manager, and Operations Manager.  This also marks the first time that Data Protection Manager and Operations Manager updates have been released via Microsoft Update (MU).

clip_image002

 

Issues that are fixed in Update Rollup 3 for System Center 2012

Update Rollup 3 for System Center Data Protection Manager 2012 (KB2751230)

Issue 1
After you upgrade System Center Data Protection Manager 2010 to System Center Data Protection Manager 2012, the tape management report does not display overdue tapes.
Issue 2
System Center Data Protection Manager 2012 Client Protection does not scale to the limits that are expected.
Issue 3
When you try to specify a client computer name in the DPMServerName attribute by using Windows PowerShell, Windows PowerShell crashes.
Issue 4
When the name of a Microsoft SharePoint site collection contains a space, and you perform a SharePoint item-level recovery operation in System Center Data Protection Manager 2012, the operation fails.
Issue 5
After you rename a SharePoint site in System Center Data Protection Manager 2012, you cannot restore the site.
Issue 6
The SharePoint Recovery Point Status Report displays incorrect data in System Center Data Protection Manager 2012.
Issue 7
A bare metal recovery fails in certain situations.

Update Rollup 3 for System Center Operations Manager 2012 (KB2750631)

Issue 1
When you use the 32-bit version of Windows Internet Explorer to start a web console, the Microsoft.EnterpriseManagement.Presentation.Controls.SpeedometerGaugeUIController controller does not work correctly.
Issue 2
When you run a Windows PowerShell cmdlet, you receive the following error message: 

Get-BPAModel is not recognized as the name of a cmdlet.

Issue 3
When you try to change a URL in the "web application availability monitoring" template instance, the change is not applied.

Update Rollup 3 for System Center Service Manager 2012 (KB2750615)

Issue 1
When you open or close the Incident form in the System Center Service Manager 2012 console, a memory leak occurs.
Issue 2
When form control objects are rooted in the Garbage-Collected (GC) Heap, the System Center Service Manager 2012 console crashes, and you receive an OutOfMemoryException exception.
Issue 3
After you change the SharePoint site language to Turkish in the System Center Service Manager 2012 portal, the display strings are displayed in English unexpectedly.
Issue 4
When you open the System Center Service Manager 2012 console by using a Citrix application, and then you open the Incident form, you experience slow performance.

 

You can find more details and instructions for obtaining and installing the update rollup on the associated KB article here:

http://support.microsoft.com/kb/2756127

 

Thank you all for your feedback on these issues.  Please continue to create support cases for issues that you encounter so they can be triaged for inclusion in future cumulative updates or service packs.

KB: MSDPM.EXE crashes during Garbage Collection causing DPMDB growth in Data Protection Manager 2007

$
0
0

imageHere’s a new Knowledge Base article we published. This one talks about an issue where MSDPM.EXE crashes during Garbage Collection causing DPMDB growth in DPM 2007.

=====

Symptoms

Every day at midnight, System Center Data Protection Manager 2007 (DPM) starts a couple of maintenance tasks which are referred to as Garbage Collection. During Garbage Collection, a couple things happen such as:

 

  • Expired Recovery Points are removed
  • Entries from the DPMDB database that are older than 33 days are removed

A problem that can occur is that during Garbage Collection, the DPM service (MSDPM.EXE) crashes and the Garbage Collection job never completes. As result, stale data within DPMDB is never completely removed and you end up with a large and ever growing DPMDB database.

Cause

This can occur if SQL encounters a deadlock and in order to resolve it, one of the store procedures involved in the deadlock is killed by the SQL Engine.

You can find something similar to the following in MSDPMCurr.errlog if this is occurring:

NOTE If DPM was installed in its default location, this file will be in C:\Program Files\Microsoft DPM\DPM

Attempt 1 failed with exception Microsoft.Internal.EnterpriseStorage.Dls.DB.NonFatalDbException: exception ---> System.Data.SqlClient.SqlException: Transaction (Process ID 63) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.Internal.EnterpriseStorage.Dls.DB.SqlRetryCommand.ExecuteNonQuery()
--- End of inner exception stack trace ---
*** Mojito error was: DatabaseNonFatalError; 0; None
--- SqlException details -----------------
System.Data.SqlClient.SqlException: Transaction (Process ID 63) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.Internal.EnterpriseStorage.Dls.DB.SqlRetryCommand.ExecuteNonQuery()
Error = 1205
Index #0
Source: .Net SqlClient Data Provider
Number: 1205
State: 52
Class: 13
Server: <Server_Name>
Message: Transaction (Process ID 63) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Procedure: prc_PRM_GarbageCollect
Line: 714
--- Original Command -----------------
dbo.prc_DLS_GarbageCollect
--- Caller StackTrace -----------------
FileName:; Method:ExecuteNonQuery(); lineNo:0; ilOffset:8.
FileName:; Method:CleanupDlsDatabase(); lineNo:0; ilOffset:119.
FileName:; Method:OnStart(); lineNo:0; ilOffset:260.
FileName:; Method:Start(); lineNo:0; ilOffset:2.
FileName:; Method:Execute(); lineNo:0; ilOffset:8.
FileName:; Method:ChangeState(); lineNo:0; ilOffset:62.
FileName:; Method:Process(); lineNo:0; ilOffset:338.
FileName:; Method:Function(); lineNo:0; ilOffset:16.
FileName:; Method:Run(); lineNo:0; ilOffset:95.
FileName:; Method:PerformWaitCallbackInternal(); lineNo:0; ilOffset:28.
FileName:; Method:PerformWaitCallback(); lineNo:0; ilOffset:40.

Resolution

This is a known issue in System Center Data Protection Manager 2007. To have this issue addressed, please contact Microsoft Support (http://support.microsoft.com/contactus/).

More Information

This issue is addressed in System Center Data Protection Manager 2010 and later.

=====

For the most current version of this article please see the following:

2758637 - MSDPM.EXE crashes during Garbage Collection causing DPMDB growth in Data Protection Manager 2007

J.C. Hornbeck | Knowledge Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

KB: Creating protection for Hyper-V VMs on Windows Server 2012 fails with Internal error code 0x809909E2

$
0
0

imageHere’s a new Knowledge Base article we published. This one talks about an issue where using DPM 2012 SP1 to create a protection group for a Hyper-V workload that is running on Windows Server 2012 fails to complete.

=====

Symptoms

When using System Center 2012 Data Protection Manager (DPM) SP1 to create a protection group for a Hyper-V workload that is running on Windows Server 2012, the protection will not complete. The symptoms vary based on the configuration of the Hyper-V server.

For stand-alone Hyper-V server

In DPM, creation of the protection group will fail with the error:
Type: Replica creation
Status: Failed
Description: Failure occurred while adding one or more of the volumes involved in backup operation to snapshot set. Please check the event log on %HypervServer%.lab to troubleshoot the issue. (ID 30290 Details: Internal error code: 0x809909E2)
More information
End time:
Start time:
Time elapsed: 00:00:05
Data transferred: 0 MB
Cluster node -
Source details: \Backup Using Child Partition Snapshot\Server1
Protection group: Protection Group 1

For both stand-alone and CSV Hyper-V

On the Hyper-V server, the application event log will show Event IDs 12292 and 13 from VSS:

Log Name: Application
Source: VSS
Date:
Event ID: 12292
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer:
Description:
Volume Shadow Copy Service error: Error creating the Shadow Copy Provider COM class with CLSID {463948d2-035d-4d1d-9bfc-473fece07dab} [0x80070005, Access is denied.].
Log Name: Application
Source: VSS
Date:
Event ID: 13
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer:
Description:
Volume Shadow Copy Service information: The COM Server with CLSID {463948d2-035d-4d1d-9bfc-473fece07dab} and name HWPRV cannot be started. [0x80070005, Access is denied.]

Cause

This is caused by the iSCSI Target Storage Provider returning an unexpected state.

Resolution

There are two options to workaround this issue.

Option One:

Create a registry key to force DPM to skip trying any hardware providers:

HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\UseSystemSoftwareProvider

NOTES:

  • Presence of this key will force DPM to use the software provider, so if you have and want to use a hardware provider do not use this key. Likewise, if you add a hardware provider in the future you will need to remove it.
  • This is a key under Agent and not a value.
  • This must be added to each node of the CSV cluster that DPM is using (has an agent on).
  • This does not require a reboot of the nodes.

Option Two:

One the Hyper-V server(s) involved, remove the iSCSI Target Storage Provider (VSD and VSS) by using the Add Roles and Features Wizard. It is a Role under the File And Storage Services/File and iSCSI Services area.

More Information

The Hyper-V servers will also have the following logged in the Program Files\Microsoft Data Protection Manager\DPM\Temp

DPMRA*.errlog

146C 1140 09/25 04:00:54.578 31 vsssessioncontext.cpp(143) [0000000001D8EEC0] WARNING </VSS_CONTEXT>
146C 1384 09/25 04:01:24.581 31 vsssnapshotrequestor.cpp(645) [0000000001D86690] WARNING Failed: Hr: = [0x80042316] : CSV2 Snapshot failed with UnexpectedProviderError (0x8004230F), Mapping it to SnapshotSetInProgress error, Volume: \\?\Volume{e1b5c403-3d99-4c5c-a521-840f1b61bb75}\
146C 1384 09/25 04:01:24.581 31 createsnapshotsubtask.cpp(1780) [0000000001D87FB0] WARNING Failed: Hr: = [0x80042316] : Encountered Failure: : lVal : pSnapshotRequestor->StartPrepareForBackup(snapshotSetId, m_fUseSystemSoftwareProviderOnly)
146C 1384 09/25 04:01:24.581 05 fsmstate.cpp(167) [0000000001D8CE70] WARNING Failed: Hr: = [0x80042316] : Encountered Failure: : lVal : pTransition->Execute(pEvent)
146C 1384 09/25 04:01:24.581 05 genericfsm.cpp(225) [0000000001D8D3F0] WARNING Failed: Hr: = [0x80042316] : Encountered Failure: : lVal : m_pCurrentState->SendEvent(pEvent, pNextState)
146C 1438 09/25 04:01:24.581 31 hypervwriterhelperplugin.cpp(535) [0000000001D8C1D0] NORMAL Component 80DC3791-5605-4EC3-AE14-1D742C8A6AF5 is a VM
146C 1438 09/25 04:01:24.607 31 vsssnapshotrequestor.cpp(544) [0000000001D86690] NORMAL CVssSnapshotRequestor::StartPrepareForBackup [0000000001D86690]
146C 1438 09/25 04:01:24.612 31 vsssnapshotrequestor.cpp(593) [0000000001D86690] NORMAL CVssSnapshotRequestor: Using provider {00000000-0000-0000-0000-000000000000} for volume \\?\Volume{e1b5c403-3d99-4c5c-a521-840f1b61bb75}\
146C 1438 09/25 04:01:24.643 31 vsssessioncontext.cpp(143) [0000000001D8EEC0] WARNING <VSS_CONTEXT><COMPONENTS><COMPONENT><WriterId>{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}</WriterId><WriterName></WriterName><LogicalPath></LogicalPath><ComponentName>80DC3791-5605-4EC3-AE14-1D742C8A6AF5</ComponentName><ComponentType>2</ComponentType></COMPONENT></COMPONENTS><SNAPSHOT_CONTEXT> SelectComponent = 1, PartialFileSupport = 0, BootableState = 0, BackupType = 1, SnapshotContext = 0, SnapshotAttributes = VSS_CTX_BACKUP </SNAPSHOT_CONTEXT><SnapShotVolumes><Volume> <Name>\\?\Volume{e1b5c403-3d99-4c5c-a521-840f1b61bb75}\</Name> <SnapshotPath>(null)</SnapshotPath> <MountPoint>C:\ClusterStorage\volume2\</MountPoint> <MountPointArray></MountPointArray> <SnapshotId>{00000000-0000-0000-0000-000000000000}</SnapshotId> <ProviderId>{00000000-0000-0000-0000-000000000000}</ProviderId> <IsHardwareProvider>0</IsHardwareProvider> </Volume></SnapShotVolumes>

=====

For the most current version of this article please see the following:

2761897 - Creating protection for Hyper-V VMs on Windows Server 2012 fails with Internal error code 0x809909E2

J.C. Hornbeck | Knowledge Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

DPM Support Tip: Reporting fails with "implementation is not part of FIPS validated cryptographic algorithms"

$
0
0

GrayAndYellowGears

When opening Reporting Services Configuration Manager, the Web Service URL and Report Manager URL fails with the following error:

Reporting Services Error
An internal error occurred on the report server. See the error log for more details. (rsInternalError) Get Online Help
Exception of type 'System.Web.HttpUnhandledException' was thrown.
This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms.

Stack info:
[InvalidOperationException: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms.]
System.Security.Cryptography.RijndaelManaged..ctor() +200 System.Web.Configuration.MachineKeySection.ConfigureEncryptionObject() +2088
System.Web.Configuration.MachineKeySection.EnsureConfig() +904
System.Web.Configuration.MachineKeySection.GetEncodedData(Byte[] buf, Byte[] modifier, Int32 start, Int32& length) +88
System.Web.UI.ObjectStateFormatter.Serialize(Object stateGraph) +1320
System.Web.UI.Util.SerializeWithAssert(IStateFormatter formatter, Object stateGraph) +248
System.Web.UI.HiddenFieldPageStatePersister.Save() +280
System.Web.UI.Page.SaveAllState() +6488
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +17240

Cause

This can occur if FIPS is enabled and using the RijndaelManaged AES which is not been certified by the National Institute of Standards and Technology (NIST) as compliant with the Federal Information Processing Standard (FIPS). Because of this, the AES algorithm is not part of the Windows Platform FIPS validated cryptographic algorithms.

See http://support.microsoft.com/kb/911722 for more information.

Resolution

Edit the web.config file of directories below per article - http://support.microsoft.com/kb/911722

%DPMInstall%\Program Files\Microsoft DPM\SQL\MSRS.10.MSDPM2010\Reporting Services\ReportManager

%DPMInstall%\Program Files\Microsoft DPM\SQL\MSRS.10.MSDPM2010\Reporting Services\ReportServer

Add the following section to the system.web section

1. In a text editor such as Notepad, open the application-level Web.config file.

2. In the Web.config file, locate the <system.web> section.

3. Add the following <machineKey> section to in the <system.web> section:

<machineKey validationKey="AutoGenerate,IsolateApps" decryptionKey="AutoGenerate,IsolateApps" validation="3DES" decryption="3DES"/>

4. Save the Web.config file.

More Information

If the SSRS log file(%DPMInstall%\Program Files\Microsoft DPM\SQL\MSRS.10.MSDPM2010\Reporting Services\LogFiles) is showing the error below, the SSRS data source does not have the "Allow log on locally" privilege defined for it in the Local Security Policy:

<ERROR>
library!ReportServer_0-2!704!09/27/2012-15:37:05:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.LogonFailedException: Log on failed., ;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.LogonFailedException: Log on failed. ---> System.Runtime.InteropServices.COMException (0x80070569): Logon failure: the user has not been granted the requested logon type at this computer.(Exception from HRESULT: 0x80070569)
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at RSRemoteRpcClient.RemoteLogon.GetRemoteImpToken(String pRPCEndpointName, Int32 type, Guid dataSourceId, String pUserName, String pDomain, String pPassword)
at Microsoft.ReportingServices.Diagnostics.ImpersonationContext.Login(CredentialsType credType, Guid dataSourceId, String userName, String userPwd, String domain)
--- End of inner exception stack trace ---
</ERROR>

To resolve, add the account being used to the "Allow log on locally" security policy.

Andy Nadarewistsch | Senior Support Escalation Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

DPM Support Tip: ID 41 Details: No connection could be made because the target machine actively refused it (0x8007274D)

$
0
0

GrayAndYellowGearsWhen trying to restore to a Recovery Database (RDB) you may get an agent timeout with the following error message:

ID 41 Details: No connection could be made because the target machine actively refused it (0x8007274D)

You may also see errors in the Exchange Application event log for VSS like this one:

Log Name: Application
Source: MSExchangeIS
Date:
Event ID: 9619
Task Category: Exchange VSS Writer
Level: Error
Keywords: Classic
User: N/A
Computer: <Server name>
Description:
Exchange VSS Writer failed with error code -543 when processing the post-restore event.

If any databases were restored, they are likely in a dirty-shutdown state.

Cause

Exchange is using ports 5718 and 5719 which are the same ports that DPM agent communication uses.

Resolution 1

To verify that Exchange is the process using the ports:

1. Open a Command Prompt window. Run the following commands at the command prompt:

netstat -ano > netstat.txt
tasklist > tasklist.txt
tasklist /svc >svclist.txt

Note In this step, the command outputs of the netstat command and the tasklist command are written to text files so that you can check the outputs more easily. Run the tasklist command together with the /svc switch because the process that is using the required ports may be running as a service.

2. Open the text files that were generated in step 1. To do this, run the following commands at the command prompt:

notepad netstat.txt
notepad tasklist.txt
notepad svclist.txt

3. In the Netstat.txt file, find any entries that correspond to TCP port 5718 and to TCP port 5719. Note the process identifier (PID) for each entry.

4. In the Tasklist.txt file, locate the PIDs that you found in step 3 to determine which processes are using the required ports. If you do not find the PIDs in the Tasklist.txt file, try to find the PIDs in the Svclist.txt file.

5. After you find out which process is using the required ports, configure the corresponding program to use other available ports. If you cannot change the program's ports, or if the program uses ports dynamically, you must stop the program.

Note If another application is using the port or ports (5718 and 5719), the ports cannot be changed. In this case, you can, instead, use the SetAgentcfg.exe tool. This tool provides the ability to change the default ports that the DPM agent uses.

To change the ports that are used by the DPM agent, follow these steps on the protected computer that is experiencing the problem. Make sure that the ports that you reassign will not used by any other applications.

1. Locate the SetAgentcfg.exe file from the DPM server. By default, the file is located at the following path:

%PROGRAMFILES%\Microsoft DPM\DPM\Setup\SetAgentCfg.exe

2. Copy the file to the protected computer that is experiencing the problem. Copy the file to the agent DPM\Bin directory. By default, the file is located at the following path:

%PROGRAMFILES%\Microsoft Data Protection Manager\DPM\bin

3. On the protected computer that is experiencing the problem, open an administrative Command Prompt window.

4. In the Command Prompt window, change to the directory to which the SetAgentCfg.exe file was copied. For example, change to the following directory:

%PROGRAMFILES%\Microsoft Data Protection Manager\DPM\bin

5. Run the following command to change the ports that are used by the DPM Agent:

SetAgentCfg e dpmra <port number> <alternate port number>

6. Restart the DPMRA service.

Resolution 2

NOTE It depends on what version of the OS is running on the Exchange server exhibiting the behavior, however the goal is to ensure that ports 5718 and 5719 are not being used by Exchange.

For Windows Server 2000\2003:

The DPM protection agent service cannot start in System Center Data Protection Manager 2007 http://support.microsoft.com/default.aspx?scid=kb;EN-US;947682

How to reserve a range of ephemeral ports on a computer that is running Windows Server 2003 or Windows 2000 Server
http://support.microsoft.com/kb/812873

For Windows Server 2008 and 2008 R2:

You cannot exclude ports by using the ReservedPorts registry key in Windows Server 2008 or in Windows Server 2008 R2
http://support.microsoft.com/kb/2665809

More Information

You can use the command below to reserve these ports using the command line:

netsh int ipv4 Add excludedportrange protocol=tcp startport=5719 numberofports=2

Once those ports are freed up for DPM, the recovery should complete successfully.

Andy Nadarewistsch | Senior Support Escalation Engineer | Management and Security Division

Get the latest System Center news on Facebook and Twitter:

clip_image001 clip_image002

App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/

The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

Viewing all 293 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>