Microsoft System Center Operations Manager 2007

Within our Operations Manager Event Log, on our RMS, we were getting Event Id: 1106 about 7 times in approx 8 to 10 minutes.

Here is the event details:
Event Type: Error
Event Source: HealthService
Event Category: Health Service
Event ID: 1106
Date:  7/3/2009
Time:  7:41:17 PM
User:  N/A
Computer:
Description:
Cannot access plain text RunAs profile in workflow “CleanerOf_domain_ManagementSever_domainname.com”, running for instance “RMS.domainname.com” with id:”{5DB28804-D206-3E8A-FD1D-64839B0521B1}”. Workflow will not be loaded. Management group “ManagementGroupName”

Cause:
=====
Appears to be group membership corruption.

Resolution:
=========

We restarted the Health Service on RMS. The event still generated 7 times in about a minute.

We then found that we were missing some entries under RunAs profiles.

Under RunAsAccount for “Data Warehouse Action Account”, we saw that the RMS had the “Data Warehouse Account” in RunAs profile. The 3 management servers had only “Reserved”.

We added the other 3 Management Servers name in there.

We did the same for the “Data Warehouse Report Deployment Account” and for the “Reporting SDK SQL Server Authentication Account”.

After verifying the RunAs Accounts, we restarted the Health Service on the RMS.

The issue was still happening. However the frequency of this event reduced to 4 times a minute.

We determined from here that the rule is failing is because of perms issue.

Looks like an issue with the Active Directory Management Pack.

Next we took the approach of verifying that Active Directory integration is working properly.

Just as a precaution, we took backup of the Agent Auto assignment settings:
RMS doesn’t have any agents reporting to it.
Agents report only to the Management Servers.

We deleted all three of the SCPs and ran the MOMADAgent.exe as directed: OpsMgr AD Integration – how it works

Next we recreated the  auto agent assignment wizard. Then restarted health service on the RMA, but it took a while for containers to get created (be aware that this may take up to an hour).

Issue continues to happen.

We were dumbfounded as we just didn’t know what was causing Event Id: 1106 to appear.

We then thought – well let’s verify the RunAs Accounts one more time -they are fine.

Verified the AD Users and Computers for the perms in the Operations manager and sub containers. All fine.

Next we decided to do some diagnostic tracing to see just what is going on…

This isn’t something that should be done lightheartedly. I recommend reading up on How to use diagnostic tracing in Ops Mgr 2007.

We saw some issues around the following:

GetPlaintextCredential failed with code WINERROR=80FF005C for worklow ‘CleanerOf_Domain_ManagementServer’, running for instance with id…

Under RunAs Profiles – > for “Active Directory Based Agent Assignment Account”, RunAs Accounts was for “Local System Windows Account”
We created a test Domain Admin RunAs Account and added it there replacing “Local System Windows Account”.
We reset the settings back to “Local System Windows Account”.

We went to the group for operations manager, “OperationsManagerAdmins”.
Added the test domain admin account, named as “TestAccount” over there.
Restarted health service and issue seems to be fixed.

As we can’t have a Domain Admin account to be used there, we reverted back the changes again.

What finally resolved the issue is that, we deleted the RMS machine name under the “OperationsManagerAdmins” group and re-added it back in…

Odd? Yes.

Strange – even more so as we didn’t actually remove it from the group – only the properties of the group. We didn’ t select ‘Apply’ or ‘Okay’ after removing the RMS server from the group… So it appears that the computer object for our RMS is corrupt w/in Active Directory or so it seems…

Corrupt computer object or not, the issue seems to be resolved.

We no longer are receiving the Event ID 1106.

If you run into this problem and need assistance, please feel free to leave a comment and I’ll get back to you asap.