CRM 2013 – Workflow error AccessCheckEx

I was investigating a bug on CRM 2013 , I got the exception below

Unhandled Exception: System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=6.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: SecLib::AccessCheckEx failed. Returned hr = -2147187962, ObjectID: f115e97d-8e19-e511-80ca-000c292122be, OwnerId: 89ed0bdd-7ecd-e411-80c7-000c292122be,  OwnerIdType: 9 and CallingUser: 2ed69167-0bcf-e411-80c7-000c292122be. ObjectTypeCode: 4200, objectBusinessUnitId: 5f964320-05f4-e411-80c9-000c292122be, AccessRights: WriteAccess Detail:
<OrganizationServiceFault xmlns:i=”http://www.w3.org/2001/XMLSchema-instance&#8221; xmlns=”http://schemas.microsoft.com/xrm/2011/Contracts”&gt;
  <ErrorCode>-2147220891</ErrorCode>
  <ErrorDetails xmlns:d2p1=”http://schemas.datacontract.org/2004/07/System.Collections.Generic”&gt;
    <KeyValuePairOfstringanyType>
      <d2p1:key>OperationStatus</d2p1:key>
      <d2p1:value xmlns:d4p1=”http://www.w3.org/2001/XMLSchema&#8221; i:type=”d4p1:string”>0</d2p1:value>
    </KeyValuePairOfstringanyType>
    <KeyValuePairOfstringanyType>
      <d2p1:key>SubErrorCode</d2p1:key>
      <d2p1:value xmlns:d4p1=”http://www.w3.org/2001/XMLSchema&#8221; i:type=”d4p1:string”>-2146233088</d2p1:value>
    </KeyValuePairOfstringanyType>
  </ErrorDetails>
  <Message>SecLib::AccessCheckEx failed. Returned hr = -2147187962, ObjectID: f115e97d-8e19-e511-80ca-000c292122be, OwnerId: 89ed0bdd-7ecd-e411-80c7-000c292122be,  OwnerIdType: 9 and CallingUser: 2ed69167-0bcf-e411-80c7-000c292122be. ObjectTypeCode: 4200, objectBusinessUnitId: 5f964320-05f4-e411-80c9-000c292122be, AccessRights: WriteAccess </Message>
  <Timestamp>2015-06-23T10:02:24.458209Z</Timestamp>
  <InnerFault i:nil=”true” />
  <TraceText>

[Microsoft.Crm.ObjectModel: Microsoft.Crm.ObjectModel.SyncWorkflowExecutionPlugin]
[0dac4467-fb18-e511-80ca-000c292122be: ]
Starting sync workflow ‘Task-Workflow’, Id: 04ac4467-fb18-e511-80ca-000c292122be
Entering ConditionStep1_step: If  Process contains data and is active
Entering SetStateStep4_step: Change record status to completed
Sync workflow ‘Task-Complete’ terminated with error ‘SecLib::AccessCheckEx failed. Returned hr = -2147187962, ObjectID: f115e97d-8e19-e511-80ca-000c292122be, OwnerId: 89ed0bdd-7ecd-e411-80c7-000c292122be,  OwnerIdType: 9 and CallingUser: 2ed69167-0bcf-e411-80c7-000c292122be. ObjectTypeCode: 4200, objectBusinessUnitId: 5f964320-05f4-e411-80c9-000c292122be, AccessRights: WriteAccess ‘
</TraceText>
</OrganizationServiceFault>

Initial thoughts on the Error

I got this error and there were a few things I found interesting

This error was surprisingly informative

The error was thrown by a non code workflow but the cause of the error was thrown by a GUI workflow which was being triggered when a workflow tried to assign an activity.

I was impressed by the level of logging which was generated by a GUI/non code workflow.

I had never thought about it but this line

Microsoft.Crm.ObjectModel: Microsoft.Crm.ObjectModel.SyncWorkflowExecutionPlugin

indicates CRM runs the GUI workflows using code, which must translate the actions into code. This is obvious but I hadn’t thought about it, until seeing the error.

You can use the callerid to find what user is doing the update and check what security roles the user has.

EntityTypeCode

In the error message you can see it mentions ObjectTypeCode

CRM 2011/2013 – Javascript to get the object type code of an entity

Each entity has an individual typecode, this CRM SDK page shows you the values of the default entities

Entity Type Code 4200 is ActivityPointer, which is interesting because the problem was being caused by an update to a task record.

Clues

AccessCheckEx failed – AccessCheckEx is something to do with security and access

In the error message you can see

AccessRights: WriteAccess

This is clearly telling us the user doesn’t have Write access, e.g. the user isn’t allow to update a certain

What was the cause

This bug was partly caused by the complexity of the CRM solution and the different customizations.

Solution complexity refer to not only the customizations which exist in the solution but the number of different customizations.  When a CRM solution has lots of different customizations e.g. workflows, plugins, business rules being triggered at the same time it makes it difficult to understand what is changing a value.

Below is what was happening

1. A task was updated then saved
2. This triggered a pre plugin on the task entity
3. The plugin assigns the case record
4. A plugin was triggered on assign of the case, which assigned all the open tasks to the new case owner
5. The plugin(s) finished
6. A workflow was triggered, which tried set the task to complete.

The error was thrown because the workflow was trying to update the task but the user only had privileges to update tasks they owned.

The reason this bug suddenly appeared was because the assign plugin was added and it wasn’t picked up in DEV testing because developers tested the code using users with System Admin privileges, which I have talked about before

The System Administrator role is a benefit and a curse to CRM developers

It’s tricky to test the effect of adding plugins

Having lots of different types of customizations adds to the complexity of your CRM solution, complex solutions are difficult to debug, understand and extend.

The Solution

Usually with bugs where the user doesn’t have the right security privilege the easy answer is to give the user role those security privileges.

For this bug it wasn’t the correct solution because the users only had access to tasks they owned and we didn’t want to suddenly give them permissions to update tasks they didn’t own.

The plugin code was running in a PRE plugin, so I couldn’t move the task completing code into this plugin.

The bug was becoming more tricky because I did want to keep the case assigning code in their but I didn’t want the assigning case plugin to run and assign the task to the new case owner because the task was about to completed.

My solution was to stop the assign plugin being triggered if was called by another plugin

Read how to do that in the blog below

CRM Plugins – Stopping infinite loops and understanding PluginExecutionContext.Depth

I then created a post task plugin to complete the task.  I didn’t need to do this but it seemed it would be easy to understand if all the changes were made by plugins.

There was an unsuccessful fix when I used impersonation to close the task as System Admin but the users didn’t like the tasks being closed by System Admin, they wanted the user who updated the task to complete the task.

You can read about Impersonation in plugins in the blog post below

CRM 2015 – Understanding impersonation in plugins and knowing when to use it

CRM 2013 error – The file is too large and cannot be uploaded. Please reduce the size of the file and try again.

One of the CRM Developers had created a report 16 megabytes in size and when they tried to import the report they instantly go the message

They searched the internet and fond a page with a likely solution

How to upload large report files in CRM

The eagle-eyed among you will have noticed the CRM developer, they had sent me my own page, awesome.

The blog post was written in March 2011 by a younger Hosk, so I have to admit I only have a dim memory of this problem, luckily for me and other people who have this problem I wrote a detailed blog about it.

This shows the benefit of writing a blog, I can often find the answers to many CRM problems which I have forgotten all about.

Where are reports in CRM 2013?

I was working on a CRM 2013 on-premise instance and I’m never sure where the reports are kept, my way of finding and adding reports is to do an Advanced find

I’m not sure if I have a modified CRM or I just can’t find reports anymore.

Max size limit

You will notice when you try and upload a large report the file size error pops up very quickly, the reason for this is there is a system setting Maximum file Size (in kilobytes)

A tad confusingly this is kept in the Email tab in System Settings

To get to System Settings

Settings –> Administration –> System Settings –> Maxiumum File Size

The downside of modifying this will mean there larger email attachments but I think you should be able to change this, upload your large report and then set the limit back down.

In the example I changed the limit to 20 megabytes.

.NET machine config

Changing the Max File limit alone won’t resolve the problem, you also have to change the machine.config.

You also need to change the httpRuntime executionTimeout setting and the maxRequestLength value. You can probably guess the maxRequestLength is the file size

<httpRuntime executionTimeout = “9000” maxRequestLength=”20480″ />

So I also had to change the machine.config in v4.0.30319 folder.  In my previous blog post I changed the v2 machine config and for my CRM 2013 instance I changed the v4 machine config.  The reason for this is this setting is in the .NET version being used by CRM.  In my previous blog I was using CRM 4 which used .NET v2 and in CRM 2013 the .NET version is v4.I changed the value in the CRM web.config but I would still get the error and had to make the change in .NET version machine.config.

I would read my previous blog post because it goes into the topic in more detail

HERE IS THE FIX

1. Go to C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG
2. Open machine.config
3. Search for <system.web>
4. Add in <httpRuntime executionTimeout=”9000″ maxRequestLength=”10240″ /> under the <system.web> (I added it right before the end)
5. Save
6. Restart IIS

Final thoughts

You will need to make these changes in other servers in other environments because if you try to import these reports where the changes outlined haven’t been made then an error will be thrown when you import the solution

Some people might be wondering why have I written another post about this error/problem.  The first blog post was for CRM 4 and I found I had to change the Maximum file Size System setting, which I didn’t mention in my previous post.

Microsoft Dynamics CRM not working? check these common causes

I started up the DEV server too start my CRM dev this morning only to be confronted with a big fat error

 

 

The same error happened on every page, so it’s a fundamental error.

Yesterday IT sent an email warning me they would be gently powering down the servers for some maintenance (I bet they really just turned them off).

These common errors are another bonus of companies moving to the cloud because these problems won’t occur or will be fixed by Microsoft.

 

Check the time on the server

Checking the time on the server seems an odd thing to do, but if the time on the server is greater than 5 minutes out from your PC then CRM will fail to authenticate the user.  This is a common problem which pops up and to resolve this you need to get the IT team (or you if you don’t have one) to make sure you setting the time on the server properly (e.g. the same as the PC’s) so it stops the servers time drifting.

 

Check the CRM Asynchronous services

This should be your first thing to check because if the ASYNC services have stopped then CRM will stop working.  If you want to learn more about CRM Asyncs, you can go here

Asynchronous service in Microsoft Dynamics CRM

To see if the Async services are running, Microsoft have a document for starting and stopping 

1. Navigate to Start, select Administrative Tools, then click Services.

You should find there are four CRM Async services and you can see if they are started

The Async processing service is usually the cause of CRM not working.

 

The problem displayed on this page was caused

 

If you have restarted the Asyncs and CRM is still not working

Check IIS is up and running

go to the Start menu

type IIS

It will bring up Internet Information Services (IIS) Manager

IIS is the webserver which hosts the Microsoft Dynamics CRM website.  Some times you can find this may be stopped (very very rarely but its something to check, so we can rule it out)

There are a couple of things to check

 

CRM Application pools are running

CRM web app will have an application pool to run the web app, you need to double check its started and running.  The app pool will usually be set to a service account, the main reason you use a service account is because you don’t need to reset the password every 60 days.  Sometimes this setting won’t have been set and the app pool password will change and until someone goes in and puts/resets the password the CRM App pool will stop working.  When the CRM app pool stops working, CRM stops working.

Its worth noting if any of the other app pools are not started because they may be stopped on purpose or maybe have been stopped accidently.

you cann see the CRMAppPool below

 

 

Is the CRM Website up

if you click on the CRM website called Microsoft Dynamics CRM then the details of the web app will appear on the right, you need to check the start is greyed out (which means it is selected)

 

World Wide Web Publishing Service

There is also a service which controls if IIS is up and running and this is called the World Wide Web Publishing Service (named in the 1990’s I should imagine, not many times you see the words of WWW).  This can be turned off to stop IIS springing back to life after the server has been rebooted.

 

SQL Server

I often think of CRM as a web application to show the contents of the CRM SQL database.  If the SQL CRM instance is down, SQL server is off or can’t be contacted then CRM will stop working.

If CRM isn’t working you need to check the SQL server (usually on a different server) is up and running and the CRM instances are running.

 

 

Windows Event Viewer

They are some common things to check if CRM isn’t working but you will also need to check for some pointers for other errors.  The place I usually start is by checking the Event Viewer log on the CRM server.

Go to the CRM Server

Start Menu – type in Event Viewer

or

Control Panel –> Administrative Tools –> Event Viewer

 

Windows Logs –> Application

Look at the errors and warnings to see if there is anything which might point you in the direction of the potential error.  It’s unlikely you will actually get an error, usually I find it’s a warning and the warning will point you in the direction of the cause of CRM not working.

 

here is the logs from the server for the error shown at the top of the blog

 

You can see there are a view warnings, a .NET, a sandbox.  Be warned there is often a lot of noise in the event logs (e.g. warning and errors which are not important).

I lookeda the .NET error

 

Process information:
Process ID: 5348
Process name: w3wp.exe
Account name: DEV\CRMAPPPOOL

Exception information:
Exception type: CrmException
Exception message: The plug-in execution failed because no Sandbox Hosts are currently available. Please check that you have a Sandbox server configured and that it is running.
System.ServiceModel.EndpointNotFoundException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #72E41A64
at Microsoft.Crm.Application.Platform.ServiceCommands.PlatformCommand.XrmExecuteInternal()
at Microsoft.Crm.Application.Platform.ServiceCommands.RetrieveMultipleCommand.Execute()
at Microsoft.Crm.Caching.DefaultSavedQueryIdCacheLoader.LoadCacheData(SavedQueryIdCacheKey key, IOrganizationContext context)
at Microsoft.Crm.Caching.CrmSharedMultiOrgCache`2.LookupEntry(TKey key, IOrganizationContext context)
at Microsoft.Crm.Caching.DefaultSavedQueryIdsCache.TryLookupEntry(Int32 objectType, Int32 savedQueryType)
at Microsoft.Crm.Application.Components.Sdk.FormControls.Web.LookupControl.ConfigureControl()
at Microsoft.Crm.Application.Components.UI.CrmUIControlBase.ConfigureControlInternal()

 

It pointed to the Sandbox service and when I checked the CRM services, I saw the sandbox service was stopped.  I started the sandbox service and CRM popped back into life.