CRM 2011/2013 Investigating CRM Form Performance issues

Performance in CRM is only a problem when performance is a problem 

The Hosk

I have been doing some work on performance of some CRM forms in CRM 2011.  When the topic of performance crops up people the topic of when you should focus on performance won’t be far away.

I’m assuming you haven’t put in the terrible performing code, customizations or processes and you have tidied/refactored customizations to a good standard.   In this scenario I only look into improving performance if there is a problem with performance or the customer isn’t happy with performance.

I can hear some of you thinking

Isn’t it better to make your CRM system run as fast as possible? 

Yes of course it is, a faster CRM system is better than a slower a CRM system but is this a priority to the customer. If the customer is happy with the speed of the CRM system and the current customizations then the customer would benefit more from a system with less bugs and/or have more functionality they would like.

The cost of spending time on optimizing your Microsoft dynamics CRM solution can take quite a long time for a small benefit of a second or half a second on some forms.

Law of diminishing returns

The usual process of optimizing solutions in CRM is there are some big performance gains in maybe 10 percent (or lower) of the CRM system, these can be found usually in

• Code
• Configuration
• Business process

After the initial great gains the effort increases and the reward (time saved) gets smaller and smaller, which is the law of diminishing returns

I like to think of the law of diminishing returns as the bacon sandwich conundrum.  I bacon sandwich is magnicant but the second bacon sandwich  is ok, additional bacon sandwiches get lower on the tastometer, which is a crime agaisnt bacon sandwiches.

I digress because this blog post is about the performance of Forms in CRM 2011.

Focus on the slowest forms

In a CRM system there are lots of forms you can look at improving the performance of, so where should  you start. The main forms you should concentrate on

The slowest forms

The most used forms which are slow

The forms you should focus on first are the slowest forms because these have either the biggest problems or the most problems.  The worse performing forms are likely to have the biggest gains.

The way I have been doing this was seeing how long it took the forms to load and what traffic

Common Culprits of slow form loading

The main culprits of slow loading forms are

• Busy Javascript onload functions
• Too many subgrids
• Lots of oData/fetchXML queries
• Tabs, sections and fields not needed on the form
• bad/slow coding

To start the code optimization, the first step was to find the bottlenecks and worst performing forms.  The next step was to get a benchmark to see how they were loading, to get an average we loaded the form 5 times and the results were quite varied.

We had our list of worst performing forms and we had a quick look at them and made some quick notes

• How many unfiltered OData retrieves
• How many OData calls, where they retrieving the same set
• Number of ribbon buttons
• potential fields/functionality which could be removed by change of business process
• size of Javascript onload function

Tactics – finding those slow form

Fiddler

One of the tools I used to analyize the slow loading CRM forms was fiddler.  I wrote a blog post about Getting Started with Fiddler and CRM.  It’s a great tool to see what calls are being made from the page, how long they took and how much data they were bringing back.

Looking at the fiddler logs I was able to investigate the OData queries and find

• Which OData calls took the longest
• Find OData calls which were not filtered
• Find OData calls which were retrieving the same data multiple times
• general loading times of sections of code

F12 Debugger

The F12 debugger is such an awesome tool for debugging Javascript.  I put in a break point in the onload Javascript and walked through the code.

We are using two main methods to find slow loading code

Looking at the code

Stepping through the code

How to speed up form loading

Does the customer/end user need all the functionality and fields on the form

Ask the customer to re-evaluate the fields and functionality on the forms.

In some of the forms we were retrieving fields from related entities but to speed up the form the customer was happy to use lookups and click those for more information if it was needed.   This saved OData queries

When the carrot of improved performance is offered, end users are often more willing to move functionality from Must Have status to Nice to Have or not needed.  These changes can remove some of the code triggered on the Javascript onload.

Improvements can come from moving fields and subgrids into unexpanded tabs where the loading can be delayed until the tab is expanded

subgrids

Subgrids take time to load and the more subgrids you have on a form the longer it will take to load.

Form load time can be saved by putting the grid in an unexpanded tab, this will not load the grid on the form load but will load it when the user expands the tab.

Remove unwanted/unused fields

CRM Developers have to refactor their code then I think CRM Forms and entities should be refactored to remove the noise of unused fields.

There is a great tool to help you with this called CRM Data Detective it will show you what fields are being used/not used by seeing if any values were written to those fields.  The CRM Data Detective is a free tool which works with CRM 2011 and CRM 2013.

OData queries

OData queries are a great way to retrieve data from related fields but if a form has a lot of them they can slow it down.  If you use OData queries makes sure they are as lean as possible by filtering them.  Below are some tips for using OData queries

• ALWAYS filter your OData queries to retrieve only the fields you need
• Group OData queries to avoid multiple OData queries
• Cache the return data in a Javascript variable if different functions need the same data

Code Logic

Often the first step in writing code is getting it to work, so I can forgive initial code to look ugly and speed along like a one legged tortoise.

It’s unforgivable for the CRM developer not to refactor this code.  CRM developer should keep their standards high and not slack off adding dodgy code but this is very frustrating to other team members who will then have to dry and debug it and find it very difficult.

Unrefactored code creates technical debt and this will have to be paid at some point.  You don’t save time but putting in sloppy code, you cost the team more time later when debugging and extending that code will take a lot longer than it would if the code had been refactored.

How to avoid slow code

Peer reviews

Developers code improves by about 70 percent when they know someone else is going to review it (a statistic I just made up but 100 percent true). I’m not saying Developers can be lazy but often time restrictions and a promise tidy the code up later (which rarely happens) will allow some basic poor coding to slip in.

If a developer knows someone is going to scrutinise their code (or even the threat it) the developer finds time to refactor the code because if hey don’t they know it will be looked at and then told to tidy it up.

Peer reviews can be seen by some developers as a bad thing but it’s a good way to learn new programming techniques and best practices.  The ultimate benefit is having more eyes look at some code will stop a lot of bugs sneaking in and keep coding standards high, both of which will benefit the project in the long term.

This article has Developer’s guide to peer reviews highlights some of the benefits

• A different perspective. “Another set of eyes” adds objectivity. Similar to the reason for separating your coding and testing teams, peer reviews provide the distance needed to recognize problems.
• The ability to assess and accelerate progress. Is the team producing the needed output at the needed rate?
• Pride/reward. Recognition of coding prowess is a significant reward for many programmers.
• Project/module familiarity. Everyone involved in the review becomes more familiar with the project and the module.
• Less rework. Do it right the first time. Changes cost more later in the life cycle. The peer review process catches many errors before they go to production.
• Fewer bugs. It’s better to discover your own problems than to have someone (like a user) point them out to you. For the same reason, you may find that many bugs will be eliminated before the code comes to be reviewed by peers.
• Improved communication. More opportunities for interaction tend to lead the team toward improved communication.
• Team cohesiveness. Working together helps draw team members closer. It also provides a brief respite from the isolation that coding often brings.

Coding standards

Companies should coding standards, otherwise the worms are well and truely let out of the can and once those worms are out it’s very hard to get them back in.

Creating coding standards lets the developers know they are expected to code to a standard and there is no excuse for the developers not knowing what the standards are.

Coding standards will also resolve conflicts of style because everyone is expected to code to the agreed company standards.

When I was a Java developer I use to love have my bracket on an If statement on the next line

IF

{

}

when Microsoft and other standards have it like this

IF {

}

This is a minor point but when you are looking at 1000’s of lines of code it can make a difference, particularly when it’s not just this small change but lots of others.  Suddenly you are in a situation where one persons code is formatted (with different variable names) completely differently making it harder to read, understand, debug and extend the code later.

Coding standards documents are great gift to new developers starting (and then constantly referring them to :-)) because it highlights how they are expected to write code and potential feedback from peer reviews.

Coding standards will help create code consistency not only between different developers but also between projects, which makes it easier for developers to switch between projects.

Common Libraries

CRM makes it easy for CRM Developers to write the same piece of code in every Javascript file on every entity and the same code in multiple plugins.  Code duplication is bad practise, it increases not only the possibility of bugs (e.g. updating code in one place but not the other duplicates) but also increases time to fix a bug because you have to make the change in all the places the code is.

Stop writing the same function in each javascript file for each different form.

Create common libraries then it only has to be written once (and well) and then used again and again.

The advantage of writing code once and used in many places is easy to maintain, debug and extend and you only need to test it once.

if there is a bug you only have to fix it one place.

Sometimes it takes more time to generalize a method but in the long run it will save you time.

Why have I put this in a blog about form loading, if you optimize this code it will save time in all the CRM forms it’s used.

Always filter queries

There is no excuse for getting all the fields back in a query because you never use all the fields and the CRM developer is just being lazy.

There are lots of retrieve code in CRM, it can be found in plugins and Javascript but make sure the developers filter, always.

Split loading times

Evalulate if you need all the data and checks to be done on the form load.  It’s possible you could move some of the functionality to

• Unexpanded tabs
• Onchange events

Unexpanded Tabs

You can put a subgrid in an unexpanded tab and load the subgrid when the tab is expanded.  If you have optionsets or other fields which need to be calculated using retrieved fields then these can also be triggered on expanding the tab. This saves time from the form loading.

This would also save time if the tabs included Maps and iFrames.

OnChange Events

For some functionality it might be possible to delay some code and remove it from the form onload to a field onChange event.

Javascript files

You can minify Javascript files which shrinks the size of the file and load quicker

There may be some performance gains from combining 2 or more Javascript files into one Javascript file

Plugins/Workflows

Similar to moving Javascript functionality from OnLoad to OnChange events it maybe possible to move some functionality to Workflows and plugins on the save of a record.  In general plugins run faster than Javascript.

pictures used

running fast, watering flow, turtle power. finding picture, picture slow down, coding standards

Advertisement

Do you fix or a restore a Confused CRM Database?

Very odd situation occurred recently and left the poor CRM database in a state of confusion.

it started yesterday when I was getting an invalidCastingException when a plugin was running.

Unexpected exception from plug-in (Execute):  System.InvalidCastException: Unable to cast object of type ‘Microsoft.Xrm.Sdk.Entity’ to type ‘ActivityParty’

I couldn’t fathom why I was getting this error because I could walk through the code in a console app, calling the same business logic/class. I could see the entities file and it had the ActivityParty included in it, what the heck was the problem

The error above was happening yesterday, I came in today thinking about the error and cause must be linked to the entities file because this is used when creating and using early bound classes.  If I refresh the entities.class it should work.

 

crmsvcutil.exe

 

I had been running the crmsvcutil.exe but I hadn’t noticed it had been erroring, instead of refreshing the entities.class.

So today I started looking at the

I started when I was trying to build the early bound entities.class, I was getting an error

 

Exiting program with exception: Mapping failed for Microsoft.Crm.Metadata.EntityMetadata property Attributes with exception Microsoft.Crm.CrmException: Mapping failed for Microsoft.Crm.Metadata.AttributeMetadata property Description with exception System.ArgumentException: An item with the same key has already been added.

The only previous time I have seen duplicate, same key error is when you try and import a solution with a field which was deleted and recreated as a different type.

This error returns very little from the internet (never a good sign) but the one article I found which seemed to make sense was from CRM MVP David Jennaway

http://mscrmuk.blogspot.de/2008/04/crm-40-upgrade-error-item-with-same-key.htmlHe found when a

user had upgraded, the upgrade process had somehow created two metadata fields of the same type. It has an interesting script to find duplicates

Nothing is working

Developers can sometimes only access CRM from the SDK they forget to check things on the CRM front end e.g. the CRM Web application.

I went into CRM and found I couldn’t publish any customizations and in fact all the plugins were not working.

We quickly reached the opinion a server reboot might fix the problems and restarted it with the reason (Hosks big problem)

unfortunately this did not resolve the problem and what we think happened was CRM let us add two fields with the same name.  There were some modifications to one of the entities in the afternoon, which we think is probably the culprit

CRM had now decided to lock up good and proper, we could export any solutions to try and edit the

What to do next

It was a DEV solution were unmanaged.  This means deleting the solution won’t do anything to the customizations. If they were managed solutions deleting them would also remove the customizations (and the dodgy double field) but it would also remove all the data (hmm it might take a while to create/import all that data)

We couldn’t import a solution over the top, wait I will rephrase that, we could but importing solutions are an additive, which means they only add things, they will not remove them.

CRM 2013 – Understanding Solutions and how they work

Restore or Fix (Stick or Twist)

Sometimes you have to make a decision to rollback to a previous database backup.  When you add the time it would take to investigate and fix the current CRM solution/database compared to rolling back to a daily backup from yesterday or maybe the day before.

In this calculation you also need to think about how many people are working on the project because during investigation it means other developers had to stop.  We think we might have known the cause but fixing it looked like tampering with the database directly.  This isn’t a great solution because we might just be kicking the problems down the road because we couldn’t be sure (if we worked out how to fix the problem) when fixing the problem we were creating more problems which we would find later.

Sometimes it’s better to lose one day of customizations versus weeks/months of potentially corrupt customizations

This is also highlights the importance of checking in your code changes into source control.  Most of the changes I have made during the last couple of days are all held in source control, so I just need to redeploy.

Changes to CRM customizations are a bit more tricky e.g.

• Entity changes
• fields
• views

These can only really be backed up by exporting the solutions they are held in, on a daily basis, although they are of course also backed up in the database itself.  It is possible to export solutions programatically, so you could create an application which you could setup to a windows daily task (although I haven’t yet seen anyone do this)

The morale of the story is, make sure you put your changes in source control because the server could get corrupted or your own computer could crash or get lost and then you could be in real trouble

picture confused from here

picture wall from here

beach picture 

stick or twist picture

Use GetValueOrDefault in early bound plugins and say goodbye to catching nulls

When writing plugin code CRM has a very frustrating feature which allows it’s fields to be null.  Note I said this was a feature not a bug 🙂

I believe this was was one of the main reasons in CRM 4 it had it’s own set of variables CrmDateTime, CrmDecimal, CrmFloat, CrmMoney, CrmNumber.  These could hold a null as well as value (I think, it has been a while since I did any CRM 4 development).

Where this really niggles me is when I can’t pass anearly bound field  into a method because the method takes a bool but the early bound field is a ?bool, which basically means it’s a bool that can be null.

So how do we code around this

Late bound

So in previous blog post I have talked about using Entity.GetAttributeValue instead of Entity.Contains

using GetAttributeValue is great because it means I could take all the entity.contains code below

if (entity.Contains("salutation"))
{
     title = ( string)entity.Attributes["salutation"];
}

The GetAttributeValue returns either a null or a default value.  This is great for bools, ints etc because instead of null I now get false or 0;

here is the list of default values

Type	Return
Numerical (int, decimal, double)	0
Boolean	false
DateTime	DateTime.MinValue
Guid	Guid.Empty

For other variables it will return a null.

Early bound

What if you are using early bound classes and don’t want to add the potential risk of putting in syntax errors in your plugin code by using late bound code.

What are the benefits of early bound code

• Readability
• No Syntax errors at runtime
• casting errors are found at compile time
• Easier to read, debug, maintain and extend

The main reason I like early bound classes is because it’s easier to understand and I don’t have to keep looking up and typing the name of the CRM fields.  It also moves most of the errors into compile time (syntax and casting errors), which means it’s less likely these errors will get through to the customers.

GetValueOrDefault to the rescue

If you want to convert the an early bound field into simple attribute then you should use GetValueOrDefault, which

The reason it’s a good idea to use this is because if you casted it to a bool or int you are in danger of creating an error because you cannot set a bool to be null!

The GetValueOrDefault will return a value (the default value for the variable type) even when the HasValue for the field is false (e.g. it’s null). Using the GetValueOrDefault means you don’t catch the error but you do have to be prepared for receiving a default value and code with that in mind.

So here it is in action

if (entity.Contains("salutation"))
bool credit = entity. CreditOnHold.GetValueOrDefault ();
entity. CreditOnHold = true ; credit = entity. CreditOnHold.GetValueOrDefault ();

 

 

The MSDN article if you want more, to be honest there isn’t much more to say about it.

FetchXML reports do not trigger RetrieveMultiple plugins in CRM 2011

 

It is by going down into the abyss that we recover the treasures of life. Where you stumble, there lies your treasure.
Joseph Campbell

The CRM forums are a great resource for learning interesting solutions and problems in Microsoft Dynamics CRM

I read this interesting gotcha/CRM Quirk this week about a CRM developer who had written a plugin which was triggered on the RetrieveMultiple event on his custom entity.  The user was wondering why his plugin was not being triggered when the user ran a report using FetchXML.

RetrieveMultiple Plugins, What, Why and How?

So you can have plugins which trigger on RetrieveMultiple for an entity.  Some readers are thinking why would you want to do that?

Reasons like

• Translate data held in the CRM database
• Exclude data from RetrieveMultiple queries
• Log RetrieveMultiple requests
• Calculate some fields in the retrieve from other entities
• Decrypting encrypted fields

There are lots of other reasons, but they give you an idea.  The user wouldn’t even know a plugin had run because they would just receive the results as normal.

The retrieve multiple is triggered in CRM when the user

• Runs an advanced find
• User clicks on a view (e.g. active accounts)
• Quick search
• Associated views
• Plugins doing retrieve multiples
• Console apps doing retrieve multiples

 

What about reports and FetchXML

So why don’t reports and FetchXML trigger the RetrieveMultiple plugin

The answer was explained in this forum post answer by CRM legend David Jennaway who is a CRM MVP, when I was starting in CRM development David had a fantastic CRM blog which I followed avidly and still do, but but unfortunately for the CRM community David rarely writes any blog posts these days (2 post in 2014).

Reports do not trigger plugins. SQL reports bypass CRM completely, and Fetch reports also bypass the plugin pipeline

 He also add this answer

Just to clarify the last post, in CRM 2011 fetch queries are submitted via the RetrieveMultiple message, whereas in CRM 4 they were submitted via the Fetch message and (bizarrely) you could handle them in plugins on the Execute message

This logically makes sense because SQL reports definitely would hit the plugin pipeline and I’m guessing FetchXML when run from reports must be run using a slightly different mechanism (I have no knowledge of this)

I’m not sure if this is the same in CRM 2013 but I’m guessing it’s probably going to be the same because I don’t think there have been changes in those areas of functionality.

You are not the end user

This post is about CRM Developers and end users, remember, it’s not a competition.

“ …pay attention to what users do, not what they say”

— Jakob Nielsen

“ You cannot NOT have a user experience”

— Lou Carbone

Why did Google create 2 separate email programs

I read an interesting article this week on techcrunch which prompted me to write this blog

Why Did Google Decide To Split Inbox From Gmail?

The article is about the new email app created by Google called inbox.  I have used inbox and it’s very different from Gmail and this confused a lot of people, why have google created two email apps?

Some people at google looked at how people used email and designed an emailed map with the majority of people in mind.  What is interesting about the article is the internal derision of the Inbox app and redesign of the email app. The article has a key section

In response, the head of the Gmail design team made a presentation entitled “You Are Not the User.” If you were not lucky enough to witness the carnage in person you could view its archived version on the internal Google+.

The presentation detailed the reasons behind every decision the design/product team made showing gobs of usability data supporting the decisions to remove advanced features that the overwhelming majority of Gmail users were never using. These features, it was argued, were unnecessarily complicating the user interface when most people just wanted a simple email client.

The top comment to this article is

“You are not the user” should be tattooed on the insides of the eyelids of every techie everywhere.

Why is this important to CRM developers

The question is why is important for a CRM Developer to know they are not the end user.  The first point is Developers should heed this quote from Ken Becker

“ The user is NOT a lower life form

— Ken Becker

CRM Developers can have a love/hate relationship with end users.  CRM Developers need end users because they are the ones who need the new CRM system the developers have been tasked to work on.  End users can make the CRM Developers world difficult by

• Poorly defined requirements
• Not very IT illiterate, making it difficult to explain functionality in CRM/customizations
• Requirements massively out of scope
• Finding bugs with the previous solution/code
• The source of Change requests

The end-user can cause the CRM Developer some headaches, but importantly they are key to providing a great a solution because they are users in the old software and will be the end users in the new CRM solution the CRM developer will be creating.  The more effectively you can capture the requirements from the end users the better the solution is going to be.

The point I am trying to make is

A CRM Developer needs to create a customization and functionality to create an effective, easy to use CRM system which offers the end-user the functionality needed to do their job.

or

The CRM Developer needs to collaborate with the end-user, not deliver what he thinks the end-user wants or should want

CRM project/solutions

Two of the common problems which can go wrong in a CRM project

1. The solution doesn’t meet the end users requirements
2. The solution is not easy to use – BAD User interface (great example of a user interface designed by a programmer, beautifully functional)

If you want to read more reasons a CRM project could go wrong, read my previous article – 13 signs your CRM project is doomed

Both of the reason’s specified above are usually caused by not capturing the requirements of the end users effectively or the developer ignoring the requirements and delivering the customizations the CRM Developer would want to use.

CRM Developers and end user interaction

CRM Developers will interact with the end user either directly gathering requirements or through technical spec/bugs raised

Potential problems which can arise when CRM Developers interact with end users

When creating customizations CRM Developers can make incorrect assumptions on the functionality required

• CRM Developers know and use CRM terminology which can confuse the end-user who is not as CRM literate
• CRM Developers assume users have the same level of knowledge of IT and CRM
• CRM developers forget their gaps in product knowledge/system knowledge/CRM Developer makes incorrect assumptions about requirements and how the functionality should work
• End users may forget/miss parts of the requirements
• When CRM developers see/hear a problem they instantly start thinking about the technical solution but the first goal should
• Customization bias from CRM developers can create customizations driven by the developer rather than the requirement

I shall discuss the above in a little more detail

CRM Developers know and use CRM terminology which can confuse the end user who is not as CRM literate

Microsoft Dynamics CRM uses lots of technical terminology entities, forms, fields, views, records, events, plugins, Business rules, workflows.  Microsoft Dynamics CRM also has out of the box data like Accounts, contacts, leads, orders, quotes, products, incidents etc etc, which I will refer to as business terminology

Both the technical terminology and business terminology use terms which end users will have probably come into contact with before, but could mean something different in CRM, this can cause confusion for end users.

It’s difficult for CRM Developers remember the time when they all the terminology in CRM was new to them and because they use CRM every day, talk to other developers in those terms it’s difficult to appreciate how confusing this can be to end users.

I have had numerous discussions with customers discussing how Leads are used in CRM e.g. Unqualified (no verified contact or account data) potential leads.  Outside of CRM (and in CRM) leads can be used in lots of different ways and mean different things to different companies.

• Some companies don’t use leads
• Use leads for marketing lists brought
• Leads have account/contact data in

The end users understanding of business terminology can add a layer of confusion to discussions and requirements which can lead to misunderstandings and misleading assumptions.

This is a difficult one to resolve but good CRM developers will have this in mind before any interaction with end users and should be aware this can happen.  One of the best tools to help combat this is to show the end-user CRM and it’s functionality as soon as possible, this will help give them a context and a visual reference.

CRM Developers assume users have the same level of knowledge of IT and CRM

I could have put in brackets (thinks everyone should have a high level of IT/CRM knowledge).  This point is one of the reasons CRM developers can look down on users abilities.  The end goal of a CRM project or writing CRM customizations is to create a better system/process using Microsoft Dynamics CRM and you have to work with the end-user not against them.

I view IT literacy a bit like language skills (e.g. Talking with another language) if you don’t need to communicate in another language then you don’t.  End users have the IT skills needed to do their job and if they don’t need to use a computer much then they don’t.

Assumptions are a lazy way to increase the likelihood of a project failure.

CRM developers forget their gaps in product knowledge/system knowledge

The other side of the coin mention in the previous section.  CRM developers are prone to assume how things should work, instead of clarifying and asking the end user how the process works.  When writing CRM customizations CRM developers will come to lots of forks in the customization where the code has to do update values.  Lots of CRM developers have chosen to do what they think the code should do or “what makes sense” but often this can turn out not to be how the customer does things.

When decisions/forks appear in customizations these needs to be clarified with the end-user/business analyst to ensure the correct decision is made.  When working with end users it pays to keep in mind

You are not the end-user

When I think about CRM developers making assumptions, for some reason I always think about the end of the ham story

The End of the Ham

A young woman was preparing a ham dinner. After she cut off the end of the ham, she placed it in a pan for baking.

Her friend asked her,”Why did you cut off the end of the ham”?

And she replied ,“I really don’t know but my mother always did, so I thought you were supposed to.”

Later when talking to her mother she asked her why she cut off the end of the ham before baking it, and her mother replied, “I really don’t know, but that’s the way my mom always did it.”

A few weeks later while visiting her grandmother, the young woman asked, “Grandma, why is it that you cut off the end of a ham before you bake it?”

Her grandmother replied, “Well dear, otherwise it would never fit into my baking pan.”

This story succinctly summarizes the sneaky, creeping consequences of assumptions which can end up having major consequences to actions in the story but functionality/custmomizations in the world of CRM.

End users may forget/miss parts of the requirements

Capturing requirements is a tricky process and there is no way to guarantee you get them all, some of the reasons are

• Different users do things in a different way using different criteria/goals
• End users are not used to writing down their processes and forget things
• Business analyst/CRM Consultants might not ask the correct questions or have access to the right people

This can lead to requirement documents which have some of the requirements missing or some holes.  Don’t make assumptions and fill in the holes yourself, but clarify the requirements with the end-user.

When CRM developers see/hear a problem they instantly start thinking about the technical solution but the first goal should

CRM developers often enjoying coding and the process of crafting a customization.  Their first instinct when hearing/seeing a technical spec or requirements is to start coding or thinking about the technical solution.

Listen to Master Yoda

It’s good practice to get as much information from the end-user as you can about a requirement before you start thinking about creating any solutions.

Get the design correct (as can be) before you start thinking of solutions

It’s a lot easier to change customizations before they are written than it is once you have already started creating the customization/code, with this in mind, try to make sure you get all the requirements and find any problems before you start developing.

Customization bias from CRM developers can create customizations driven by the developer rather than the requirement

Customizations can sometimes be made to fit CRM developers requirements because

• The developer wants to try out some new functionality/3rd party tool
• The developer is comfortable with one type of customization and will choose that method (e.g. When you have a hammer, everything looks like a nail)
• The developer can bully the user

This section is really about the developer leading the user down the customization the developer wants to write and not the customization best suited to the requirement.  CRM development is a collaboration with the end-user to create a system which will help them do their job.

The driver behind this problem is the CRM developer is using personal criteria for choosing the customizations rather than doing what’s best for the end-user/customer.

You are not the end user

You are not end user is something you should keep in your mind when interacting with end users or end-user requirements via technical specs/functionality requirements.

The best CRM solutions and customizations are created collaborating with the end users to create a solution which gives them the functionality needed to effectively do their job.  The only way to do this is do capture all their requirements.

There is a great quote from Gerald Weinberg on a great blog post No Matter What They Tell You, It’s a People Problem

“no matter what they tell you, it’s always a people problem.”

The quote was talking about programming problems, but this applies to CRM development.  A lot of failed CRM projects fail due to projects not delivering the functionality the customer needs to do their job.  Two major sources of problems in turning requirements into Microsoft Dynamics solution

• Not getting the full requirements
• Not turning the requirements into functionality/CRM solution

At the heart of the those two problems are people problems, which can steer the project down the wrong path.  Once the code and customizations have been written/created it’s harder, more time-consuming and messier to fix those problems.

The emphasis is on the CRM developer/consultant to try to avoid these problems by  realising they are not the end-user, they have different knowledge, skills and you have to manage yourself and the end-user.

pictures

Who are you from here

google techcrunch here

CRM 2013 – MB2-703 – Business Rules Exam Test Questions

Here are the Hosk’s Multiple choice questions for Business Rules for the MB2-703 – CRM 2013 Customization and configuration certification

For the other Exam summaries, Exam notes and other test questions click the link below

https://crmbusiness.wordpress.com/mb2-703-crm-2013-customization-and-configuration-certification/

These are exam questions which are multiple choice.  They are in similar style to the real exam questions, although of course they are aren’t the same but they should test your knowledge on Business Rules to help you prepare for the exam

 

CRM 2011/CRM2013 – Investigating a casting error with ActivityParty

I was getting an odd error whilst casting a CRM dynamics spell, no it wasn’t a spell it was casting an ActivityParty to an Entity (although some people might say ActivityParty is rather magic or at least very complex)

Talking of bad casting, this would have been terrible

Burt Reynolds as Han Solo in Star Wars

 

Casting Error

error in getproduct Unable to cast object of type ‘Microsoft.Xrm.Sdk.Entity’ to type ‘Hosk.Dynamics.Entities.ActivityParty‘.

It was on a plugin on the ServiceAppointment entity.  What I was trying to do, is when the user updated the ServiceAppointment Entity.  I would check the resources (which can either be a user or equipment).

If it’s equipment, I can then look up the value of a product and show a few product details on the ServiceAppointment record.

I was using early bound records but when I tried to get the serviceAppointment.Resources I got a casting error.

The odd thing was I could step through the code calling it from a console app but I was getting an error when the plugin was running.

The error was so odd I had to get someone to look at my code to check if I was doing a schoolboy error.

I knew it wasn’t a problem the Resources field was empty because I was checking it, it definitely had a value, the only problem was I got an error every time I tried to get it.

 

Finding the Error

 

To find the error, first I changed the code to throw an InvalidPluginException and this then puts out my tracing lines in the plugin.  Which is good because I recently blogged about mocking up tracing and have been using tracing whilst debugging some plugins

CRM 2011 – You can now add tracing to your plugins

CRM 2011/2013 – using ITracingServicing and Mocking it up

So using the tracing line I could work out where the problem was occuring and see the casting error – FACE to FACE with the beast

The question was what value was in the resources field?

Debugging with the Plugin Registration tool

I used the plugin registration tool to set profiling on my ServiceAppointment then this spits out a file with all the serialized field values.  You can then use this file to rerun the code in the Plugin Registration tool and attach the debugger to the plugin registration tool.

This is a lot better method than remote debugging because remote debugging stops the CRM server from running, so can hold up other developers/testers on that CRM instance

Here is a good article if you need to use the Plugin Registration tool for some debugging

http://blog.sonomapartners.com/2013/03/debugging-crm-2011-plugins-with-the-plugin-profiler.html

I finally got to the ServiceAppointment.Resources, instead of a List of ActivityParty I found it had this

This was the line causing the casting error

 List<ActivityParty> resources = (List<ActivityParty >)serviceAppointment.Resources.ToList();

The error it was throwing was this

System.Linq.Enumerable.CastIterator<Ntt.Dynamics.Crm.HondaPLCC.Entities.ActivityParty>

Now that I was stepping through the code, I could have a look at the Entity attribute and retrieve the Resources value using Late bound code and this worked fine.

 

var entities = ((EntityCollection)  entity[serviceAppointment.GetAttributeName(x => x.Resources)]).Entities; 
                ActivityParty resources = null; 

                if (entities. Any())
                {
                    var firstOrDefault = entities. FirstOrDefault();
                    if (firstOrDefault! = null)
                        resources = firstOrDefault. ToEntity< ActivityParty>();}
                

 

So the error was linked to the early bound class and there is some problem with it.

I had already spent way to much time writing this simple plugin, so I just used the work around getting the value from the Entity class and used late bound methods to get the data.

I did think about correctly the entities file but this would have been overwritten when anybody refreshed the early bound entities.  So I decided to go with the code above and retrieve the resources using the early bound entity.

 

Plugin Registration tool

Using the plugin registration tool is an excellent way to debug plugins on a server but it can be done on your local machine, so it doesn’t impact the other developers working on the project.

All CRM Developers should know about the Plugin Registration tool, I mention this because some new CRM Developers have never used the Plugin Registration tool because they only deploy plugins using the CRM Developer toolkit.

So dust off the plugin registration tool and go a bit old school on your CRM.

CRM Developers don’t know what they don’t know!

There must be a happy medium somewhere between being totally informed and blissfully unaware.

Doug Larson

Unaware

I read a fantastic article today on boingboing with the title

Why we are unaware that we lack the skill to tell how unskilled and unaware we are

It starts with a great quote

Each one of us has a relationship with our own ignorance, a dishonest, complicated relationship, and that dishonesty keeps us sane, happy, and willing to get out of bed in the morning. By David McRaney

 

The article mentions Nescience, so I thought I would use it but first lets understand what it means

nescience

1.

Lack of knowledge or awareness; ignorance.

How is this relevant to CRM Developers

When you start out as a Developer and CRM developer you don’t know much and the main problem is

You don’t know what you don’t know

or

No I didn’t type that line twice or incorrectly, what I’m trying to say is junior CRM developers/ people starting CRM development have a lot to learn.

• Knowledge of all CRM customization techniques
• CRM Development Best Practices
• Writing well structured code
• Using the right Customization for the right job
• CRM Infrastructure (e.g. Servers and components)
• Testing and debugging
• Converting customer requirements into solutions
• Learning new CRM functionality
• Lots and lots of other things

Getting it working

The first stage of improving is getting hands on experience with CRM development/customization.  At the initial stage this is where you know the least and don’t know enough to realise how much you have to learn.

The initial steps are getting practical examples to work e.g. Getting your hands dirty by creating entities, fields, views.

• Creating new entities/fields/views/relationships and other GUI customizations
• Creating workflows/business rules
• Writing plugins/custom workflows
• Integrating WCF webservices with CRM
• Writing CRM Applications

This article about Knowing and Not Knowing is a great article which has a great visual and quote

The important realisation is understanding the areas which you have to learn and making a plan to learn about them

The key stage is moving from

Don’t know what you don’t know

To

Know that you don’t know

 

Doing it right

The CRM developer will at some point have got experience doing most of the different CRM customization types and will be comfortable with the process of creating them all.

The good CRM developers will start to move ahead of the average/bad CRM developers because they will start to produce good quality customizations and write plugins/custom workflows/WCF which are well structured/readable.  These customizations are easy to read/understand and easier to maintain and modify

Good CRM developers have a detailed knowledge of how CRM customizations work and the underlying CRM infrastructure.

 

How to learn what you don’t know

I started off this post by mentioning most people lack the skill to tell how unskilled and unaware they are

Sometimes this is because of ignorance and you simply don’t know what you don’t know.

There are actions you can take to learn about CRM Development

 

Read blogs

great examples of good CRM custmoizations, written by great CRM Developers.  Good blog posts not only show you new functionality but also teach you the logic behind it and throw in a few best practices as well.

Example of new functionality you haven’t used yet.

Read Books

I recently wrote a blog about some great books for developers

Recommended Books for Developers

Books offer something different from blogs and articles.  They are focused and in depth look at a subject.

 

Correct Attitude

At some point the good CRM developer realizes he has a whole bunch of stuff he should learn about, but the list is big and the only way to tackle it is to constantly keep learning and improving on a daily basis.

Learning to be a better CRM Developer is like eating an elephant, one bite a time.

Don’t be afraid to question what you don’t understand, Ask fellow CRM developers at work or raise questions on forums.

 

CRM Customizations – Doing

Writing customizations will help you learn and improve.  Experience is a key for CRM developer, you will get to a level where you are comfortable with the process of creating CRM customizations.

Once you can easily create customizations the next stage is creating great customizations.  Customizations which will be easy to understand, maintain, extend and debug.

 

Learn from your colleagues – Peer Reviews

You can talk to fellow CRM developers about CRM development, the best way to do things, tips on creating customizations, mistakes to avoid.

Having your code peer reviewed is a great way to get feedback on your customizations

 

Learn from your mistakes

You will create bugs, you will choose customizations which don’t work because they are technically not correct.  You will have peer reviews where your code gets criticized.

Don’t take peer reviews personally, but learn from them, improve and don’t make the same mistakes in the future.  I personally find mistakes stick more vividly in the memory

 

Learn from previous coding – the good and the bad

You will have to work on projects written by other CRM developers.  Use this as an opportunity to find out the good customizations and the bad customizations.  Focus on why code is good and why the code is bad.

You take the good things and remove the bad things from your own customizations/coding.

Learn in public

A great way to learn is by writing blog posts, answering forum questions and learning with feedback and interaction from other CRM developers.

It’s even more fun learning with other people

 

known unknowns

I will finish with this classic quote from Donald Rumsfeld

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.

Donald Rumsfeld

CRM 2013 Tool Review – Solution History for CRM 2013

This blog post is about history, so I shall with this quote

Those who do not remember the past are condemned to repeat it.

George Santayana

 

Guido tweeted a link to this tool and the title interested me

Solution History for CRM 2013 click the link below to download it

https://solutionhistory.codeplex.com/

What does it do

The tool shows a historical view of all the solutions and their components which have been imported into a CRM organization

How does it work

It’s an add in into the excellent XRMToolBox written by Tanguy and others.  You get the latest version of the XRMToolbox and drop in the dll.  This will then load up a new tab in the toolbox.

You may not know but when ever you import a solution into CRM it creates a importJob record

I have written a blog post on follow the progress of a solution import by doing a query on the importjob table.

CRM 2011/2013 – Progress of solution import

 

The CRM Solution History tool doesn’t show the progress of a job importing, I just thought I would add that as a bonus topic on the subject of importJob.  The CRM Solution Hisotory tool displays the contents of importjob table and split the data up into nice GUI format, which shows you when solutions were important, the components inside the solutions and who imported them.

below is the screen shot from codeplex

Why is it useful

In most/all CRM projects you will have a DEV, TEST, PROD environments as a minimum, I have  worked on projects where we use to have 3 DEV environments, 2 Test environments, PREPROD and PROD.

The challenge when you have lots of environments is keeping track of what

Who and when imported solutions

The tool shows you who and when a solution was important. This can be useful to allow you to work out if certain changes were included in the solution and if one solution has overwritten another solution.

When working on CRM projects with multiple developers, it’s important to be able to see what developers have been importing solutions

What files were included in the solutions

The tools show what type of components were included in a solution and what individual files.

Unmanaged solutions

Unmanaged solutions can be used to sneak across some changes between Dev and Test were forgetten.  This can either happen in a temporary unmanaged solution which is subsequently deleted or it can be the same umanaged solution but the number is incremented.

It also shows unmanaged solutions, which can often be used to sneak things across you forgot or for quick fixes.

Fragmented CRM environments

Different systems can get fragmented and it can be hard to know what components have been imported, who imported them and when they were imported (overwriting each other)

Solution hiding

Sometimes the same solution can be imported multiple times without the version number being bumped up.  This can make it difficult to keep track of different environments (this usually happens internally between Dev and Test).  This tool will allow you to see the date of the last import and give you an understanding what was included in the solution and if certain changes were brought across.

Potential Uses of the Tool

Production Deployment

It’s always a nervous time when you press the “Import” button and kick off a solution import on a production server and the last thing you want to do is have to do things wrong and have to import an older solution.

This tool would give you a method to check what has been deployed before and compare it with other servers.

DEV and TEST SYNC

In development environments with multiple developers working, it can be hard to keep track of what has been imported, particularly between the DEV environment Test environment.

In DEV you can also see that sometimes things get forgotten and get brought across as an unmanaged solution or another mini solution.

This tool would help keeping track of what has been brought across and if your latest code has made it across.

It would also help you to see what components you need to include in the managed solution to production and avoid missing out a key component.

Does it work with CRM 2011

The importJob table hasn’t changed (to my knowledge) from CRM 2011 to CRM 2013 so luckily the tool works fine with CRM 2011 (well it worked with my CRM org).

So the good news is YES it does work with CRM 2011 and it will should work with CRM 2015 I’m guessing.

More Tools

If you want more CRM tools why not check out my list

https://crmbusiness.wordpress.com/hosks-microsoft-dynamic-crm-development/crm-2013-tools/

Recommended Books for Developers

A while ago Dave Berry tweeted an article on the evolution of the windows operating system

http://arstechnica.com/features/2012/10/windows-8-and-winrt-everything-old-is-new-again/

It was an interesting read and it reminded me when I started out as a Java Developer

I stumbled upon a list of great books for developers held on coding horror

http://blog.codinghorror.com/recommended-reading-for-developers/

Looking at the books, most of them are not focused just on just programming but also focus on the areas and processes around developing (all the bits Developers don’t like!)

His life changing book – Code Complete 2, is a monster at over 800 pages.

The books on the list are classic books, which often appear on people’s lists of Programming books I wish I read at the start of my career.

This reminded me of reading the first mind blowing book I read about programming

Effective Java: Second Edition  by  Joshua Bloch

Effective Java to me is similar to the author’s introduction to Code Complete 2.  When I first purchased the book I was a keen Java developer trying to learn to be a better developer.  I remember reading the book and being amazed and the simplicity and genius of the coding advice (to put it into context I was a rubbish developer at the time)

I wrote quite a few articles on developing, for those of you who don’t know I was prolific Java blogger, my blog was called

A Funny Java Flavoured Look at the World

Here are a few of my more interesting Java programming related

The Learning Lifecycle of a Java Programmer

10 tips on writing reusable code

10 presentation tips for developers

My most popular blog post was, which has the solution to a common problem in Java it seems

trouble using | (pipe) with the String.split method

Head First Design patterns

The next book I read was Design patterns and once again it blew my mind that people could write code which was that good.

In some peoples eyes, it’s probably sacrilege to put this book here instead of GOF design pattern – Design patterns : elements of reusable object-oriented software.  I started reading that but I found it hard to understand the design patterns, where as Head First Design patterns wrote in a very accessible way, which at that moment I found easier to understand.

The next book I read was

The Pragmatic Programmer: From Journeyman to Master

I loved the pragamatic programmer and many of the pragmatic book series.  The advice they give is common sense and is similar to experienced developer guiding you to use the right practices.  I found this book very useful when I moved companies because it gave me a good set of programming practices and standards.

Finally in my Programming life changing books I read

Refactoring: Improving the Design of Existing Code

by Martin Fowler  (Author), Kent Beck  (Author), John Brant (Author), William Opdyke (Author), Don Roberts(Author)

Remembering back in the day

Looking at the list of books reminded me of when had the double whammy of learning about design patterns and great coding practices from the effective Java book.  I suddenly realized I had a lot to learn.  Back then I also amazed when I learnt of the term Refactoring, I just thought of it as tidying and improving.

It’s seem bizarre to think I was writing code without knowing some of these concepts, but this stage of learning comes to everyone.  You go from knowing nothing, learning a bit and thinking you are the bee’s knees and then something will happen and you realize you have a lot to learn and it’s going to take you a while to learn it.

This introductory section on coding horror, does a great job of summarizing this

I graduated from college in 1992, and entered the field of professional software development at that point, at least in terms of being paid to do so. I loved it, but I really had no idea what I was doing. I was a young, inexperienced developer working in small business, where there aren’t a lot of other developers to look to as mentors. Nor was the internet a factor; the internet didn’t really hit until ’95 for most people. I was living in Denver at the time, and I frequented the Tattered Cover, a great independent bookstore. Code Complete was originally published in May 1993; I stumbled across it while browsing the computer book section at the Tattered Cover sometime in 1994. I was floored. Here’s this entire book about becoming a professional software developer, written in this surprisingly friendly, humane voice. And it was backed by rational research and real data, not the typical developer “my brain is bigger than yours” chest-thumping.

I had found my muse. Reading Code Complete was a watershed event in my professional life. I read it three times in one week. It immediately became my Joy of Cooking. I didn’t even know it existed, but it showed me that if you loved food enough, it was possible to go from being a mere cook to a real chef.

One of the most striking and memorable things about Code Complete, even to this day, is that Coding Horror illustration in the sidebar. Every time I saw it on the page, I would chuckle. Not because of other people’s code, mind you. Because of my own code. That was the revelation. You’re an amateur developer until you realize that everything you write sucks.

Enthusiasm 

Reading the list of books and remember the positive effects some of the books had on me has made me want to read some of the books on the list I haven’t read.

In fact the author suggests if you haven’t read

The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition (2nd Edition)

Then shame on you.

So with those words ringing in my ears, I have decided to start The Mythical Man – Month and hopefully get round to the other books.

Thinking about the books on the list also reminded me there is more to development and particularly CRM development than just writing code, customization examples.

There are also the processes, procedures, best practices and all sorts of aspects to CRM development.  Thinking about the article I wrote earlier this week

13 signs your CRM project is doomed

The points are relevant to CRM developers and could save them lots pain but there isn’t much of it on coding/customization.

Books on CRM Development

It struck me as I was reading this list of books are there any books/sources which CRM Developers should read, which focus on CRM.

I would certainly recommend CRM Developers read some of the books on the list, many of the books are classics in the developer/programmer world.  Most of the books do not focus just on coding, but lots of different aspects of development.

There has really only been one programming book in CRM I can remember

Programming Microsoft Dynamics® CRM 4.0 (PRO-Developer) 

This book focuses on CRM 4, I don’t think it would be of much value for Developers today (unless you work for a company who has CRM 4) but it was very useful at the time.

This book looks like it might be useful, but I haven’t read it and just looked at the contents page.

Crm 2013 QuickStart

It’s written by CRM MVP’s and seems to focus more on developing and customizing rather than learning CRM.

CRM in the Field is also a pretty good book, with each chapter written by a different CRM MVP. It’s a bit like a longer version of tip of the day.

Most of the other books are focused towards people starting out using CRM rather than starting out developing for CRM.

Certifications

If you are starting out as a CRM Developer I would probably advise you to do the two Certifications

MB2-703: – Microsoft Dynamics CRM 2013 Customization & Configuration

If you want some study material for MB2-703, check out the resources here

MB2-701 – Extending Microsoft Dynamics CRM 2013

The customization and config certification will go through all the front end/gui customizations in CRM 2013 and give you a good knowledge of all the functionality.  If you are a CRM developer you will have probably used 80 percent of the functionality, so it will be a case of learning the new functionality you haven’t used yet.

The extending CRM 2013 certification will run through the basics of coding in CRM 2013.

The benefits of the certifications is you will get a knowledge of customizations you haven’t used and to pass the exam you will read, learn and repeat the exam criteria that it really sticks in your brain, so you instantly know things like

• Business rules are often referred to as portable business rules because business rules work in the browsers and CRM for tablets/phone applications (e.g. portable and everywhere)
• Custom solutions developed using future versions of Microsoft Dynamics CRM cannot be installed into earlier versions without first being ‘down-leveled’ to match the earlier version
• Only security roles which exist in the root business unit can be added to a solution file
• Access Team templates are enabled on an entity basis and you have to enable Access Teams on the entity in the communications and collaboration section

If you want to learn plugin development, Hosk CRM Dev has a couple of good playlists to help you get started and each video should have a blog post to go with it.

CRM Development Foundation Playlist

CRM 2013 Plugins Playlist – which features the following videos

CRM 2013 – Setting up Developer Toolkit for Microsoft Dynamics CRM

CRM 2013 – Understanding Solutions and how they work in CRM 2013

CRM 2013 – Create a simple plugin in CRM 2013 using the CRM Development Toolkit

CRM 2013 – Simple Plugin – Redeploying, improving and updating

CRM 2013 Plugin – Step by Step guide for a Post Account Create Plugin using the Developer Toolkit

YOUR LIFE CHANGING DEVELOPMENT BOOKS

If you have any classic programming/development books which had a big impact on you, please leave a comment