Unit tests are a vital part of emerging code design

Don’t limit yourself. Many people limit themselves to what they think they can do. You can go as far as your mind lets you. What you believe, remember, you can achieve.

Mary Kay Ash

When a developer writes code there is always an aspect of design to it.  Some developers spend more time on initial upfront design others like to start coding straight away.    The design and structure of code will emerge to a more efficient structure during the writing of the code, this process can be helped by writing units to assist the refactoring process.

Designing Code

I like to encourage developers to pause and think about the design of their code before they start writing code.  The benefits from developers thinking before coding

Think before you act

It’s quicker to change a design plan than design and code

This reminds me of the carpenter saying

measure twice and cut once

Thinking = planning

They are stopping to think about the task and creating a design is similar to looking at a map before you set off driving towards a destination.

• A brief design gives you a plan
• code designs emerge

You initial design will not be the finished design because designs are high level.  When you focus on individual parts of the design more questions and problems will come into view.

It’s like viewing something from far away, you see some general shapes and details.  The closer you get the more detail you see.

The role of unit testing in code design

Many people believe unit tests are written to catch bugs, it’s true writing unit tests could find some additional bugs you might have missed.

reusable tests for all developers.

It’s true unit testing has many benefits but here are the benefits to code design

Benefits of unit tests to code

• Refactoring
• Simpler code design
• Code Documentation

Refactoring

the main benefit I have experienced with unit tests recently is the ability to refactor my code and run tests against the code to make sure it still works and hasn’t broken anything.

Simpler code design

When code is written to be tested the code design is simplified so testing the code is easier.  It’s easier to test code which does one thing and adhere’s to the single responsibility principle.

Code Documentation

Unit tests provide a working documentation of code.  Allowing other developers to easy see what the code should do by the tests and run/debug the code using the unit tests

This article (which is very good) THE ACTUAL COST OF TESTS – PART I, adds  additional benefits to unit testing

• Code coverage
• bugs in tests

The goal of code (design)

The way most developers work is they aim to get the initial code working and aim to tidy the code later.

This approach is driven by two factors

• working code not matter how ugly can be shipped
• The developer has proven their logic is sound

Primary requirements of code

1. The primary requirement of code is deliver the functionality required

long term requirements of code

1. Readable (expressive names, logically structured)(
2. Modular/independent code to manage the effects of change (cohesive code, loose coupling)
3. Reuse
4. Easy to maintain/debug/enhance the code

The primary and short term goal of code can be achieved with minimal effort in the design of the code.  The code will deliver the required functionality but usually in spaghetti code with these characteristics

• Big classes and Methods
• classes\methods doing many things
• high complexity
• tight coupling

The above code is what I call Legacy code and developers have given up trying to keep this code manageable because the task of refactoring the project has become to big to contemplate.

An important aspect of refactoring is removing duplicate code, Martin Fowler has a great article on repetition.  Removing duplicate code has numerous benefits such as

• Create reusable classes/methods
• Reduces code
• Better designed code

Unit Tests and refactoring

If you code has unit tests it allows developers to refactor the code with confidence because they can easily test the code still works by running the unit tests.

Unit tests fit in with the way developers write code (get it working first) but without unit tests the refactoring and structuring the code for the long term is  considerable more difficult because the developer would need to manually test the code is working.

Code designs emerge during the coding process, developers see more efficient ways to structure code which usually involves removing duplicate code, writing smaller more efficient code.

Code design can be viewed in similar way to writing a novel.  Novels go through many drafts and revisions whilst the writer removes unneeded sections and poorly written sections.  Some novelists can change the core structure/characters of books during revisions as a byproduct of investing more time focused on the task.

Code design is a similar process because it’s easier to restructure a working code design and improve a current design than it is to create a perfect code design first time.

Problems and efficiencies of code only come into view when a developer moves closer to the problem.

The key points

Without unit tests developers won’t refactor code to deliver the long terms benefits of well designed code.

Code and projects where code is not refactored reach legacy code status very quickly.  The developers who have to manage and work legacy code will take longer and longer to interact with the code (debug/enhance/extend/understand).

The end result is not writing unit tests will cost you more time and developers jobs are less enjoyable

CRM 2015 SP1 – Turbo forms use asynchronous JavaScript web resource loading

I was reading this blog post from the CRM Lady

Turbo Forms in Microsoft Dynamics CRM 2015 (v7.1.0)

The article points to some good resource learn more about the new turbo forms.

The article from the CRM Team has a wealth of interesting information and lots of healthy warnings for you

Microsoft Dynamics CRM Online 2015 Update 1 – New Form Rendering Engine

What are the new turbo form

Microsoft have been doing lots of tinkering how the CRM forms are rendered.  This graphic shows the effect

// // // // // //

When I saw the graph above my first thoughts were

1. WOW, Awesome
2. How did they get such a dramatic improvement
3. Will this break my customizations

How did Microsoft Improve loading times

I will quote the article

There are 2 main changes that have been made: loading process of the form, and handling of cache.

The article indicates they have moved to parallel loading and are caching more data.

 

Hmmmm Parallel Javascript loading

The increased form loading is great but parallel loading can have some downsides.  I first came across Javascript parallel loading in CRM 2011 rollup 12, you can read about here

• CRM 2011 – Things learnt when reviewing Javascript code on form loads
• Intermittent errors with JavaScript personalization post UR12- Library is undefined
• CRM 2011 Polaris Asynchronous Javascript Loading

To speed up form loading in CRM 2011 they switched to parallel loading and the result meant lots of Javascript methods suddenly stopped working.  The reason for this was because one Javascript file was calling a method on another javascript file.  This worked when the Javascript files were loaded in an order but when you load them all at once sometimes one Javascript file tries to call another dependant Javascript file which is still loading.

So suddenly lots of Javascript errors appeared and all developers had to write a Javascript wait file, like CRM MVP Scott Durow talks about in this article

This is a significant change and could cause lots of errors if you have Javascript code in your OnLoad functions which call other Javascript files.

I noticed a new section appeared in the mdsn article Write code for Microsoft Dynamics CRM forms

Manage library dependencies

As a performance optimization, Microsoft Dynamics CRM forms load JavaScript web resources asynchronously and in parallel. This means that the order in which the libraries are configured for a form does not guarantee that a library will be fully downloaded and initialized before another library might attempt to use one of the objects defined in it.

If you have code that depends on another library to be fully downloaded and initialized, the most straightforward approach is to combine both libraries within a single JavaScript web resource with your code below the library code. A more sophisticated approach is to use libraries such as head.js or require.js to control how the separate libraries are loaded.

 

Is Parallel loading good or bad

There is a quote in blade runner

Replicants are like any other machine. They’re either a benefit or a hazard. If they’re a benefit, it’s not my problem.

Upgrading to CRM 2015 SP1 could cause big problems to Javascript customizations.  It’s good practise to split up your Javascript code to separate files to make it reusable and maintainable but now Microsoft are recommending we put everything in one file.  This could turn out to be one big massive Javascript file, which is ok for performance but maintaining and working with this file is difficult.

This change could catch many CRM developers out because the errors are intermittent, sometimes the javascript files will have loaded in the correct order and work.

It will involve developers writing Javascript code which waits until all the Javascript files load which will cancel out any benefits from parallel loading.

It could be seen as an opportunity to restructure Javascript code to have a self contained loading script and then move other Javascript functions to OnChange events.  This would be a double win of increased loading time due to parallel loading and running less code.

You can use Legacy form loading style

There is a system setting which contains a global setting to control which form loading style you use.

It’s a great idea for Microsoft to put this in because the new speedier form loading code might take some while getting use to it.  When CRM 2011 polaris release came it, it caused a raft of blog posts because CRM Javascript customizations suddenly stopped working.

Settings -> Administration -> System Settings -> General. Select “Yes” under “Use legacy form rendering”

In a perfect world it would have been great to have a entity setting rather than a global setting for the form rendering.

I thought I would write about this because it could effect a lot of people and I never spotted this change until today.

 

Hosk’s Top CRM Articles of the week – 15th May

Article of the Week

Customizing Lead Qualification Process in CRM 2015

Great article from CRM MVP Jukka, going through customizing the lead process in CRM 2015

Best of the Rest

Good coding practices – Information hiding

A Hosk article looking at the good coding practise of Information hiding and encapsulation and why you should be doing it in your code

Querying Many to Many Relationships in Microsoft Dynamics CRM

A useful blog showing you how to query many to many relationships in CRM advanced find

Information to Get started with Unit Testing with Microsoft Fakes and Microsoft Dynamics CRM 2013

CRM Developers should be unit testing, here is some information to help you get started with Microsoft Fakes

Using Queues in Microsoft Dynamics CRM

detailed post on queues in CRM

Hierarchical Visualization – Features & Limitations

How to Retrieve all the Access Team Members for all Accounts in Dynamics CRM

It uses SQL but it’s interesting to understand the underlying configuration

update CRM 2015 with a single request

No more special messages in CRM 2015 SP1 if you use the new update request

What is the !! Javascript operator!!!!

A double !! operator in Javascript but what the hell does it do?

Microsoft Dynamics CRM 2015 / 2013 / 2011 SDK Example Index

A big long list of examples in CRM SDK

Getting CRM Dev toolkit to work with Visual Studio 2013

There is still no CRM 2015 Developer toolkit and you can’t use the current DEV toolkit with Visual studio 2013 but you can if you hack a few things.  Come on Microsoft help the CRM developers out a bit and get stuff working with the latest versions

Why two CRM addins within Outlook?

Why are there two CRM Addins!

Zsolt Zombik Dynamics CRM Blog’s Top CRM Articles of the week – 15th May

Another Article of the week collection!! Such a great idea I have decided to do one myself

programming

Mature Developers

Announcing TypeScript 1.5 Beta

Principle of abstraction

SOLID Principles of Class Design

Ebook – 97 Things Every Programmer Should Know – Extended

other

Warren Buffett on How he Keeps up with Information

Nassim Taleb: How to Not be a Sucker From the Past

Last Weeks Top CRM Articles

Hosk’s Top CRM Articles of the week – 8th May

Useful Hosk Links

Hosk list Of CRM 2013 Tools

A list and review of CRM 2013 tools, this will probably work in CRM 2015 as well

Hosk’s CRM Developer Articles

A collection of my favourite CRM Developer articles I have written

MB2-703 – CRM 2013 Customization and Configuration Certification Information

All the CRM 2013 content to help you pass the exam

HoskWisdom – Hosk Developer Quotes

 Words of Wisdom from the Hosk.  I have written over 900 articles, surely I should have said a few memorable things

Good coding practices – Information hiding

I was reading code complete 2, which is an excellent book and I would recommend all CRM developer read it, you will learn lots of things.

The book mentions information hiding, this may be a new concept to some CRM developers because it’s not a term you hear very often.

Steve McConnell the author of Code Complete 2 has a great article on it called Missing in Action: Information Hiding

Information hiding would be a by product of code using the S.O.L.I.D principles

What is information hiding

I find it helpful to use multiple definitions, design principles it can be difficult to understand and different perspectives can be beneficial.

Programming Practices

Information hiding focuses on hiding the non-essential details of functions and code in a program so that they are inaccessible to other components of the software. A software developer applies information hiding in software design and coding to hide unnecessary details from the rest of the program. The objective of information hiding is to minimize complexities among different modules of the software. Note that complexities arise when one program or module in software is dependent on several other programs and modules.

Information Hiding

The basic idea is that if code chunk A doesn’t really need to know something about how code chunk B (which it calls) does its job, don’t make it know it. Then, when that part of B changes, you don’t have to go back and change A.

 

Hosk Definition of Information hiding

Information hiding is not just about hiding data but hiding how it implements those behaviours.   Classes  have public methods but hides the private methods and variables which implement the logic behind the public methods

Why is information hiding important

Information hiding reduces dependency on class variables, allowing classes/methods to act like black boxes, hiding logic from the consumers of the class.  Hiding variables reduces potential coupling and dependant classes, which will reduce the effect of change on your code.

Information hiding is not just hiding variables but the methods and internal logic.  Hiding information results in less code being dependant on concrete classes/implementation, allowing internal workings of classes without needing to change any other classes.

This blog post highlights some good points about information hiding

Why you should care about information hiding

Information hiding makes methods and variables private, hiding the internal logic from other classes.  Developers can change the internal logic without changing dependent code.

Methods should only accessed is via the public methods, hiding the internal workings of the class.  CRM developer can change the internal workings of the class and not effect any of the classes which call those methods.

If classes have public variables, public variables might be used by other classes creating a dependency, if you change a public variable you need to check the calling classes still work.

Changing the logic of public variables/methods can effect any dependent classes which will result in more code changed, more testing and an increased chance of creating a bug.

To help understand information hiding it’s helpful to think the CRM SDK.  When we call IOrganisationService.Update(Entity).  We don’t have access to any fields, we have no idea how the code updates the CRM database or what steps it does.

Microsoft could change the IOrganisationService.Update(Entity) to use a different language but because I only consume the public method none of my code would need to change (assuming it works the same way).  Code calling the CRM SDK is loosely coupled.

How you use Information hiding outside of coding

Information hiding and Encapsulation in OOP has a great definition of information hiding

“Showing only those details to the outside world which are necessary for the outside world and hiding all other details from the outside world.”

 

Real life examples

Facebook

You can hide private information from certain people to stop them using it and only show your name.

Computer operating system

Windows/Linux give you a nice GUI operating system to manage your computers but underneath there are lots of private files and programs.

The article Information hiding and Encapsulation in OOP  has a good example

1. Your name and other personal information is stored in your brain we can’t access this information directly. For getting this information we need to ask you about it and it will be up to you how much details you would like to share with us.

Encapsulation

Information hiding isn’t the same as encapsulation but I’m not entirely sure of the difference. Encapsulation and Information hiding share a common goal in hiding business logic.

Wiki quote

Hiding the internals of the object protects its integrity by preventing users from setting the internal data of the component into an invalid or inconsistent state. A supposed benefit of encapsulation is that it can reduce system complexity, and thus increase robustness, by allowing the developer to limit the inter-dependencies between software components^{[citation needed]}.

 

The benefit of encapsulation,  hiding business logic by using private methods and interfaces/abstract classes.  Encapsulation limits the effects of change.  Encapsulation hides/removes dependency on the concrete code behind your public methods

Information Hiding helps reduce the effects of change

Code changes all the time and there are lots of things which can result in code change

• Change in requirements
• Fixing a bug
• Refactoring
• Extension

Developers cannot control change but they can control how the code will be affecting by change.  Good code design using the S.O.L.I.D principles will reduce the effect of code changes in the code.

Information hiding is one of the tools developers have to reduce the effect of change, hiding the information will create loosing coupling/low dependency and using interfaces/abstract classes will isolate your concrete implementation code from the rest of the system and create loose coupling.

This design principle is described in the Law of Demeter, which described excellent in this article  SOLID Design principles

I find it useful to refer to these principles when writing code because it helps to create

Information hiding helps reduce the complexity of code by creating loosely coupled code and more modular code.  Information hiding reduces the dependencies on classes, a class coupled with multiple classes in a system, it’s difficult to understand and change the code because you need to understand how the change would effect all the dependent classes.

Information hiding helps reduce coupling by hiding concrete logic which helps mitigate the effects of changing the concrete logic.

The more independent/loosely coupled code you have the more code reuse you will have.

Benefits of Information hiding

• Simplifies the code
• Reduce complexity
• Limit dependencies in code
• Improved code readability/Increased understanding

futher reading

http://c2.com/cgi/wiki?InformationHiding

http://en.wikipedia.org/wiki/Information_hiding

High Cohesion – Loose Coupling

information hiding with Javascript

Why you should care about encapsulation

Information to Get started with Unit Testing with Microsoft Fakes and Microsoft Dynamics CRM 2013

This blog will highlight useful information for any CRM developers wanting to learn about Microsoft fakes and Microsoft Fakes with CRM.

I have recently decided I should be unit testing my CRM code, to be honest I knew I should have been doing it before but I had excuses not to

• The projects I had worked on didn’t have unit tests
• The other developers were not unit testing their code
• The company/project didn’t stipulate unit tests should be written
• Unit testing CRM code is difficult

You can read about this in more detail

Experiences of Unit testing with Microsoft Dynamics CRM Projects

If you want more reasons why CRM developers should write unit tests then read this article it has 18 reasons benefits of unit testing your code

Why CRM Developers should unit test their code

Since I have started unit testing my CRM code not only is my code designed better (smaller classes/methods which are easier to test).  My code is tested and I have caught more bugs, particularly bugs where the testing diverted from the happy path and tested the alternative and exception path tests – Don’t just test the happy path.

The biggest benefit to having unit tests is you can refactor and change the code with confidence because after you have finished making changes you can test your code with unit tests.

What Framework should I use?

I wasn’t sure what framework I should use to test my code.  I had previously worked on a project where they used NUnit, another project used RhinoMocks.

I started writing some unit tests with RhinoMocks but found difficulties when trying to test some code using Metadata because the Metadata was a read only collection.  Microsoft Fakes is an isolation framework which allows you to create shims to overwrite read only data.

CRM 2015 – Understanding CRM Metadata

I had problems setting Attributes collection on the Metadata record.  This was a mixture of me not knowing standard unit testing methods and Attributes is a read only collection.

Many developers were put off using fakes due to cost.  Fakes was free in Visual Studio Ultimate (which is expensive).  The prohibitive cost put off many developers who turned to other free/cheaper testing frameworks.

Sometime in 2013 Microsoft bundled in Microsoft Fakes with Premium license (which is much more common).

Microsoft Fakes is included in Visual Ultimate and Premium in release 2012 and 2013, the key point is Fakes is free with Visual Studio Premium edition.

I had heard good things about Microsoft Fakes, it had shims which would allow me to overwrite read only collections.  So I decided to start testing with Microsoft Fakes.

Start learning about Microsoft Fakes

When I started writing a unit test, I quickly released I was going to have to do some fast learning.  The terms used by Microsoft Fakes were completely alien to me

• Shims
• Stubs
• and no mention of the word Mock
• Isolation Framework

There are two main hurdles when learning about Microsoft fakes

• The terminology
• Getting into the testing mindset
• Writing tests for the first

To begin with it was tricky to understand the structure and terminology of Microsoft fakes, let alone start testing my tricky CRM code.

Microsoft Fakes Introduction General

This section highlights content I found useful whilst learning about Microsoft Fakes in general without focusing on using Microsoft fakes with to Test CRM code.

A good, quick (7 minutes) video introduction to Microsoft Fakes

Below is a good general article on Unit testing with Visual studio

Writing Unit Tests for the .NET Framework with the Microsoft Unit Test Framework for Managed Code

Microsoft’s main page on Microsoft Fakes

Isolating Code Under Test with Microsoft Fakes

The Isolating Code Under Test with Microsoft fakes is a great introduction to fakes and it explains what Stubs and Shims and gives you some code examples.

Free Microsoft Fakes Ebook

Better Unit Testing with Microsoft Fakes

The ebook above gives a good introduction to Microsoft Fakes but the majority of the ebook is comparing Fakes to other testing frameworks, explaining how to convert your tests/logic from another testing framework to fakes.  It’s useful (and free) but for users getting started with Microsoft fakes you will probably stop reading half way through.

Unit testing with Fakes with Visual studio Premium 2012

This is a good article which is a good introduction to Microsoft fakes with the bonus of the source code being available for download.

CRM and Microsoft Fakes

The best single resource for getting started with writing unit tests using Microsoft Fakes and CRM is a series of 10 blog posts by Zhongchen Zhou

Dynamics CRM 2011 Unit Test Part 1: Introduction and Series Contents

This a fantastic tutorial, going through all the major elements in CRM you need Stub/Shim to enable testing of your CRM code.  Below are links to the fakes CRM testing.  I recommend clicking on the introduction which gives a description of each blog.

• Part 2: Microsoft Fakes with LINQ query
• Part 3: Microsoft Fakes with OrganizationServiceContext
• Part 4: Microsoft Fakes with CrmOrganizationServiceContext
• Part 5: Microsoft Fakes with LINQ query through IOrganizationService
• Part 6: Microsoft Fakes with OrganizationServiceContext through IOrganizationService
• Part 7: Microsoft Fakes with CrmOrganizationServiceContext through IOrganizationService
• Part 8: Microsoft Fakes with IOrganizationService
• Part 9: Microsoft Fakes with CRM plugin
• Part 10: Microsoft Fakes with workflow activity

The examples are for CRM 2011 but CRM plugin development is based around the IOrganisationService and OrganizationServiceContext and these are still in the same in CRM 2015.

You can also download the source code shown in the tutorials in the link below

https://code.msdn.microsoft.com/Dynamics-CRM-unit-test-93be6676/view/SourceCode

Having the code is useful because you can run the tests yourself and walk through the code to follow the unit testing flow (you can also copy parts of it into your unit tests)

Unit Testing Plugins using Microsoft Fakes

A short article which runs through setting up Microsoft fakes, it goes through all the steps to manually test a CRM plugin.  It has useful examples stubbing the IOrganisationService.

OrganizationService.RetrieveMultipleQueryBase = (query) =>
{
var result = new EntityCollection();
return result;
};

CRM 2015/2013 Testing Framework

Wael Hamze with help from Ramon Tebar (MVP) have done some great work and created a CRM 2015/2013 testing framework.  This is a framework for testing plugins/custom workflows.  The framework does a lot of mocking of the common plugin objects.

The framework is useful but I think the business logic code should be kept inside plugins.  I prefer to split it up into smaller focused classes and methods but I know a lot of people will be interested in testing the plugins, if so this project will speed up the process.

The project is very well documentated, so you should be up and running in no time.

https://xrmtestframework.codeplex.com/

Not only is this framework excellent but there are some very useful videos showing you how to use the framework

xRM Test Framework for Dynamics CRM 2015

Webcasts and Slides which has videos and presentation on the subject.  There is lots of source

 (Webinar Recording) – CRMUG SIG: Technical/Developer – Unit Testing Plug-ins in Dynamics CRM 2013 (June 2014)

 Technical/Developer – Unit Testing Custom Workflow Activities in Dynamics CRM 2013 (July 2014)

Technical/Developer – Integration Testing Plug-ins in Dynamics CRM 2013 (July 2014)

Books (Unit testing in general)

I’m not qualified to say what books are good but I’m reading and enjoying

The Art of Unit Testing: with Examples in .NET

 

What is the !! Javascript operator!!!!

I saw some Javascript code which the CRM Developer Pacman was working on and it had a bizarre If statement which used a double exclamation mark!!

One exclamation mark means not but what does two exclamation marks mean Not Not?

Below is the code in question

var roleName = null;
$.ajax({
    type: "GET",
    async: false,
    contentType: "application/json; charset=utf-8",
    datatype: "json",
    url: odataSelect,
    beforeSend: function (XMLHttpRequest) { XMLHttpRequest.setRequestHeader("Accept", "application/json"); },
    success: function (data, textStatus, XmlHttpRequest) {
        var result = data.d;
        if (!!result) {
            roleName = result.results[0].Name;
        }
    },
    error: function (XmlHttpRequest, textStatus, errorThrown) {
        //alert('OData Select Failed: ' + odataSelect);
    }
});

The code was doing an OData query and then checking the result with the if Statement

if (!!result) {

      roleName = result.results[0].Name;

}

What does the !! exclamation do

It’s quite clever because the first exclamation mark converts the object into a boolean value, the second exclamation mark checks to see if the boolean is false.

In C# developer terms its like casting the object to a boolean and then doing an inverted check (e.g. if not false)

This two Stackoverflow articles discuss and explain the !! exclamation in more detail but this answer and the truth table helped me understand

Double Exclamation points

''        ==   '0'           // false
0         ==   ''            // true
0         ==   '0'           // true
false     ==   'false'       // false
false     ==   '0'           // true
false     ==   undefined     // false
false     ==   null          // false
null      ==   undefined     // true
" \t\r\n" ==   0             // true

Using the table above my understand is the !! exclamation in the examples checks to see if the result object is not 0, undefined, null because all those values would be converted to a boolean type of false.

Here is another good discussion on the topic

What is the !! (not not) operator in JavaScript?

Whilst researching this topic I found the article below

Truthy and falsy in JavaScript

The article talks you understand how Javascript uses true, truthy, false and falsy in JavaScript

 

Hosk’s Top CRM Articles of the week – 8th May

Article of the Week

Microsoft Dynamics CRM Online 2015 Update 1 (7.1.0) Product Documentation is Available!

The documentation for CRM 2015 SP1 is fantastic.  This link shows all the documentation for the new features

CRM 2013 – Managed solution problems with out of sync solutions

Hosk blog posts about solutions, managed solutions and problems with branching solutions

Best of the Rest

What are the limitations of Microsoft Dynamics CRM Online and how do you work with them?

CRM Online has some limitations this article looks at how you can create solutions to work around them based on an interesting white paper from Microsoft

Query and visualize hierarchical data

A page on the hierarchical data structure in CRM

Improved Business Rules in CRM 2015

A look at the changes in business rules in CRM 2015

toLookup and toEntityReference helper methods

a nice helper method

Error Logging Options in CRM

logging options in CRM

Convert FetchXML to SQL query

a tool to convert FetchXML to sql

CRM 2015 – {Javascript Code Example #2} – Save a form asynchronously with call back function

JavaScript code to implement a callback function

“Mind the gap” to make your CRM charts look better

Great blog post from CRM MVP on charts

Dynamics CRM 2015 Update 0.1 Full Text Search Quick Find Performance Feature

The dynamics support team have a feature to improve wildcard searching with CRM on premise

programming

Why TypeScript is Hot Now, and Looking Forward

getting started with test driven development

Other

How a Scientifically Proven Technique Can Improve How Well You Learn and Remember

Local boy Elon Musk unveils Eskom’s end, the battery to set South Africans free

Stephen Hawking – Master of the universe

Last Weeks Top CRM Articles

Hosk’s Top CRM Articles of the week – 1st May

Useful Hosk Links

Hosk list Of CRM 2013 Tools

A list and review of CRM 2013 tools, this will probably work in CRM 2015 as well

Hosk’s CRM Developer Articles

A collection of my favourite CRM Developer articles I have written

MB2-703 – CRM 2013 Customization and Configuration Certification Information

All the CRM 2013 content to help you pass the exam

HoskWisdom – Hosk Developer Quotes

 Words of Wisdom from the Hosk.  I have written over 900 articles, surely I should have said a few memorable things

CRM 2013 – Managed solution problems with out of sync solutions

Solutions were a fantastic concept added into CRM 2011 allowing CRM developers to easily group their CRM customizations in groups (solutions) and easily move their customizations between CRM organisations.

Solutions in CRM offer the CRM developer great flexibility but which can lead to complex multiple and overlapping solutions mixed with branched/versioned solutions.

or

The CRM developer can use unmanaged solutions 🙂

I have written many blogs on CRM solutions

• CRM 2013 – Understanding Solutions and how they work
• CRM 2011 – Solutions and publisher prefix
• You can’t import a CRM 2011 solution into CRM 2015
• CRM 2011/2013 – Progress of solution import
• CRM 2013 – Exporting your changes in a solution from your CRM 2013 trial

Solutions are great but when they go wrong they are awful and can take lots of time poking around trying to find out the cause of the problem.

I’m sure many CRM developers have had problems trying to import a solution where someone has deleted a field, only to add it again as a different type (covered in SQL error whilst importing solution)

Here are some common errors I have encounted with solutions

Solution import Errors

• CRM 2011 – Solution not importing – Timeout errors
• CRM 2011 – SQL error while importing solution in crm 2011
• CRM2011 OptionSetId Cannot be Changed when Importing Unmanaged Solution
• CRM 2011 – error when importing solution

Solutions brief introduction

If you want to learn about solutions read this blog CRM 2013 – Understanding Solutions and how they work, below is a quick recap

• A solution is a way to manage and deploy groups of customizations.
• Managed solutions are read only and the customizations cannot be modified
• Unmanaged solutions copy the changes into the default solution; the solution container is merely a wrapper to move the customizations between CRM organisations.

One of the major headaches with managed solutions is if you remove a managed solution, it removes all the customizations and the data.  This combined with the customizations being non editable (read only) can lead to many frustrating situations which the CRM developer has to work round because they cannot uninstall and install a CRM solution due to the loss of data.

Why use managed solutions

Choosing your solution strategy is an important and scary decision because it can be difficult to swap back from a managed solution to an unmanaged solution.  There is usually some anxiety before you make the decision and push the button and import a managed solution.

There are many choices you have to make in CRM which are very hard to undo like

CRM Entity ownership – How do you decide?

I have talked to many CRM developers who hate managed solutions and only deploy unmanaged solutions. I have been asked numerous times why anyone would use a managed solution, so lets investigate the reasons.

The logic behind a managed solution is two fold

1. Managed solutions are read only to protect the customization creator from users and CRM developers taking and changing the customizations
2. Stopping users from changing the customizations which could result in the customizations not working or working incorrectly.

Managed solutions are great for CRM resellers who created a solution for CRM which acts like a products.  E.g. an Autonumber solution.  They can sell the Autonumber solution, it’s gets deployed and the users can’t change any of the code to stop it working and other CRM developers can look at the code to steal the code/ideas.

This scenario is ideal for managed solutions because if the users change their mind and don’t want to use the solution they can uninstall it and it removes all traces of the solution and its data.

Why doesn’t everyone use unmanaged solutions?

If managed solutions can be problematic why doesn’t everyone use unmanaged solutions?

Unmanaged solutions are great but there are some things to consider

When importing unmanaged solutions, your changes will be written to the default solution.  This will overwrite all

This article Use solutions for your customizations

Never import an unmanaged solution unless you are sure you want to accept all the customizations in it and allow any of those customizations to overwrite any customizations you previously created.

Unmanaged solutions means all customizations are editable, this means users could possibly change a customizations and stop them working as intended by

• Deleting a field
• Add conflicting customizations
• Modifying customizations
• Importing unmanaged solutions could overwrite customer’s modifications or change how they work.

Unmanaged solutions can avoid conflicts but can lead to other problems, often problems caused by overwriting customizations which are not initially obvious.

Managed solutions

Below I discuss some problems CRM developers can experience when using managed solutions.

Multiple managed solutions can be tricky in general

Managed solutions can be problematic is when you have overlapping managed solutions where you have different managed solutions which have the same entity in both.

Managed solutions can have problems when different managed solutions overlap when values and files can overwrite each other and CRM developers being unsure what will happen when one solution is imported over another.

The worst problem can be conflicting problems which cause a manage solution to fail.

Managed solutions out of sync

Managed solutions can be problematic when different versions/phases of the same managed solution are deployed in different environments (e.g. prod, pre prod, dev) with different functionality being delivered.

To visualise this scenario it helps to think of CRM solutions as branches in source control.  Users want changes made to an earlier version but later versions have already been modified.

The CRM developers is in a tricky situation of adding fields/customizations to an older solution but cannot bring these changes to the later branches because it could/would overwrite changes made in the later solution.  This can lead to manually adding fields into the different versions of solutions which can lead to problems

Problems with branched solutions

I recently experienced a problem with branched/versions of solutions which I will explain the problem (I will try, it’s complex), the cause and the solution.

When I use the term branched solution, I am referring to the same solution but with different versions.  I find it easier to visualize and explain the problem if you consider these solutions as code branches.

Resolving conflicts

Any developer who has gone through the process of merging changes through code branches knows it can be a difficult and slow process, involving merging the code, working out if the merged code works and importantly checking the merged code hasn’t broken anything.

When you merge code this can lead to a conflict (this happens when the same piece of code has changed in code being merged) and the developers needs to decide what to replace.

This highlights one of the problems CRM developers have, we don’t do the merging, CRM does and if there is a conflict!  A conflict in customizations

• Plugins
• Workflows
• Web resources (JavaScript)

Conflicts usually involve the imported customization overwriting the current customizations.

Branching solutions problem

I had multiple versions of a solution which was managed

e.g.

• Hosk Solution version 1
• Hosk Solution version 2
• Hosk Solution version 3

• Hosk solution version 1 deployed in production/Live
• Hosk solution version 2 deployed in pre production
• Hosk solution version 3 deployed in Dev and currently being worked on

The solutions above are the same solution with entities, JavaScript, plugins in.

• HSV1 (Hosk solution version 1) was the current live solution and is not being changed
• HSV2 is in preproduction and being tested with some functionality
• HSV3 the current development functionality

Understanding the solutions and their states is easier if you visualise the separate solutions as a waterfall.

Remember HSV2 was created from HSV1 but is now very different and HSV3 was created from HSV2 but now is very different with lots of new and cool functionality.  I have tried to show this by creating them as different colours and shapes.

A situation can occur where a bug is found in Hosk version 1 – HSV1, the CRM developer must decide which solution the fix should be put in.  This problem needs some extra fields to be added to an entity as well as some code.

Where to put the fix?

We can’t put the fix into the latest version HSV3 because we couldn’t deploy this solution to live until all the customizations were finished and tested.

If we put the change into HSV2 we would then need to put all the customizations and changes into live and the customer hasn’t finished testing the customizations.  We would have a problem where HSV2 and HSV3 are out of sync.

The choice made was to put the fix into HSV1 (production solution), add the fields and code, test it on the HSV1 organisation we have and deploy the fix to live.

We now have a problem because the CRM solutions HSV2 and HSV3 are now out of sync.

There are fields and code in the live CRM system which doesn’t exist in HSV2 and HSV3.

One of the main concerns is the current development might not work when deployed with the changes in HSV1, so we have to sync the environments.

How do you get these changes into HSV2 and HSV3?

The fields were manually added to HSV2 and the code added/merged.

The plan was to export HSV3 as unmanaged.

Import HSV2 into HSV3 and then reimport HSV3 with the previous exported version, this will have added the new fields.

Problems occurred when we tried to import HSV2 managed into HSV1 Managed.  The reason it complained was because of duplicate fields.

We added new fields in HSV1 and HSV2.   These fields have the same schema name but importantly different guids.

CRM doesn’t see the new fields as the same field because they have the same schema name; it sees it as a CRM developer trying to add another field with the same schema name.

Gulp, we now can’t import HSV2 solution into the HSV1 environment!

Managed solutions make this tricky

Managed solutions make things tricky because if you remove a managed solution it takes away all the precious data with it.

One of the reasons we ran into this problem was because we didn’t use any other solutions to move new fields/customizations between the different solutions.

Solving the problem

Created a new organisation with HSV1 unmanaged – the CRM org we will call Operation SYNC

Deleted newly added fields/views/dashboards in HSV2

Export HSV2 unmanaged – call this HSV2 modified

Imported HSV2 modified solution onto HSV1

We now have the customizations with the same guids in HSV2

Merge code customizations

This organisation now could be used for the HSV2 environment (old one had to be trashed)

The same technique was used to get the changes from HSV2 to HSV3.

How to avoid Sync/branching solution problem

The key to the problems I experienced was you need to be aware you have different branched solutions.

If you need to add changes to the earliest solution (e.g. HSV1) you need to have a mechanism to bring these changes through the other branched solutions.  It’s important you bring the SAME changes through the CRM systems otherwise (if you add the fields manually) you will be get duplicate errors.

The solution we used above seems easy when you read it but it took a whole day of creating organisations, importing solutions and we weren’t sure it was going to work (fingers and legs were crossed) until it worked.

If the solution didn’t work we hadn’t considered the alternatives but none of them would have been very pleasant, which is why we put of making any further plans.

The first step to avoiding this scenario is don’t create duplicate fields in different branched solutions.  This will lead to duplicate fields and cause solution importing problems.

Ideally it’s better to add new fields in the latest solution; you wouldn’t have to worry about syncing your solutions.

Sometimes this isn’t an available choice because the customer needs a resolve a bug in the production solution.

If need to make a change to an older version you need to view your solutions like a water fall and make sure the changes flows down through all the solutions to ensure the fields have the same guids in all your solutions and as good CRM developers know we should never create duplicate customizations.

You could create a patch solution and import it to your branched solutions but you will need to export/import.

Say you want to add new field called HoskField

• Export current branch – HoskField doesn’t exist
• Import patch solution – Hoskfield added, other entity customizations are older
• Import current branch – Refresh entity customizations, HoskField exists

The most important message in this blog post is for people know the danger of branched CRM managed solutions and the potential problems.

What are the limitations of Microsoft Dynamics CRM Online and how do you work with them?

will all CRM instances be online is the future?

Why haven’t I worked on more CRM online projects?

I was contemplating the questions above reading the white paper about creating solutions for CRM online

Microsoft Dynamics CRM Online patterns & principles for solution builders

It’s a white paper highlighting the limitations of CRM online and how Solutions architects will need create solutions to work with the strengths of CRM online and work around the weaknesses.

Is CRM Online the future of Microsoft Dynamics CRM?

Microsoft would like to think CRM projects are all heading online, Microsoft is spending lots of money creating data centres , bringing their top software offerings to the cloud(Exchange, Sharepoint, Dynamics CRM, Azure, Office, etc etc etc).  Making it easy for all the software to work together on-line (e.g. removing barriers)

Microsoft acquired online products to extend Microsoft Dynamics CRM

• Parature
• Social Engagement
• MDM – Marketing

From a cost point of view (looking at it holistically), it seems sensible use massive data centres which are maintained and look after, rather than individual companies having to host servers, hire skilled people to maintain them.

Hosk experience of CRM online

So far I have only been involved in small/simple projects successfully working with CRM online.

The majority of projects I have worked on and have known about have been with CRM On premise.  I wonder if I have worked on complex CRM projects because I am a CRM developer, it’s perhaps more likely the projects I work on will involve complex customizations and with the previous versions of Microsoft Dynamics were better suited to CRM on premise.

I have heard of projects moving from CRM online back to CRM on Premise but no projects moving from CRM On premise to CRM online.

CRM online is growing?

Microsoft Dynamics CRM online is growing in popularity.  I say it but I couldn’t find much information to back that up

Good Momentum For Microsoft Dynamics CRM Reported At Their Fall Analyst Event

The Dynamics product is doing well. The numbers speak for themselves: 12% revenue growth in FY13; Dynamics AX and CRM growing by double digits worldwide and 30% in the Americas and Asia; and CRM Online growing by 80% in FY13, with two out of every three new customers opting for cloud. Microsoft Dynamics has 359,000 customers and 5 million users, while Microsoft Dynamics CRM has 40,000 customers and 3.5 million users.

The post is a few years old but I’m going to going to assume Microsoft Dynamics CRM online is growing.  I will assume Microsoft will actively continue to grow the Microsoft CRM online offering, which will result in more CRM online projects for CRM Developers to work on.

It’s easy to envisage a future where companies pay for virtual cloud servers and services instead of paying for skilled IT guys to look after their own servers.

At the moment my personal view is big CRM projects and complex CRM projects are not a good fit for CRM online.

Limitations of CRM online

Below are some of the limitation of CRM online

Indexing

A lot of CRM projects I have worked on have involved SQL indexing.  I understand indexes are not automatically added because when indexes need to be added it has to be in a specific area which needs a performance improvement.

How does indexing work with CRM online?  how do you find out where the SQL server is running slowly?

The white paper says you can get Microsoft engineers to look at indexes through a support request but I wonder how this works and how effective it is?

Increasing performance – How?

CRM on premise infrastructure is designed to allow more resources to be added in a modular way.  e.g. if you need more performance in the SQL server, you can add more SQL servers.  If you need better performance on the CRM front end, you add more CRM front end servers.

How does this get managed on line?

Data privacy

Some companies don’t want to their data in the cloud no matter how secure Microsoft says it’s is.

Sandboxed plugins

Sandboxed plugins can’t access any 3rd party dll’s, which can be quite limiting

Custom ASP.NET pages

Custom ASP.NET pages can’t be done in CRM online, so no iFrames showing custom pages.

Reports are FetchXML

CRM online reports can only be written in FetchXML because CRM developers have no direct access to the CRM database.

Size of database

Online database charge by the gigabyte.  CRM databases can grow pretty quickly but how do you shrink POA table if gets to big.

The emphasis will be on CRM support engineers to reduce the POA table size but the Microsoft as a company do not benefit from this (because they charge by the GB), so there is conflicting goals.

Reading the document has made me aware of techniques to move data out of the CRM database into services like SharePoint (or similar) but database management could be a full time job.  The alternative of course is there will be so much space available in the future where database size is not an issue.

The current relationship between megabytes and gigabytes could be the same relationship between gigabytes and terabytes in the future and the talk about database size seem silly.

Hosk Thoughts

if Microsoft keep investing in the Online infrastructure and products and more companies use CRM online then I will inevitably get increased exposure to CRM online.

I view CRM Online similar to CRM mobile in the fact it’s the direction the industry is moving but currently this seems to be a slow process and I expect the majority of CRM project to be CRM on premise.

I’m not sure why Microsoft are delaying service updates to CRM on premise?  It’s going to make CRM developers and resellers life increasingly complex and frustrating, delayed enhancements don’t seem appearing enough to change peoples suitability or desire for CRM version.

In my experience big or complex CRM projects are more suited to On Premise solutions and it would be extremely difficult to do these projects online with CRM onlines current limitations.

If Microsoft changed the online version to host servers online and the CRM developers could access the servers then it would be a very compelling argument, a half on line/on premise solution.

Understanding how CRM Online solutions can work

Reading the paper for the first time, I felt it was unusually worded because it highlighted a lot of limitations but portrayed them as benefits

Microsoft Dynamics CRM Online patterns & principles for solution builders

This line jumped out at me

Everybody has felt a little guilty when they wrote some bad code in the past. Well, in Microsoft Dynamics CRM Online, the temptation to write icky code has been removed in a lot of places, especially where it comes to resource utilization.

Thinking about the document it raises some good points because Solution architects will need to design different solutions for CRM Online than they would if their were designing a CRM solution for a CRM on premise.

Developers don’t like being limited but if more CRM Solutions and customizations will be in deployed in the cloud it’s knowledge CRM developers will need to have.  Part of a CRM Developers work practise is to adapt to new versions of CRM and CRM online is a different version to CRM On Premise.

The part of the document I found worrying is the areas it mentions lack of resources which might be available to you.

My experience with CRM performance issues has involved a lot of investigation around infrastructure (networks, resources, etc), SQL database performance (query performance, adding indexes).  Code optimization and database locking investigation.  I will give an honourable mention to the POA table and security configuration as potential causes of poor performance.

Considering the above any investigation would rely heavily on Microsoft support engineers, this is must be a potential concern for any organizations considering large CRM projects online.

These two quotes from the document, which discuss the potential downside of CRM online

no portion of these shared resources are dedicated strictly to the instance running your solution – they’re shared. That means that you must design your solution to accommodate potential scenarios where these resources don’t perform your requests immediately.

Also, be aware that while there are no strict limits on the amount of workflow jobs you can send to the queue, if you or any of your neighbors sharing the resource are using an inordinate amount of resources, you may have a governor placed on your usage

Hard limits with CRM online

Plug-ins	2 minute timeout
SQL	30 second timeout for database transactions
Running workflow jobs	Fair use – no specific hard limits, but the resource is balanced across organizations
Direct database access	Not allowed

Where the document is really useful is it focuses on how you should construct CRM Online solutions to utilize the strengths of the CRM online offering.

If you have large long running processes then you can create services in Azure to do this work and take the heavy work load, nicely avoiding hitting any of the hard limits set in a CRM Online solution.

I have no experience of CRM Online using Azure so it was interesting to read the methods to  deploy code as

Azure Iaas

Deploy code to IIS in the cloud running in a virtual machine (VM).  The same as hosting a webservice on your own server or usually in a new web application on a CRM server.

Azure PaaS

I’m not quite sure how this works, it seems you don’t configure IIS but just deploy your code and deploy it as a cloud service.  This seems to be more like a windows service (but in the cloud).  The code needs to be written in a certain way to be run a cloud service.

Configuration over code

Microsoft in recent versions of CRM have been steadily improving the no code solutions with improvements giving improved functionality to mobile devices and more customization choice for CRM Online

Code                  GUI Customizations

Javascript     –     Business Rules

Plugins         –      Real time workflows

Managing Storage

CRM online charges by the Gigabyte/per month fee.  CRM Developers need to manage the CRM database size by storing information in other cloud services such as Sharepoint online and azure storage (Microsoft naturally recommend their own products).

The conclusion of white paper

Creating solutions for the cloud is different. It requires thinking about the system architecture in a different way. But making this leap is an imperative for almost anybody selling business or government solutions today. Keeping up with the market requires us to make the necessary adjustments, and allows us to embrace the attendant benefits of building solutions for the cloud.

Initially I thought highlighting the limitations of CRM online was an odd thing to do.  Looking at the document again it’s seems like a good idea because it’s important CRM Developers know the limitations.

Offering advice and techniques to work around these limitations shows you what is possible.  The white paper points to techniques, software and services CRM developers will need to become more familiar with in the future with CRM on line projects continuing to increase in size and complexity.

The increased complexity of CRM online solutions will need developers to create CRM customizations using no code tools such as business rules and real time workflows in conjuction with code deployed and running in Azure services.

This may explain the raft of new synchronization enhancements added into CRM 2015 SP 1, which I talked about in the blog post below

What’s new in CRM 2015 SP1 for developers, customizers and admins

The white paper won me round in the end, highlighting ways around some of the major limitations with CRM online.

Many CRM Developers will have no experience of deploying code in azure and consuming it within CRM.  I would say not just CRM developers but many CRM resellers won’t have experience of creating solutions using code deployed in azure.

This were my thoughts on CRM online, please leave your thoughts, experiences in the comments

Hosk’s Top CRM Articles of the week – 1st May

Article of the Week

Great detailed blog on conditional formatting with CRM Charts

Highlighting contribution using conditional formatting in CRM charts

The article of the week is a pair of blogs by Adam Vero on charting.  The article will show you how to tinker with the chart XML to give you more control over what is displayed.

Best of the Rest

CRM 2013 – Create Enumerations for option sets

How and why you should Enum for your option set in your CRM code

GETTING STARTED WITH MICROSOFT DYNAMICS CRM 2015 ROLLUP FIELDS

A look at rollup fields in CRM 2015

Experiences of Unit testing with Microsoft Dynamics CRM Projects

Hosk experiences of unit testing in CRM projects

LINQPad 4 Driver for Dynamics CRM is available on CodePlex

A new version of LINQPad has been released

Don’t just test the happy path

Don’t just test the happy path, test the other paths as well

Setting up your Dynamics CRM Development & Source Control Environment

CRM 2013 – Investigation into replacing the Assign functionality

Hosk was trying to find a solution to a requirement

GetAttributeValue demystified

GetAttributeValue will stop putting null checks in your code

CRM 2015 – How to change the option set default value

Find where the option set default value is kept

Debug / Unit Test CRM 2011 JavaScript Rest/Soap in visual studio for on-premise CRM installation

Good article on unit testing Javascript in CRM

Why you shouldn’t put unsupported customizations in Microsoft Dynamics CRM

Don’t go down the dark side of CRM development, don’t put in unsupported customizations

Deployment Options

The deployment options for CRM

Microsoft Dynamics CRM Online patterns & principles for solution builders

White paper on building solutions for CRM online

What is max degree of parallelism option setting in SQL Server?

An SQL server setting you can change, but what does it do

programming

Get Visual code here

Microsoft releases free code editor for Mac and Linux!!

C# Coding Conventions (C# Programming Guide)

Screaming Architecture

Great article from Uncle Bob on architecture

TEST YOUR SAD PATH FIRST

Happy path testing

Lessons Learned in Software Development

other

The Timeless Nature of the Herd Mentality

Mental Model: Bias from Overconfidence

Last Weeks Top CRM Articles

Hosk’s Top CRM Articles of the week – 24th April

Useful Hosk Links

Hosk list Of CRM 2013 Tools

A list and review of CRM 2013 tools, this will probably work in CRM 2015 as well

Hosk’s CRM Developer Articles

A collection of my favourite CRM Developer articles I have written

MB2-703 – CRM 2013 Customization and Configuration Certification Information

All the CRM 2013 content to help you pass the exam

HoskWisdom – Hosk Developer Quotes

 Words of Wisdom from the Hosk.  I have written over 900 articles, surely I should have said a few memorable things