Hosk Top CRM Articles of the week – 27th Feb

Article of the week

Diving into the Product Structure of Microsoft Dynamics CRM 2015 (and meet ET)
A detailed post and video on the new product structure

Best of the Rest

CRM SQL Query go faster trick – SQL Server trace flag 8780

Training & Adoption Kit for Microsoft Dynamics CRM
A good bunch of links for new CRM users.

9 reasons to replace excel spreadsheet with CRM
How CRM can replace a business excel

XRMtoolbox updated
A couple of interesting things. Field Level Security bulk updater has been updated and XRMToolbox has been moved to github

Why the advanced find is a CRM Developers best friend

Why CRM Advanced find is soo useful to CRM developers

Tip #329: Resolve Missing Record Dependencies
resolving missing dependencies

Why your CRM code and customizations should be simple
The benefits of writing simple customizations

SYSTEM-WIDE ALERTS IN DYNAMICS CRM 2015
You can do system-wide alerts in CRM 2015

Using “Roslyn” to Catch CRM Coding Issues
A detailed and interesting blog post looking at the new features and how to use them with CRM

Custom State Model Transitions in Spring Release for Dynamics CRM 2013
This is a really cool piece of functionality, managing what statuses a record can move to. I still haven’t come close to using it yet.

What are the hidden costs of excluding user adoption in your Microsoft Dynamics CRM implementation plan?
A good article looking at the hidden costs

Unit Testing Plugins using Microsoft Fakes
A great post on testing Microsoft plugins with Microsoft Fakes

 

CRM SQL Query go faster trick – SQL Server trace flag 8780

A developer was looking at some reports to try and improve performance and they were advised by Microsoft to use a trick which I had never heard off setting SQL Server trace flag 8780

I know what you are thinking

“Hosk, you didn’t even know about turning on SQL Server trace flag 8780, amateur”

Everyday is a school, so prepare yourself for today’s lesson

What difference did turning on the flag

The performance was dramatically improved, changing the report speed from 14 seconds down to 1 second!

BOOM

I’m interested tell me more

The report was an SQL report and to turn on the flag you simply put the this line at the end of the SQL statement, after the where clause.

option (querytraceon 8780)

e.g.

select name, accountid, telephone1

from account

where name = ‘hosk’

option (querytraceon 8780)

The good thing to note is you can turn this flag on for individual queries.

What does the flag do

The developer called it the “make it faster” setting on SQL. The query in the report was joining three different tables and one of them was contact.

warning – the description below is my understanding, it could well be wrong but it will give you an idea of what I think is happening, more research by yourself would need to be done if you are thinking of using it.

When SQL queries run on SQL Server the SQL server creates an execution plan. SQL Server calculates the most efficient way to retrieve the data you want. When the query is compiled if the execution plan can’t find a quick it will time out and then go with the best one it had found so far.

There could be other more effective execution plans for your SQL statement and if you gave the execution plan a bit more time it could find them.

When I mention time, I’m talking milliseconds (which is a long time for SQL Server)

In the report we were joining the contact table but using a contact guid but the execution plan was timing out because the execution plan was doing a full table scan of the contact table (2 million rows).

Whilst it was trying to scan the whole contact table it timed out and gave up (who can blame it)

SQL Server trace flag – 8780 to the rescue. This flag gives the sql optimizer more time to find the most effective (e.g. quickest) way to retrieve the data.

You set the flag (which can be set server side or for each individual query) this increases the time limit before the optimizer timeout occurs.

The result in our reports meant the execution plan found a better way to optimize the query and the reports ran a lot of faster.

What are the downsides

Traceflags in SQL Server are undocumented and unsupported by Microsoft (unless they advise you to use it 🙂

I view traceflags as developer tools written by developers but not fully tested in all scenarios. This means Microsoft doesn’t know how the queries will react in all scenarios.

The obvious danger by turning off the time limit for optimizing a query I guess is it could keep on optimizing for a long time. The problem is no one really knows what it’s going to do

When to use it

If you have some reports which are taking a long time to run and you have been asked to optimize them, turning on the this flag for individual queries (e.g. reports) could save you a lot of time.

It’s unsupported so I would suggest you only use it when you really need to

Further reading

I couldn’t find a lot of information on this mysterious trace flag but these three articles will help you learn more

Optimizer Timeout or Optimizer memory abort

 Is it possible to give the optimizer more or all the time it needs

SL Server Trace flag – 8780

Why your CRM code and customizations should be simple

 

“Simplicity is the end result of long, hard work, not the starting point” – Frederick Maitland

 

“Simplicity is the Ultimate Sophistication” Leonardo da Vinci

In a previous blog post I wrote about complexity in code and customizations and the problems this can cause.

The problems with complex code and complex CRM Customizations

Complex code is hard to understand code because classes and methods do so many different things it’s impossible to understand how it works without stepping through the code to understand exactly what is going on. – Hosk

A quick recap

Here are some of the main culprits of complex code

• Large methods – Monster methods
• confusingly named classes, methods and variables
• methods which do lots of things
• not consistent
• tight coupling
• dependent code
• poorly structured code

The problems associated with complex code

• No Reuse
• Complex Code is hard understand/read
• Complex Code/customization is difficult to debug, maintain and extend
• Creates complex bugs
• Encourages more of the same

Code should be well structured, split up into logical classes and methods and the everything should have meaningful/logical names.  One part of good simple code is to make it readable, which makes it easy to understand when a developer is scanning the code looking for bugs or see where they should extend the code to add new functionality.

Methods should be in small and do one thing, this should ensure it takes out decoupling.

A lot of the work involved in keeping the code and customizations simple is managing the complexity of the CRM solution.

Writing code and customizations which seem simple is very challenging, it takes time spent on design, coding standards and will power.

Keeping the code simple is worth the effect because it’s so much easier to use and work with.  When you see great code it’s a thing of beauty, often surprisingly simple looking and will make you think you could easily write it yourself.

Keep it simple stupid

When you are writing code, keep in mind the KISS principle, below is a great summary from Wikipedia

The KISS principle states that most systems work best if they are kept simple rather than made complicated; therefore simplicity should be a key goal in design and unnecessary complexity should be avoided. The phrase has been associated with aircraft engineer Kelly Johnson (1910–1990).^[3] The term “KISS principle” was in popular use by 1970.^[4] Variations on the phrase include “keep it short and simple” and “keep it simple and straightforward”.^[5][6]

Like most great advice in life and in coding, it can be broken down so it’s easy to understand but challenging to implement.

The KISS principle is described by the code it creates, simple and elegent.  When I  see an example of great code I am often amazed by the beauty and simplicity of it.  Code which adheres to the KISS principle is using small, tidy, precise, does one thing and does it well.

Make your code simple

if you do the above the code will be

• Simple to test
• Simple to maintain
• Simple to debug
• Simple to extend

The benefits of simple code is it’s simple to

Easy to Understand

If your code simple and the complexity removed then understanding the code is easier.

reuse

A good way to test if your code is simple, is to see if you could easily write a unit test for your code.  If you code is well written and the methods and classes are independent and have loose coupling.  Focused code is easier to write unit tests for because the methods do one thing and the tests only have to test the methods do the one thing.

Code which has loose coupling is very modular which means you can use it in other parts of the code or in other projects because the methods are not dependent on other methods in the project.

Making code for reuse sounds easy but I have found it’s unusual, which is why this quote has a ring of truth to it

code reusability in CRM projects is seen as often as bigfoot or the loch ness monster – Hosk

For more Hosk Quotes check out my Hosk Developer quotes page

No Extra Code

Simple code can be made to be extendable but it should not include any extra code which isn’t needed.  Simple code should go through refactoring so not only does it not contain any unneeded/unwanted code but the complexity is managed the quality is high.

Don’t add any extra functionality which might be used in the future.  The code should have been designed to be extendable.

Consistency

There should be CRM standards document which all developers should have read and stick to.  Code consistency should also be considered when designing code solutions so the code is the same in different projects.

When code is structured differently when written by different developers it can make a project difficult to understand.

Easy to understand

Simple code should be easy to understand because it’s logically structured and consistent.  The code should be readable

What does this mean for a CRM developer

It’s easy to say “make your code simple” but how do we do that.   I would also focus on making your CRM customizations simple, including business rules, workflows

Front end

Looking at CRM from a user’s perspective, CRM should be as easy as possible to use.  CRM forms should be organised so a user can intuitively use the form.

An easy way to understand this is Good CRM design should not make users think

a few key points from the blog post which make CRM simple to use

• Don’t make them think
• Users are like sharks – they don’t stop moving and clicking
• User like back buttons – it helps them to not get stuck
• Users like searching
• give them fewer options – filter options

The benefits of a simple user interface/CRM Form design

• it’s easy to use
• Users like using it
• fewer barriers for using the system
• Gets support on the customer end

Simple CODE

The key to simple code is the classes and methods focus on doing one thing.  As soon as classes and methods start doing more than one thing, they become hard to name, the code becomes coupled.  Code which does more than one thing becomes complex

Independent code

When trying to write simple code you should try to focus on writing independent code.  To write independent code means  classes and methods are focused and do one thing.

Easy to name variables and methods

When you write simple code you will find it’s easy to name classes, methods and variables and if you are struggling to name one of those its usually a side effect the code is trying to do more than one thing.

The great boon of writing simple code is it’s easy to understand because classes, methods and variables do what they say they do. Complex code is hard to understand code because classes and methods do so many different things it’s impossible to understand how it works without stepping through the code to understand exactly what is going on

Small Methods/functions which do one thing

You should write small methods, the reason for writing small methods is it will focus the methods, this makes the code reusable and easily tested.

Small simple methods are easy to understand, maintain, debug, extend.

reduce complexity

• Classes and methods should do one thing
• Independent code – the code isn’t coupled
• Small focused methods

Don’t use lots of different customizations

There are lots of different types of customizations in CRM

• Javascript
• Business rules
• workflow
• plugins
• Dialog
• Console Application
• Web Services

When I decide what type of customization to use in CRM for a particular problem, I usually go through this process

1.  What customization type can provide the functionality needed

2.  What customization type is best suited to provide the functionality

3.  What customizations already exist in this area and in the project

The process creates a list of possibles, which I whittle down to probables and then I need to match the probable solutions with the existing solutions in the project.

The reason for this is because if you have lots of overlapping customizations it makes maintaining, debugging and extending the CRM solution as a whole a lot more difficult.

I have experienced bugs where a plugin has a changed a value of a field, which then gets overwritten by a workflow.  The reason for this is plugins are usually syncronise and workflows are usually asyncronis.  In the bug I was investigating the workflow was running after the plugin.

I have also seen many questions from users on the CRM forums asking how Javascript and Business rules interacting.  Business rules do not trigger javascript on change events.  This makes understanding the customizations difficult because sometimes the Javascript onchange is triggered and sometimes it’s not.  It also means the CRM developer has to be aware business rules do not trigger Javascript on change events and make sure they create the customizations with this in mind.

To simplify it (which is what this blog post is all about)

Increase in types of customization = increased complexity

Your goal in creating a customization is not only to add the required functionality but good developers will concentrate on creating customizations which are simple because simple customizations are easy to understand, debug, maintain and enhance.

Consistency

Customizations in a project should be consistent, even if it isn’t the best way of structuring the customization.  An example could be if a project only uses Linq queries and these Linq queries are in separate classes for each entity.  When someone else writes some plugin code they should use the same structure.

Plugins

The most important part of writing a plugin is it does what it’s meant to do and provide the functionality needed.

Goal of a plugin

1.  To deliver the functionality required

2.  To produce quality code with the fewest bugs in

3.  To produce code which is easily maintained, debugged, extended

4.  To produce code which is easily understand

To create a plugin which has simple code it means you have you should not put it all in one big method directly in the plugin, yet how many times have you seen this in a CRM project?

Split up the code into classes which work logically e.g. entities or business logic/retrieves etc.

create small methods which do one thing.

Summary

Creating code which is as simple as possible really is breaking the code down into small independent parts which work together as a whole CRM solution.

Managing complexity is a never-ending task for a CRM developer, the better you do it the easier it is to work with the CRM project and code within it.

The main benefit of breaking the design and code into small simple parts is it’s easier to understand small simple parts and how they fit together.  This becomes more important when CRM developers work on multiple projects or have had worked on another project and come back.  Simple code will be easier and quicker to understand when you have lost the understanding of the business logic associated with the project.

Creating simple code and customizations looks easy (when you see the end product) but is extremely challenging to do.  Complexity of a CRM solution can quickly increase if the CRM developers are not focused on managing complexity and keeping the CRM solution as simple as possible.

Why the advanced find is a CRM Developers best friend

Microsoft caused quite a bit of unrest when they released CRM 2013, suddenly making the Advanced find button difficult to find, if you are interested I wrote about that in this post – Good CRM design should not make users think

In this post I want to write why I love the advanced find functionality in Microsoft Dynamics CRM.

I view the CRM Advanced find as an awesome swiss army knife

To new users and inexperienced CRM developers it looks like a simple tool to search the records in CRM.

To an experienced CRM developer it’s one of the key weapons in the CRM Developers armoury which when used correctly can save you loads of time and make your life a whole lot easier.

It can speed up navigation, help you create queries for your plugins, export and enrich data and even help you fix bugs.  In some cases when being brought onto an existing product, I have used the advanced find to go directly to records of a certain type because I didn’t know where to look in the CRM system.

What is advanced find?

Advanced find is a tool which allows you to create queries against the CRM database.  It allows you to select the fields you want to see in the results and filter the query with conditions.

You can then save this advanced find query as a personal view

The Advanced find is a lot more than just a tool to query CRM because in many situations it can

• Save you time
• help you understand
• view field values and related entity field values
• speed navigation

The reason CRM Advanced find is so useful (and why users and CRM developers were annoyed when Microsoft made it hard to find) is CRM is really one big database full of useful data.  The CRM GUI

• view the data using a web browser
• validate the data input
• organise the data into forms etc
• Add security
• organise using the data
• Development to automate adding/viewing data

The CRM GUI (website) has lots of functionality for validating and displaying the data.  This is really useful but for CRM Developers they often want to navigate the system quickly and are not interested in the GUI and business logic but just want to get to certain records.

The advanced find lets you search all the records in your CRM database and apply a filter condition to allow you to only bring back the records you are interested in.

One important consideration to remember when using the advanced find is it does apply your security profile, so you will only see what records your security profile has privileges to see.

Quickly find the records you need

The beauty of advanced find is it can bypass most the GUI and get to the record you want you want with a couple of clicks.

It can also bring back groups of records you want, which is more difficult to do with the front end GUI

The great thing is you can save the advanced find into a view so you can repeat the process even quicker next time.

When you don’t know the business logic

In some projects a CRM developer can be brought into the project in the bug fixing phase. It can be difficult to fix bugs when you don’t understand the business logic or how the system works.

Using the advanced find can help you easily navigate to the entity records you are interested because with the advanced find you can view all the entities in alphabetical order.

This can help you if the bug exists on a specific record type and you don’t need to understand the business logic, e.g.

• Javascript form errors
• Simple plugin triggering on update

FetchXML is a great place to create your query

When I have to create a query in a plugin, I usually start with an advanced find to get the logic correct.

You can then download the FetchXML and you can see which fields you need to use and how the filters should be structured

You can use FetchXML in Javascript

You can use fetchXML’s directly in your javascript. I usually use OData queries and the odata query tool.   One of the limitations of OData is it doesn’t have a distinct attribute, so there is no way to not retrieve duplicate records.

The query I was interested in doing needed to filter out the duplicates so I did it in FetchXML.  When I didn’t filtered out duplicates (the OData query) it was returning 1500 records, which slowed down the form load but when I used FetchXML with distinct set to true, it returned 25 records.

FetchXML directly in plugins

You can run FetxchXML code directly in your CRM plugins instead of query CRM using  Linq or QueryExpression.

I go through the process in the blog post below

 Build Queries with FetchXML instead of QueryExpression

One of the advantages of using FetchXML in a plugin is you can create the query in Advanced find which can sometimes be easier

Create saved views

Saved views can save you lots of time because you are only viewing the records you are interested in and filtering out all the other entities.

Export data

Using the Advanced find to select not only the records you are interested in but also the fields is very useful.

Being able to export these records and send them to someone who doesn’t have access to CRM is a very handy trick to customers.

Exporting the data will also allow you to move the data between CRM organisations and environments (e.g. Dev, Test, Prod)

Enrich Data – e.g. export, change, import

The good old Enrich data functionality, which is has the good, the bad and the ugly all in one.  It’s called Enrich the data but what it really means is you can export some records and when you import them it will automatically know which records to update.

The enriching data functionality works by adding the Guid of the records to the export file (and some checksum columns etc) which are read by CRM when you re-import the data.  If it sees the values it knows you are not importing new data by updating/enriching existing data.

Bulk Edit

Once you have a list of the entities you are interested in, it’s easy to bulk edit those records and assign them all the values.

Bulk editing data in this way can add data which bypasses some of the validation in the GUI forms.  The reason for this is business rules and Javascript do not run because they are only triggered on the CRM form.

Hosk Top CRM Articles of the week – 20th Feb

Article of the Week

This article explains how you can prepare for a CRM 2015 upgrade.

CRM 2015 Upgrade: A Few Things You Can Prepare Ahead of Time

 

Best of the Rest

I wrote a very long blog post on why rushed code and projects doesn’t save time, If you haven’t read it then I think you should
Why rushed projects/code doesn’t save time and reduces

 

 

Very good blog post by CRM MVP Jukka about how to use bookmarks to speed up your CRM 2015/2013 navigation

CRM Navigation Hacking with Bookmarks

A great blog post looking the new CRM 2015 hierarchical security model

Exploring the Hierarchical Security model in CRM 2015

 

How to add users in CRM 2015, could be trickier than you think

CRM 2015 – How to add users

 

When you write a plugin, you often need to debug it to see what it’s doing. Videos are always good when understanding development.

Good video on debugging plugins in CRM 2015

 

Understand what it means to put a plugin in the sandbox

Understanding Plugin sandbox mode

A new FetchXML tool

FetchXML Builder for XrmToolbox

Useful, just the kind of thing a customer would ask for

Dynamics PDF – Convert CRM Reports to PDF & Attach to Email

 

How to Load test using the visual studio ultimate edition (warning costly).

Load Testing Microsoft Dynamics CRM 2013

 

If you are upgrading to CRM 2015 and you have some customizations to upgrade then use the custom code validation tool to check they are ok
Microsoft Dynamics CRM 2015 Custom Code Validation Tool

If you want to some words of wisdom from the Hosk

HoskWisdom – Hosk Developer Quotes

 

Why rushed projects/code doesn’t save time and reduces quality

The idea for this post came when I was driving and listening to the Zen and art of Motorcycle maintenance.

The book is full of interesting questions and will make you look at the world and has some fantastic concepts

It unsurprisingly talks quite a bit about Motorcyle maintenance, which I thought would be boring (I’m not a fan of maintenance with my soft developer hands). One part of the book focuses on how a  mechanic fixes a motorcycle, he doesn’t see the bike as a physical set of tires, metal, etc, he sees it as a whole set of logical systems all linked together. So when the mechanic is fixing a the physical chain, he is fixing the theoretical mechanism which is used to move the wheels.

Quality

A recurring theme in the book Zen and the art of Motorcycle maintenance is Quality, it devotes many pages to it.

Here are a couple of good quotes from the book

“The place to improve the world is first in one’s own heart and head and hands, and then work outward from there.”
― Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values

Quality tends to fan out like waves.

ROBERT M. PIRSIG, Zen and the Art of Motorcycle Maintenan

You can read about Quality as mentioned in the book by clicking the links below

Pirsig’s metaphysics of Quality

Quality (philosophy)

Whilst listening about quality, I got the idea for this blog post and I pulled over on the hard shoulder and started scribbling in my one of my many super hero notebooks (this one was amazing spiderman)

Alternative titles and quality

The title of this blog post could have been any of these and it will give you an idea of what the blog post is about.

Rushed projects lead to technical debt
Rushed code = Poor Quality code
Quick projects cost you time later
Why quality is important
more speed less quality

The blog post started on the topic of rushed code and rushed projects create the same amount of code but the rushing part reduced the quality of the code and customizations.

It became clear to me whilst writing, it wasn’t due to the speed of the code but the quality of the code was reduced due to the development processes which were omitted.

Rushed Code and Customizations

I’m 99.9% sure all CRM developers and any other developers have had to create some rushed code at some point. Let’s define what I mean when I refer to rushed Code/customizations and projects

Rushed code/customizations/projects involve knowingly missing out steps to deliver the project in a shorter time-scale.

rushed code (usually implemented in rushed project) will miss out steps to deliver code quicker

The assumption made by a lot of project manager/customers is they can remove parts of the development process without reducing the quality of the end product.

The key question is what parts of the development process are removed, to answer this we need to understand the steps in CRM development

1. Gather requirements
2. Design architecture
3. Design code
4. Write code
5. Test code via unit tests
6. Peer review
7. System Test (GUI)

The goal of the process is deliver the functionality, creating a solution which is as simple as possible with the fewest bugs

Why simple as possible?

I have written about why code complexity is a problem before in the blog post The problems with complex code and complex CRM Customizations

The benefits of Simple code and customizations are
Simple to read and understand
Simple to maintain
Simple to extend
Simple to debug
Simple to test

If you replace the word simple with the word quicker or easier, this explains why simpler code and customizations are so beneficial

The opposite of simple (in my mind) is complex code, below are some of the problems caused by complex code

• difficult to read and understand the code
• hard to test
• difficult to debug
• tricky to maintain and extend
• hard and sometimes impossible to test
• So what steps get removed

The steps most commonly removed from the development process are usually the parts of the process which don’t involve creating customization and code.

• Code Design
• Unit Test
• Testing
• Refactoring
• Peer review

As you can see all the non-code parts are removed because the project manager/customers cannot or have chosen to ignore the difference between good code/customizations and bad code/customizations but are merely judging the end product by number of lines of code/number of customizations.

Why are non-code processes removed?

Customers/Manager believe developers are only really working when they are writing code or creating customizations, I’m not quite sure where this idea came from but here are my best guesses

• When developers first start developing (particularly if self-taught) they start learning by creating code. Inexperienced developers believe developing is writing code and do not yet understand the value of the non-code related parts of development.
• Code/customization is the end product, so the shortest amount of time creating this must be the most efficient uses of the developer
• Some developers do not enjoy or see justification for the non-code writing parts of development and don’t do them

It’s a question of quality?

This strikes me as similar to someone building a house but instead of planning and designing the house they just get straight out and start putting down bricks. The end product would be a house but is more likely to be a house which could be huffed and puffed down by a big bad wolf.

When/why is rushed code and customizations

rushed code/customizations are created when things have to be done quick, the kind of estimates which leave enough time to create the code/customizations with no provision for design or things going wrong

Here are the common causes of rushed code/customizations and projects or reasons for missing out development stages

• Emergency Fix
• tight/impossible project deadline
• Change request
• Quick/Dirty fix
• Poor development processes
• inexperienced developers
• Lazy/poor developers

I originally attributed the removal of development stages to projects/customizations being rushed but it occurred to me the non code development steps are also willing removed by some developers. These developers are usually either inexperienced or poor/lazy developers.

A developer who believes he can write code with little design and no peer review or unit tests will create poor quality code more often than they create good quality code – Hosk
What motivate rushed code/removal of development steps

I will refer to the removal of development steps as rushed code, even though this happens sometimes by choice and not under time constraints, the end result in my eyes is rushed code.

There are two key types of people who create rushed code

Customers
Lazy developers

Customers will often push for short time-scales, going against the estimate and advice of developers.

Lazy/poor developers will rush the code because they do not want to either spend time doing a proper code design or often don’t like writing test units or other developers peer reviewing their code.

You can understand the initial logic of customers believing projects delivered in a shorter time scale must be better value.  The customer and the project manager can pat themselves on their backs at getting more for less.

It’s my belief they should be kicking themselves up the backsides because by rushing projects you generating poorer quality projects, which will be a lot more difficult to maintain and debug and likely to contain more bugs.

It is a known practise for some companies to lower bids for projects but the only way they can deliver projects at lower costs is by spending less time creating the code.  Less time spent creating code means the removal of non coding development process.  The consequence of removing the non coding but quality increasing development steps is a reduction of quality in the end product.

Reduced price = reduced quality

A lot of the time the cheaper version of a product is cheaper because the quality is reduced.  If you compare the Apple Ipod to the many cheap MP3 players (which look remarkably like the iPod) there is a difference in quality.  The cheap MP3 players are

• Feel cheaper
• break more easily
• hard to use

In this example you pay less and you get an inferior product, the quality of the product is reduced along with the price.

Rushed code I feel falls into the second category of reduced quality.  The main side effect of reduced quality is increased complexity and a lot of technical debt.

Why do customers ignore quality

When customers push for shorter delivery dates they are agreeing to creating a finished product of reduced quality.

This seems odd behaviour which the same customer wouldn’t do outside of work.  If this customer was getting a house built or hiring a plumber to install a boiler or hire a mechanic to fix their care.  Would the customer

• Would the customer hire someone who was really cheap but not very good?
• Would the customer get the job rushed and agree to corners being cut?
• would the customer allow the house to be built with minimal design and not testing of the house?

The question is why is this behaviour seen as ok with CRM projects, particularly when some CRM projects are big investments for companies.

I believe customers are judging the successful delivery of a project using the wrong criteria.  Projects should not be deemed successful if a version of the CRM project goes live by a certain date.

Successful projects should be judged by

• What functionality was delivered by the project
• How many bugs were found in the project
• Did the project release go without problems

For a CRM project launch to be successful it’s vital the project is released with as few bugs as possible and with key functionality included.  Early setbacks in a project or a disastrous initial release can stop the CRM software from being used by the end users because they can easily lose faith in new software if it’s full of bugs and doesn’t work very well.

A successful release of a CRM project will give the users confidence in the system.

If you get it wrong on the initial release of the CRM project, sometimes you won’t get another chance to win the users over.

Why Quality is important

Apple are a great example of quality products.  Most of the apple products (iPhone, iPod, iPad) are quality products which are easy and fun to use.  This quality allows Apple to sell their products for a high price because the user knows they are buying a quality product and they consumers are willing to pay more.

Quality in code is an interesting concept because CRM Developers can look at some code and in most cases will instantly know if it’s good or bad code but if you ask a CRM developer what are the characteristics of good code, they will stop and think and won’t be able to give you an instant answer.

The design phase of a project and design of code are probably the most important phase of a project and creating code.

The design and structure of code, will sort and organise the code in small simple parts.  This greatly reduces complexity of the code.

The simpler we can make the code the easier it will be to understand the code in six months time when our business knowledge of the code and the system has diminished.  When developer a CRM project, during the development the CRM developer will have very high levels of business knowledge.  The CRM developer will understand the code in detail  will understand the business requirements and rules of the customer.

When the development phase is finished and the CRM developer works on other projects but later works on the original project, the business knowledge and code knowledge of the system will have reduced.  If the code is simple, the CRM developer will easily be able to understand the logic of the code and enhance and debug the code.

If the CRM solution is complex the CRM developer needs the high level of business knowledge to understand the code, this is where rushed projects really suffer.

Rushed projects suffer from technical debt because the development processes which get removed

• Testing
• Refactoring
• design
• peer reviews
• documentation

The processes which get removed to save time (or the CRM developer can be bothered to invest the time) are all focused on the quality of the code.   The tasks focus on ensuring bugs don’t exist, the code is as simple as possible and can be tested.   All the processes reduce the number of bugs in the CRM solution and are designed to keep the code well structured and lean.

What does rushed code look like

Rushed code like its distant cousin bad code has a smell.  Code starts to smell when it isn’t right, learn more about code smells by reading the wiki article.

When creating code it’s a constant battle to keep the quality high and the code simple, as soon as the standards start to drop the complexity of the code will start to rise and any interaction with complex code takes longer.

You must control code complexity.  Uncontrolled complexity will drag the code down until no one wants to touch it.

Rushed code and projects is where less time has been spent on designing, planning, structuring the code, testing, reviewing and this will increase the complexity of the code.

Rushed code = greater code complexity

Here are some of the main culprits of complex code taken from here

• Large methods – Monster methods
• confusingly named classes, methods and variables
• methods which do lots of things
• not consistent
• tight coupling
• dependent code
• poorly structured code

The other sign of rushed code is no unit tests and a lack of documentation.

Consequences of rushed code

Quality manufactured goods last a long time and are usually a pleasure to use, the same is true for quality code.   Quality code is great to use for CRM Developers when they are reading/understand, debugging, maintaining and extending.

CRM Projects are expensive investments for companies so it surprises me they want to reduce the quality of the solution created because this I believe should the goal of the CRM development.  Create a high-quality CRM solution which delivers the required functionality and do it right first time.

I believe customers, managers and other people pushing for rushed code should be asked

What is the goal of the CRM project, is it to create a CRM solution quickly or is it to create a quality CRM solution.  Is it time or quality?  The person asking for the rushed code should also be made aware the time saved now should be reinvested later to bring ensure the quality of the CRM solution is kept high.  If the time is not put in later then modifying, maintaining, debugging and extending the code later will all take longer.

The person asking for the rushed code should also be made aware the time saved now should be reinvested later to bring ensure the quality of the CRM solution is kept high.  If the time is not put in later then modifying, maintaining, debugging and extending the code later will all take longer.

Borrowed Time

Producing rushed code is like taking a  short-term loan out, if you don’t pay it back soon then the interest will keep building up.

The currency used with projects and code is time.  Rushing the code will save you a bit of time now but the long-term effects of the rushed code will cost you more time later

Rushed code is like paying for a car using your credit card and then paying back the bill slowly each month. If you want to buy a car you should get a proper long-term loan out.

CRM projects do not seem like the place to want to cut corners and reduce the quality of the end CRM solution by rushing the project to shorten delivery times.  Particularly if the company plans to use the CRM solution for many years to come.

One of the key consequences of creating rushed code and rushed CRM Projects, is any further enhancements or code interaction will take more time and resources.

Rushed code and projects is where less time has been spent on designing, planning, structuring the code, testing, reviewing

The code will contain more bugs

The CRM Solution/Code will not be optimized and could run slower

The CRM solution will have increased complexity

You will have no tests to run against the code base, increased chance of missing bugs.  Unit tests are a valid way for new developers on the project to understand and step through code

Business logic

It’s important to create documentation and tests when you creating the project because you knowledge of the business logic (e.g. customer logic and code logic) will diminish the longer you spend away from the project.

No documentation and unit tests mean you have to relearn everything and this will be more difficult if the code has been rushed because it means the code will be more complex.

More Bugs

Rushed code will likely to include more bugs.  The reason for this is because no unit tests have been written (which can find bugs), no peer reviews of the code.

Bugs found later in a project will cost more time and effort to fix.  You can read the two discussion to find out more

Is it significantly costlier to fix a bug at the end of the project?

Software Defects – Do Late Bugs Really Cost More?

Without having statistical evidence it would seem to me bugs found later would take longer.  If you take into account the process of the customer raising and logging the bug, time to understand and then recreate the bug.  Bug hunting will probably be harder if the person hasn’t used that part of the system or knowledgeable of the code (e.g. not in the project zone).  Moving the code through all the CRM environments.  It seems logical that it’s harder to find bugs later on and then the fixing and promoting the bug fix into live will take longer the later you find the bug in the development process.

Technical Debt

Rushed code will have technical debt due to the short cuts taken to produce the code and customizations faster.

If you don’t know what technical debt is read this wiki article and it’s basically refactoring and changes which you owe the code/project.  Technical debt are code restructuring changes you have put on the list called TO DO Tomorrow.

Just like items you put on the TO DO tomorrow list, they don’t always get done, which means the technical debt keeps growing, which consequently makes tackling it a bigger and bigger problem.

Poor readability

When developers look at code they scan it to get an understanding of how it works.  Before you can debug or extend code you have to understand what it’s doing and how it works, this makes readability very important 

If you struggle to read the code, how the hell are you meant to fix it – Hosk

Rushed code can be inconsistent and complex because it was created quickly.

Summary

Rushed code or not creating the code and customizations using all the development steps will create complex code of reduced quality.

Working on rushed projects is not enjoyable, good CRM developers do not like creating rushed code and customizations of reduced quality.  It feels wrong at the time and hard to use later on.

When you create rushed code and customizations any time saved on the creation of the code will be lost when the code is worked on later.

When you create rushed code you are agreeing to create a CRM project of lower quality

CRM 2015 – How to add users

I had to go through one of the CRM 2015 rites of passage this week, I tried to do the what I thought would be easy task of adding a user to CRM.

I have added thousands, no Millions (warning dramatic exaggeration being used) of users, so I wasn’t expecting to have any difficultly.

I went to check what users were added to CRM

I navigated to

SETTINGS –> Administration

where has users gone?

Did I have the correct security role?

they always use to be visible

Not only security is missing by security roles, field level security, business units and a bunch of other

WHERE IS IT?

It was the kind of frustration only using CRM can bring out of me, like when Internet explorer slowly crashes all open pages one at a time.

If you want to read more frustrations of a CRM developer

go away you paperclip video

So where is it?

On SETTINGS, scroll along DA DA

Security

Here are all your security needs

Why did I miss it?

The question was why didn’t I see this, I even scrolled along looking for it and still missed it.

It’s the menu system used in CRM 2015, it’s not easy to use

• It’s slow to use
• it’s tricky and precise
• It doesn’t seem intuitive

I have talked about the design in more detail in this article

Good CRM design should not make users think

FRUSTRATING

the fact it can disappear just as you look for which option you might need is very frustrating and I can’t work out why Microsoft have designed the navigation like this.

It seems to me Microsoft Dynamics is using a design for tablets where space is at a premium but the reality is most CRM users are using desktops and possibly two monitors, so it’s space which is the primary concern for users but ease and speed of navigation.

I read a good article about an alertnative method of navigating CRM by book marking key areas, this offers a quick one click way to move between key pages in CRM

CRM Navigation Hacking with Bookmarks

I will mention the best navigation improvement is

 CRM 2013 One Click Navigation

You can read my review of it here or watch the video below

Here is my list of top CRM 2013 tools

Understanding Plugin sandbox mode

Plugins

Plugins in CRM are great ways to run powerful .NET code and our one of the CRM Developers most powerful tools in his toolbox.

There are lots of steps you need to go through when creating a plugin and there are lots of things which can error.

The knowledge you need is usually built up with a bit of theoretical knowledge and then a lot of practical trying.  The amount of knowledge you need to write a plugin is one of the reasons Why .NET developers struggle with CRM Development

If you are starting out writing plugins, I would recommend my youtube plugin playlist

I would recommend reading my blog post on common plugin errors, which you will certainly experience at some point CRM 2011/2013 – Common Plugin Errors and Isolation Mode

The common errors talks about the  post discusses a deployment common error you might experience, which is you cannot deploy plugins with isolation mode = none if you are not a deployment administrator.

If the isolation mode = “Sandbox” then any CRM Developer who has the security role of Administrator can deploy plugins into a sandboxed environment.

Where can you see the Isolation mode of a plugin

When you create a plugin you can choose two different types of isolation mode

• Sandbox
• None

You may have seen these when you are deploying plugins in the CRM Developer toolkit, it’s in the RegisterFile.crmregister file, you can see each assembly has

IsolationMode= “Sandbox”

You can also see what isolation mode of a plugin if you open the Plugin Registration Tool.

1. open the Plugin Registration tool
2. connect to CRM instance
3. Right click on an Assembly and choose update
4. Now see the Isolation mode

Confusion

When developers talk, naming and terms used can add confusion and slow down people’s understanding.

A plugin assembly has a setting called Isolation mode and this can be set to None or Sandboxed, but you will rarely hear any CRM developers use the term isolation mode.

Instead they will say things like these

• A plugin is sandboxed
• CRM online plugins must run in the sandbox
• it’s a normal plugin

When a CRM Developer mention Sandbox/Sandboxed and plugin they mean the plugin has an isolation mode which equal “Sandbox”

CRM online plugins can only run with an isolation mode = sandboxed

A normal plugin is a plugin which has an isolation mode =  “none”

Often used but not really understood

CRM isolation mode is used all the time by all CRM developers who create plugins but often most CRM developers only have a vague idea of what Isolation mode means.

Most CRM Developers will have heard about Isolation mode with regards to CRM online.  The reason for this is CRM Plugins created for CRM Online organisations must have Isolation mode = “Sandboxed”

What is the Isolation mode?

The first (and best) place to start with understanding Isolation mode and Sandboxed plugins is with this great MSDN article which can be found in the CRM SDK (which should be the first place you look for documentation)

Plug-in isolation, trusts, and statistics

You will notice the isolation mode of a plugin is at the assembly level and not the individual plugin level.

This is why when you create a new plugin there is no setting for isolation mode because it’s at a DLL\project level and you can have many plugins inside one project (or inside one dll).

Isolation Environment

Plugins with an isolation level of “Sandbox” run in an isolated environment and as you can imagine this environment is more secure.  Key point

The sandbox is more secure and some actions are restricted

The restrictions are a bit of mystery, in the sense there is not one all encompassing list of code which will cause an error in a sandboxed plugin.  This can be very frustrating when you deploy a plugin in the sandbox and it throws an error, finding the cause of this can be difficult and frustrating.

The Plug-in isolation, trusts, and statistics article states

isolated environment, also known as a sandbox, a plug-in or custom activity can make use of the full power of the Microsoft Dynamics CRM SDK to access the organization web service. Access to the file system, system event log, certain network protocols, registry, and more is prevented in the sandbox.

The article gives a good description of what this means by saying Plugins deployed in the sandbox are partially trusted (e.g. you only partially trust the plugin and are limiting it’s powers) and plugins which are not in the sandboxed are fully trusted (by you the developer) and you are happy to let them interact with the file system, registry, call other dll’s and web services etc.

Sandboxed and non sandboxed plugins

I recently had some experience with sandboxed plugins because a project I was working on moved all the plugins out of the sandbox.  The first result was I couldn’t not update or publish any plugins because I wasn’t a Deployment admin

The second side effect is I remember the sandbox has its own service.  If you have ever installed CRM you notice there are Sandbox CRM Service and standard CRM Service.  Most CRM Developers know of the services when they do one of the many IISRESETS + Service restart, which usually happens if you have DLL’s in the GAC.  The sandbox service runs all the sandbox plugins.

The sandbox service runs all the sandbox plugins.

I was remote debugging, so instead of attaching the remote debugging to the

• w3wp.exe – standard plugin
• CRMASyncService.exe – if the plugin is asynchronous
• Microsoft.Crm.Sandbox.WorkerProcess.exe – for sandbox plugins

One reason I personally don’t like remote debugging is if someone is debugging a standard plugin, then by debugging the w3wp they stop the whole server whilst and everyone else who is using it.  I have seen some scenarios where other developers and testers have had to stop whilst the one developer debugged his plugin (poor show).

To get around this you can change the plugin to sandboxed and then when you debug you only stop the sandbox service

Sandbox limitations

I have stated previously it’s difficult to know what the limitations are but here some of the limitations taken from the msdn article

• Access to the file system (C Drive)
• system event log
• certain network protocols
• registry
• You cannot access any other DLL’s
• IP addresses cannot be used
•  Only the HTTP and HTTPS protocols are allowed.
• In isolated mode you cannot call any external DLL’s\DLL’s in the GAC

This blog had some good restrictions in a bit more detail

• Attempting to use the AppDomain.CurrentDomain.AssemblyResolve event
• IO.Path.GetTempPath() [System.Security.Permissions.EnvironmentPermissionException]
• Any filesystem access code [System.Security.Permissions.FileIOPermissionException]
• Attempting to use the EventLog [System.Diagnostics.EventLogPermissionException]
• Attempting to use IsolatedStorage [System.Security.Permissions.IsolatedStoragePermissionException]
• Any references to Thread.CurrentThread caused a security failure.

Someone tweeted me a great article on security and sandboxed plugins

How to: Run Partially Trusted Code in a Sandbox

The article is fantastic but it does highlight the difficulty developers can experience using the CRM SDK, sometimes the information is there but it’s hard to find.

CRM Online limitations

I believe CRM online sandboxed plugins cannot use LINQ queries, which throws an error due to another transaction being created in the LINQ query.

Custom workflows cannot be used in the CRM online sandbox

Correction – You can call WCF webservices 

I originally put you couldn’t call webservices but I have been corrected thanks to a great comment from Andrii Butenko

To use webservices all you need is to use WebClient if possible like https://msdn.microsoft.com/en-us/library/gg509030.aspx

In case only WCF is available you can call it but you have to manually build binding https://mscrmmindfire.wordpress.com/2013/06/14/calling-external-web-service-from-a-crm-2011-plug-in/

CRM 2015 Online allows custom workflows

Another great comment has caused me to update this post.

Custom workflow activities can now be registered in isolation mode in CRM 2015 online

https://msdn.microsoft.com/en-us/library/gg334752.aspx

Example error

I came across an error which prompted me to write this blog post.  I was trying to serialize some fields and pass them between plugins using shared variables

This forum post below is the same error

https://social.microsoft.com/Forums/en-US/9ce3c4e6-0587-4913-89f8-f3942d7bac93/using-systemwebextensions-in-crm-plugin

Unhandled Exception:

System.MethodAccessException: Attempt by security transparent method ‘TestingSeam.CrmTesting.Execute(System.IServiceProvider)’ to access security critical method ‘System.Web.Script.Serialization.JavaScriptSerializer..ctor(

This error was unusual because it was complaining about access to a critical method.  I wondered why serialization was critical method but thinking about it, serialization must involve writing the data to disk and sandboxed plugins are not allowed to do this.

A Gotcha also occurred when someone changed the plugin to sandbox mode, it stopped the plugin working, so I had to leave a comment in the code to say, this plugin won’t work if the plugin is sandboxed.

Why put plugins in the sandbox

So the question is why would people choose to put plugins in the sandbox.

The most common reason is they are using CRM online and you have no choice because all plugins have to be

Runtime statistics are created  PluginTypeStatistic entity records. These records are populated within 30 minutes to one hour after the sandboxed custom code executes

https://msdn.microsoft.com/en-us/library/gg334752.aspx

This url has an interesting benefit

1. If the sandbox worker process that hosts this custom code exceeds threshold CPU, memory, or handle limits or is otherwise unresponsive, that process will be killed by the platform. At that point any currently executing plug-in or custom workflow activity in that worker process will fail with exceptions. However, the next time that the plug-in or custom workflow activity is executed it will run normally. There is one worker process per organization so failures in one organization will not affect another organization.

Why haven’t you listed all errors

Some readers maybe dissapointed I haven’t listed all the methods and exact lines of code which will cause sandbox plugins to throw security errors.

Whilst writing this blog post I realised the reason Microsoft have listed all the causes of security errors in sandboxed plugins because there are potentially lots of them and if they tried to list them all, they would miss some. In fact I would say it’s impossible to list them all, so there is no benefit to trying.

The way to think about the sandbox is it’s a bit like a sandpit and the code is a child.  As soon as any code/child tries to get out of the sandpit we throw an error and a parent comes along, tells the child off and throws them back in the sandpit.

This is why sandboxed code is safer because it can’t get onto your server and cause havoc.

Summary

The limitations of sandboxed plugins can be a significant influencing factor for many CRM solutions to be on premise.

The limitation of not being able to call webservices and DLL’s in the GAC can be quite restrictive.

There is a solution using ILMERGE, which is a method of merging a number of different DLL’s into one DLL, which will allow you to use the plugin in a sandboxed environment.  I would be cautious about this approach because if you ask anyone who has done it, they will tell you it wasn’t easy and debugging a merged DLL can be impossible.

Here is a good article on ILMerge issues