How to create realistic Test Data for your CRM Project and why you should

True happiness comes from the joy of deeds well done, the zest of creating things new.

Antoine de Saint-Exupery

Poor quality and inconsistent data in your CRM database = bugs and odd behaviour out of CRM customizations and CRM demo’s – Hosk

Test data which doesn’t replicate live data in your CRM environment can cause bugs and create havoc in demo’s but few CRM developers or companies have realistic dummy data available to import.

Have you been in these situations

Whilst showing new functionality, unexplained errors disrupt the demo

or

Setting up a new CRM environment, finding the process is bottle necked by bad data (which always takes longer than you think to import and get right)

Create good data once and import many times

Being able to import dummy data into different CRM organisations and different CRM projects is a valuable resource, making it worth the effort to create your dummy data files.

Useful  and reusable data is contacts, accounts, tasks, phone calls, emails, Address, Text

Developers and Data

All CRM systems need data to use it. without data your beautiful CRM solution is like a car without petrol, it won’t do anything. – Hosk

CRM developers don’t like dealing with data requirements for a project (because it’s a task which isn’t coding).

Getting your data in a format which can be imported into a CRM organisation is great because often during CRM projects you will need to setup and move data between lots of different CRM environments e.g.

• Dev
• Test
• Customer Test
• Pre-Production
• Production

Characteristics of good CRM project are well structured, organised and all aspects are planned and prepared.  Data preparation and importing between environments should be easy with minimal effort required by the developers.

Avoid data technical debt by ensuring data files are kept up to date and stored in source control, all developers should no the location and the steps to import the data.  Don’t burden one developer with importing the data, it’s not pleasurable for the developer and the project can become dependent on the developer, which can cause problems.

On a chaotic CRM project the data import/export process will be ad hoc, not defined and a confusing and time consuming process, prone to missing chunks of data.

The time spent on creating and updating the data to move between CRM organisations is time well spent.  In the long term it will make a unpopular tasks manageable and will save you time (particular with bugs caused by bad data)

Importing data is simple

Microsoft Dynamics CRM comes with an easy to use Data Import Wizard for import small to medium amounts of data.  Below is a guide how to use the Data import wizard.

CRM 2013 – Importing users using the Data Import Wizard

The data import wizard is a great tool for importing small amounts of data quickly.   The data enrich functionality when used with the advanced find is a great way to bulk edit

• Select records
• Update records (could be a non CRM user)
• Import and update those records

The enrichment process is explained in my blog post Why the advanced find is a CRM Developers best friend.

CRM 2016 may has some new functionality called  the Bulk Data loader, which might offer an alternative for importing data. You can read about the new CRM 2016 features in the blog post – What’s new in CRM 2016 and why you should read the preview guide

Sample data in CRM

Microsoft includes sample data and you can add and remove it by going into

Settings –> Data Management — Sample Data

Click on the Sample data you can install\uninstall the data.

Step by step instructions can be found here

Adding and Removing Sample Data in Dynamics CRM 2013

before you can import data you need to create some.

Why create realistic dummy data?

Good data in CRM database can help find bugs, bad data can help create bugs – Hosk

Setting up environments and data importing are often unpopular jobs for CRM developers (Developers favourite activity is coding and any other activities get in the way!)

In most projects you will need to create accounts, contacts and maybe some records.  Consider it’s likely you will need to set-up lots of different environments so you want to get on top of your data requirements.

What is developer/test data?

If I use the term bad data I am referring to what I call developer data.  Developer data is data created by developers or testers which doesn’t accurately replicate the data created in production or the data which will exist in the production environment.

You could create lots of data with Test e.g Test1, Test2, Test3, perhaps with a number counter appended.  The problems I have found with poor test data

• Not testing your customization’s with realistic data, can delay finding bugs until you hit the live system with real data
• Test data can make understanding and using the system confusing
• It can give a bad impression to the customer (similar to the broken window theory)
• https://en.wikipedia.org/wiki/Broken_windows_theory
• Bad data can cause bugs
• Good data can find bugs

Realistic data or random data can help find bugs in your system.  Developers will put simple data into a system (usually as fast as they can), this allows bugs to not found.  This is one cause of bugs being uncovered in the live system, when real data is imported into the system you can find errors appearing

• The code doesn’t like comma’s, hyphens or quotes
• length or size errors
• null errors
• unexpected types of data

Data imported into a CRM organisation stretches, pushes and pulls the system to see if it brakes.

Why do developers create developer data

Developers do not and often don’t need to know how the end users will use a feature.  In many situations developers will be given a technical specification document with a list of required features to implement.  A developer will learn enough about the customization to create it but will lack the detailed knowledge of the business analyst who captured the requirements and wrote the functional specification.

If a developer isn’t clear how the functionality will be used it’s difficult for the developer to put in realistic data, instead developers input data in the correct format (e.g. string, number).

A developers goal when developing is to test the code navigates through the various paths of code, read my blog post to learn more about this – Don’t just test the happy path

• Happy path (when it works)
• Alternative path (other routes through the code/customization)
• Exception path (when things go wrong)

The developers goal is to put in as small amount of data to be able to test the different code/customization paths.

The unrealistic data used by developers and developers use of System Administrator security role highlight why companies should test functionality with testers in a different CRM environment from the developers CRM environment.

Don’t let bugs get into production environments

Benefits of realistic data

The benefits of creating realistic dummy data is the ability to import data into your existing CRM project and future projects.

• Contacts and Account data are reusable
• You do the effort once, reap the rewards of your efforts many times over
• Customer safe data ready to important
• Save time and effort creating the data for every project
• Ease of use

I found contact data import data I created for a project using the the website Mockaroo.  I got the files and imported them into my test system, first go without problems.

boom 500 contacts with realistic names, phone numbers and email addresses which imported straight into CRM.

Make sure you import the data and fix all the errors.

Good data should import with out errors.

Tools to create Test data

I have found three websites to create test data, please leave other good sites in the comments.  I looked at free websites to create the data

Database Test Data

A basic website with an easy to use interface.   It’s not as complex as the other two does have a lot of useful functions.  The biggest plus for this site is there is no limit on records created.

• Auto-increment
• Username
• Create words, sentences and paragraphs
• Email, Phone Number, Address
• DataTime, Date, Time

link – Database Test Data

Pro’s

• Easy and quick to use
• Export file in JSON, CSV, XML
• No Limit on data exported
• You can save recipe (columns and data structure)

Con’s

• 3 export choices
• Smaller choice of columns

Mockeroo

Url – Mockaroo

Mockaroo is a great site with a lot of functionality to create common columns in data import.  The preview function is handy.  The is a limit of 1000 rows of data unless you would like to sign up for a data plan which starts at $50 dollars a year.

Mockaroo organises the fields you can add into various section, the location field creation functionality has the options below

The IT has useful fields

Pro’s

• Easy to use
• Export file in CSV, JSON, SQL, and Excel formats.
• You can save recipe (columns and data structure)

Con’s

• Limit to 1000 rows

GenerateData

http://www.generatedata.com/

Generatedata offers some excellent functionality to create complex fields based on custom lists of data.

The main problem with GenerateData is it limits the export to 100 rows unless you donate.

Below shows an example of contact export I did, you can see fields using custom lists

Prospect Standard – Gold, Silver, Bronze.

Pro’s

• Easy to use
• can create complex data
• lots of export types

Con’s

• Limit to 100 rows

Summary of tools

Database test data is great for creating a lots of data but is limited to basic contact, account type of data.

GenerateData is great but the limit of 100 rows means you are only likely to use it for data with lots of complex fields.

Mockaroo sits nicely between the two websites above, it has complexity and 1000 rows is decent size.

Don’t forget you can run the tools many times to generate more data.

You can always do some Excel manipulation to create extra fields and use the exported data as the core.