Support Portal ContactGet in touch

Data Management using the Dynamics SDK

   Words by Paul McQuillan

   on 19/02/2018 08:00:00

Dynamics CRM gives us access to an API that allows us to Read, Update, Create and Delete Records from Apps that we develop – this has been a core part of Dynamics CRM since its inception, as Webservice driven development has become the norm.

Using this API allows us to perform bulk operations to help manage the data we store in Dynamics.

From this we have developed an App that can help us to manage data from a Source to a Target – this can be a CSV file into Dynamics CRM, or copying data from a build of Dynamics to another, or copying data within a single build of Dynamics.

This last example is we will look at here.

In this example, we have a Dynamics Customer who was previously using 2 Options Sets using a single set of Global Option Set Values – however they have since reviewed their system and decided that the 2 Options Sets should have their own individual values and not use a Global Option Set.

Field A – using Global Option Set

Field B – using Global Option Set

For this, we need to create a New Option Set and set the unique values this will use

Field C – initially the same values but not using the Global Option Set

However as a new field this is then empty of any of the existing data, whereas we need to copy over the previous values from Field B to Field C.

To do this we can use the following steps:

Step 1 - Create a New Over Time Task

  1. Name, a name for our Data Task: Copy Field
  2. Entity Name, the Database Name of the Entity we will be reading from CRM: account
  3.  Rule Name, the logic that our task will carry out: OverTimeTaskImportData
  4. Connections, here we can supply the Name of the Source and Target Connections – either selected from our list of Connections, or directly by typing a new Connection String
  5. Fetch XML / SQL Statement, we supply the XML of the Advanced Find for how we want the Task to read the Source Data from the Source Connection, and so govern which Source Records will be handled by the Task.
  6. All Columns, this yes/no flag defines whether all the Columns will be read from the Source. Setting to YES then ensures that all the Columns from the Entity read from the Source Connection will be read into the Task, NO will mean that only the Columns specified in the Fetch XML above will be supplied into the Task. The default here is NO to help manage performance, and in some incidences, YES to bring in all columns can be useful to us.
  7. Top Rows, this defines a cap on the number of Rows read back from the Fetch XML Query for the Source Connection. This can be useful when testing a new task, as we can run on 1 or 2 records to test the logic is working as we expect.


Step 2 – Define our Data Map

With the Task created, we can setup the Data Maps for how to copy the data from Source to Destination.

In our simple example, we want to define a Map from our original Option Set field B to the new Option Set field C.


Step 3 – Link Map

When bring the data from the Source and into the Destination our logic needs to know when to Create a New Record vs Update an Existing Record, and we do this via a Link Map.

This Lookup allows us to match on a certain Field or Condition to define whether to complete as CREATE or UPDATE in the Destination.

In our example, we can keep this very simple and just match on the Primary Key for each Account record and in Dynamics this is simply ‘accountid’


Step 4 – Run the Task

With the Task and Data Map in place, we can run this.

Our initial approach should be to run on a single record or small group using the Top Rows parameter highlighted earlier when creating the Task – this gives us a safe sample to test our job.

Once run for our sample, we can check the results using a quick advanced find:



We should also look into the Audit History for one of our Sample Records to make sure nothing else has been updated that we don’t want.


This testing step is crucial to avoid running a misfiring task on many records before we are certain it is working – but in our above example, we can see the new Option Set C being populated with the previous values from Option Set B.

So we are now ready to run on a larger sample, and so amend our Top Rows Number and re-run.

Step 4 – Execution Results

With each run, our results are tracked against the Task we have created.


This allows us to have a top-level view of the number of Records affected by our Task and whether any Errors in any of the Rows were reported.

This process gives us a quick and useful way of helping run bulk logic on groups of records in Dynamics – providing a way that different developed Rules can invoke the SDK in the context of a controlled and reportable Task to handle Imports, Bulk Workflow Execution or other Data Cleansing.

For larger migrations or system integrations, there are higher scale tools such as Scribe or Inaport which can perform more complex or tiered imports of data – but this small App allows our Support Team to help Customers with Regular Imports, Data Cleansing, Usage Reports and recovering accidently deleted data from the Audit Trail without the cost of investing in a larger migration solution that might be more of a hammer to crack a nut.

Behind of this is the concept of leveraging the API behind Dynamics can help Customers get the best from the platform, and using small repeatable pieces of development to help more easily achieve business requirements – either as part of a Project or as part of ongoing Support.

Share this Article

Search Articles

Filter Articles

CRM Tech DocMan

Recent Articles

CRMCS Quick Start Guide: How To Produce a Microsoft Teams Live Event Dynamics 365 Marketing: Lead Scoring and Sales Acceptance Designing and Developing Microsoft Power Apps Portals Thank You for Attending CRMCS’ Webinar - Achieving B2B sales excellence with Dynamics 365 & Microsoft Teams Thank You for Attending Our Webinar - Achieving B2B sales excellence with Dynamics 365 & Microsoft Teams Webinar: Discover How CRMCS Have United Dynamics 365, SharePoint and Microsoft Teams To Create Sales Excellence Ignite your workflow by adding DocDrive365 to Office 365 The CRMCS guide to everything you need to know about integrating Teams with Dynamics 365 Saving Time By Keeping Documents In One Place TDE Database Encryption with On Premise Dynamics The Key to Successful Compliance in 2020 Part 2: Let’s get GDPR Compliant with Microsoft Power Automate Top 3 Essential Tips for Remote Working Dynamics 365 Marketing: Top 5 Best Features Dynamics Day in the Life - Puma Investments Can you use Teams to amplify collaboration in Dynamics? Part 1: Using a Scheduled Power Automate to Trigger Expiry Date Reminders The secrets of successful document collaboration in Dynamics CRMCS launches new AppSource approved DocDrive365 Dynamics Day in the Life - Moneypenny Release Management Add the App to Dynamics DocDrive365 Security: Day One - Getting Started with Dynamics to SharePoint Permissions Building a New Scheduled Process using Flow
  • "Paul has made a real difference to how my team of 24 people record and store valuable customer data and sales opportunities. Highly recommended."

    James, Operations Director

  • "Understanding your business allows us to advise when to implement aspects of CRM and, likewise, when not to."

    Paul McQuillan, Managing Director

  • "Dynamics 365 and CRMCS have made a real lasting difference to our business, allowing us to replace older systems that were holding back our performance."

    Grahame, Chief Operating Officer

  • "James worked well with us to help connect CRM with Outlook and relate how this might benefit our team using CRM for Property Care."

    Natalie, Property Care Supervisor

  • "Matt was really good with helping us run User Testing on the new Compliance Module of our CRM System."

    Tom, Compliance Administrator

Prefer to go old-school?

Write to us using the below addresses.

Head Office
CRM Consultancy
61 Oxford Street
M1 6EQ

London Office
CRM Consultancy London
Grosvenor Avenue

Content © CRM Consultancy.