Dual write – Initial sync
Contents
1. Overview................................................................................................................................ 2
2. Prerequisites .......................................................................................................................... 6
3. Lab steps ................................................................................................................................ 6
3.1. Check dependent maps ................................................................................................. 6
3.2. Run initial synchronization............................................................................................. 7
3.3. Change tracking update ............................................................................................... 12
3.4. Performance optimization ........................................................................................... 17
1. Overview
This topic provides general information for dual-write integration between Finance and
Operations apps and Dataverse for initial sync operation.
Before you start Dual-Write for a mapping, you can run an initial synchronization to handle
existing data on both sides: Finance and Operations apps and customer engagement apps. You
can skip the initial synchronization if you do not have to sync data between the two
environments.
The initial synchronization lets you copy existing data from one app to another bidirectionally,
so the initial sync process is 2 leg process:
Leg FinOps to Dataverse: we export data from FinOps via DIXF and import it to Dataverse via
OData.
Leg Dataverse to FinOps: We export data from Dataverse via Odata and import it to FinOps via
DIXF.
In the DIXF area, the source data format used for the Import/export project will be CSV-
Unicode. To avoid format issues with numbers
We do recommend customers for integrated scenarios to keep Language locale = en-us for csv-
unicode.
Depending on the scenario, the customer will have to follow one approach or other depending
on data volume.
Approaches recommended:
Single-threaded tables: First migrate data into the finance and operations app, and then trigger
the initial synchronization to move the data over to Dataverse. Based on lab testing that
Microsoft has done, this sequence has better performance than synchronization from
Dataverse to finance and operations apps.
Multi-threaded tables: First migrate data into Dataverse, and then trigger the initial
synchronization to move the data over to the finance and operations app.
Please kindly consider the following matrix for guidance on options to be considered by
customers.
There will be other considerations as well along with limitations to consider on Initial sync
scenarios documented in the official documentation: https://learn.microsoft.com/en-
us/dynamics365/fin-ops-core/dev-itpro/data-entities/dual-write/initial-sync-
guidance#constraints
2. Prerequisites
Lab prerequisites are: Dual Write installed and environments linked.
-Finance and Operations app environment with Power Platform link enabled.
-HOL_Dual write installation and setup: Dual write needs to be configured and Solutions
applied in Dual write UI.
3. Lab steps
3.1. Check dependent maps
Due to relationships, it may be possible that some mappings have dependent mappings that to
avoid data issues we need to complete the initial sync first. Let’s consider as an example, to
have customers synchronized, we need customer groups first.
To confirm if a mapping has dependent mappings (dependencies) we can confirm it while
clicking on Run button (if the mapping is stopped).
Connect to Finance and Operations > Data Management > Dual write tile.
Select Main account (msdyn_mainaccounts) map, click Run.
If you see the check box Show related table map(s) it means that the mapping has dependent
mappings.
Click Show related table map(s) checkbox and confirm the dependent maps. On this case, to
run initial sync for Main account without issue, you need first to have initial sync run for Chart
of accounts, Currencies and Exchange rate type.
3.2. Run initial synchronization
You can run initial sync for multiple mappings at the same time and it will be done by the
system in the correct order. During this exercise you will be guided to run initial sync, it may be
that you face any issue due to data and further troubleshooting is required (please review later
HOL for troubleshooting along with following TSG
https://supportability.visualstudio.com/DynamicsFO/_wiki/wikis/Dynamics%20FO/750463/-
Dual-write-Initial-Sync-steps-against-Demo-data-for-GAB-maps)
If you don’t select Initial sync checkbox, mapping status will switch from Stopped to run it,
however no initial sync will happen.
For this exercise, select initial sync checkbox for the mappings that are in Not running state.
Master for initial sync can be ignored as we’ll always run two legs. Then click Run.
Once the mapping have the time to process status change (for that time you will see a
progress bar as status), go back to Dual Write main UI and filter in filter by Initial sync.
Navigate map by map to review the status.
For example, let’s review the map named Chart of accounts. Later proceed with the same
review for the other maps.
Select Chart of accounts (msdyn_chartofaccountses)
Open Initial synchronization tab. This mapping is not company specific, reason why there is a
single activity rather than one per company.
Click on Name column (DWM project).
We’ll see 2 lines in the summary window.
Leg FinOps – Dataverse in pink. 12 records (Upserts) where exported from FinOps via Data
management and imported in Dataverse via Odata
Leg Dataverse – FinOps in orange. There are no records exported from Dataverse so no import
(Data management) done in Finance and Operations.
Verify that both legs have completed successfully. In case there is a failure clicking on every leg
line you will see more details about the issue.
Verify what has happened in the Finance and Operations side.
To locate the DIXF projects created (export or/and imports) – even though on this specific
example just an export data in leg 1 happens and there was nothing to move from Dataverse
to FinOps in leg 2 (see that FinOps job is empty for leg 2), please proceed with the following
steps to locate both projects.
Take the first part of DWM project, on this example DWM-f9d1fff3.
Navigate to Finance and Operations > Data management workspace and if top grid search by
contains and the picked DWM project. Click on Enter.
Currently you will see the export project, in case we have an import as well, the search will
show both data management projects.
If you delete this project, it will be recreated if you run initial synchronization again.
Open the project to explore how it is created by Dual write initial sync. Confirm that Source
data format is CSV-Unicode, format used for initial sync.
If you would like to check the history select Job history button to see all the executions ran for
this DIXF project.
As the Execution Id is shown directly in Dual Write UI for the DWM project, take the FinOps job
Id from the summary window: as example: ExportPackage-10/11/2022 3:13:37 PM-6695e754-
8cf9-4a82-b103-5d4898e468d1 to validate what happened on the export execution.
Navigate to Finance and Operations > Data management workspace > Execution history grid.
Filter by your Execution ID.
Select Execution details.
Select Download file to see the data exported from Finance and Operations that has been
created successfully in Dataverse side by initial sync.
3.3. Change tracking update
The first run of initial synchronization pushes the whole data set. This behavior is known as a
full push. When change tracking is enabled in finance and operations apps, the subsequent
runs are an incremental push that is based on the last run map version. When change tracking
is disabled, subsequent runs cause a full push.
To confirm how initial sync triggers full push or incremental push, let’s enable change tracking
for one of the entities that has Dual write map and run twice initial sync. In between will
create one record.
Navigate to Finance and Operations > Data Management workspace > Data entities. Search
Customer groups, select it, click Change tracking and mark primary table.
Confirm that change tracking has set for Customer groups entities to Primary table.
Navigate to Dual write UI (Data management workspace > Dual write).
Search by Customer groups (msdyn_customergroups) – map should be not running – click Run:
Confirm that dependent mappings ran initial sync and are in running state, mark your mapping
with Initial sync option and select Run.
Review the initial sync details and observe the upsets by company. Navigate to Customer
groups form for one of the companies to confirm that it matches with the number of customer
groups using path from below screen.
Stop Customer groups (msdyn_customergroups) map from Dual write UI for this lab exercise
purpose.
Once the mapping is stopped, navigate to DEMF company if you have this company for Dual
write usage and open customer groups. Create a new customer group selecting New button:
Customer group = CustGHOL
Description = CustGHOL
Terms of payment = Net10
Then Save.
Go back to Dual Write UI map, select Customer groups and select Run with initial sync again.
Wait for the initial sync to end and review the company where you created the testing
customer group record.
Check the initial synchronization status and upserts for your company, on this case DEMF and
confirm how many records were inserted/updated:
You should have 1 as the initial sync happened this time in incremental push due to change
tracking.
Click on the relevant DWM project and identify the DIXF execution id:
Navigate with it to Data management workspace. Identify the execution id and select
Execution details.
Download the file and confirm the customer group that was exported. Confirm that it is the
record created for this lab step.
3.4. Performance optimization
Enabling parallel processing (Data management advantage in Finance and Operations) can
reduce the time needed to import data from Dynamics 365 customer engagement apps and
Microsoft Dataverse to finance and operations apps.
To enable parallel processing in finance and operations apps, complete the following steps.
Log in to your finance and operations environment.
Go to Data management > Framework parameters.
Select Entity settings and select Configure entity execution parameters.
Add the parameters for parallel processing as in screenshot below:
Import threshold record count – The number of records that must be met before parallel
processing is enabled.
Import task count – The number of threads (tasks) to run in parallel.
Select Save.
Go back to Dual write UI, and stop (in case of have it running Customer Groups map)
Navigate to Dataverse and create multiple customer groups.
Run initial sync for customer groups map and confirm if the import from Dataverse was fast –
review the Import execution Id created in the leg Dataverse to FinOps.
