Mule provides four different event processing models
- Synchronous flow
- Asynchronous flow
- Batch Job
- Scatter Gather
Choice of the model depends on the use case. In the article we will explore different use cases and the appropriate event processing model to use in each of the use cases.
Use case 1 – One way sync one-directional synchronization integration solution between two enterprise systems: from a RDBMS database (such as MySQL) to a Salesforce CRM system.
Requirements –
- approximately a million rows of customer data are added to the database from another integration application.
- The Mule application must quickly forward these changed database rows to Salesforce.
- Only new or modified database entries should be processed.
- Records must be transformed to match the Salesforce data structures.
- Salesforce bulk operations should be used.
- Auditing and traceability of each record is critical.
- Acknowledgements must be sent to other endpoints and logs when records or files are received and when data is successful written to the target database.
Questions to ponder
- What are the options to read in changes from a source database?
- Design
- What components can read the database, as soon as a row (s) are changed in the database?
- Database on Table Row
- Scheduler & Select
- How can changed records be written to Salesforce?
- One record at a time
- Can records be written in bulk to Salesforce, or must they be written one record at a time?
- There are a million records, what will be the performance impact of writing one record at a time
- How to trigger the process?
- Connection Management – manage, monitor and restart database connections
- Types of errors and error handling
- Processing latency
- How fast and how much data can be written in the database
- How can the data synchronization between the system be limited to only new or modified records?
- How does Salesforce signal database state synchronized with Salesforce?
- How does Salesforce signal the Mule that records were successfully transferred or not transferred?
- Use transactions? & rollback
- Use watermarks to synchronize writes to Salesforce? or rollbacks
- Impact of duplicate records in Salesforce
- Missing records in Salesforce
- What components can read the database, as soon as a row (s) are changed in the database?
- SLAs
- real time/ near real time
- restriction on how fast data can be read
- scheduled data be processed at fixed rate
- Avg. number of records per scheduling cycle
- Highest number of rows to be processed during a scheduling cycle
- Design