Posted by: Sasirekha R
bulk, file transfer, Integration
While most of the aspects discussed so far are related to online integration, it is worthwhile to mention that 80% of today’s integration relies on basic technologies like file transfers. File Transfer based integration though has the negative factor of High Latency, brings in lots of benefits and hence should be the preferred choice wherever possible.
File Transfer Make the data available by transporting a file – to any format like xls, csv, xml, flat file – that is an extract from the application’s database so that other applications can load the data from the files.
The plus points are:
- Local copy of the data exists – and high performance (for both applications)
- Maximum level of Isolation – each application can be independent of each other – and only communicate on a planned basis for data replication
- Security constraints can be enforced at file usage level
- More efficient as it is getting chunk of information at one go
- Can be used in both Push/Pull mode, depending on the business requirement
- Preferred for Updates – as the Validations and data integrity checks can be built in before bulk updates
The downsides are:
- High Latency – needs Latency Tolerance
- Data replication and Synchronization to be handled – if the communication is two-way
Other factors are:
- Isolation possible – using data transformation
- Need to have Simple Conflict resolution Logic – involving human intervention, if required.
- Medium Complexity to build
- Use Wherever possible – within the context of data integration
- Exploit application’s Import and Export facility
- Ideal if there is latency tolerance (if considered from pure business perspective, this becomes applicable in large number of real-life situations).