In this article, we discuss data replication, the process of copying data from one location to another, as it seems this trend is coming back into fashion. Many companies are now moving their ERP systems into the cloud, which results in them losing the visibility of their own database, so they create a process of creating a data repository to run reports and analytics.
Data replication is not a new concept; I began working with ERP systems in 1991. One of the challenges we faced was that the ERP system I worked with had a proprietary database, restricting us to using the ERP’s internal tools to create reports (it was written in C on Unix!). However, as businesses sought greater flexibility in choosing their preferred reporting systems, the need for an independent database became apparent. During that era, popular relational databases included Oracle, Sybase, Informix, and DB2. At the time, Microsoft SQL server was still in its infancy and not seen as a serious database for large volumes of data.
Oracle Headquarters in Redwood Shores, CA. I took this photo in 1996 while flying over the bay area with my friend Woody Woodworth in a small Cessna Aircraft.
At the time, Crystal Reports (acquired by Business Objects in 2003), Business Objects (acquired by SAP in 2007) and Cognos (acquired by IBM in 2008) were the most popular reporting systems and companies wanted the freedom to use them instead of using legacy reporting systems provided by their ERP providers.
In the case of the ERP system I specialized in, a need arose for a replication mechanism that would enable companies to generate reports using their preferred reporting systems from a copy of their database. At the company where I worked, we initially replicated data nightly, which was sufficient. However, later on, when I joined a company in Silicon Valley, we developed a real-time replicator that seamlessly transferred data from the ERP system with its closed database to the database of the user’s choice. This replicator proved highly successful and gained adoption among major corporations.
Although this story may seem antiquated, I am sharing it because I’m now observing a similar need for data replication in today’s cloud-based ERP systems. With the rising popularity of such systems, access to the underlying database becomes limited, leaving companies restricted to the reporting systems provided by their ERP provider. Since businesses value the freedom to select their preferred reporting or business intelligence (BI) systems, the demand for data replication has resurfaced.
Infor CloudSuite is one of the most successful cloud-based ERP systems globally. Evolving from the lineage of Baan (Infor LN), Movex (Infor M3) and Lawson (Infor S3), Syteline (Infor CSI) and more. Infor successfully transitioned its ERP system to a comprehensive multi-tenant cloud environment branded Infor CloudSuite supporting different industries.
While customers migrating to cloud-based ERP systems benefit from advantages like reduced hardware requirements, simplified system administration and automated version upgrades, they’re now facing a new challenge. Their data becomes locked within the cloud, limiting their reporting options exclusively to those provided by the ERP provider. This scenario mirrors the past and once again, replication of data becomes necessary. Working with Infor enterprise customers for more than 25 years and seeing this need with some of our customers who are migrating to the cloud, we developed a native solution for data replication from the Infor CloudSuite environment.
Infor Cloudsuite makes data available inside Infor Data Lake, which is part of Infor Data Fabric. Querying the Infor data lake using SQL is possible but can be slow due to several factors inherent to the nature of data lakes, like being unstructured and lack efficient indexing structures. To solve this problem, some customers prefer to replicate data into a relational database/data warehouse for easy and quick access, cutting down the processing time from several seconds to mere milliseconds.
Leveraging on Infor’s data streams, our solution fetches the streams published from the ERP system into the datalake and create a relational database replica, which can locate in another private cloud environment or on-premises database. While replicating data, we can optimize the data warehouse by transforming the data, selecting specific columns instead of all available ones, and creating additional calculated columns.
This data replication functionality is part of B2Win Suite – B2Data, a comprehensive data integration reporting/system that simplify and streamline data integration, data processing and data delivery. Several companies already employ this replication solution, allowing them to access their own data using the familiar and fast SQL approach. While they may opt to utilise B2Data to access the data, they retain the flexibility to choose another reporting or BI system, maintaining the separation between ERP software, the database and the reporting/BI system that companies are accustomed to.
