Designing an ODS / DW with high availability and consistency
It's widely recognized that database sizes are growing significantly, and that the growth is being forced by many factors, such as companies requiring more data to be available online for longer (e.g. to comply with government regulations) or an increasing amount of data being digitized for storage. This extent of data explosion has given a momentum for business intelligence application as well. The business intelligence application gathers and stores data for analyzing historical, current, and predictive views of business operations. The gathering and storage of data, on which these business analytics are done, is done by either data warehouse, data mart or ODS (Operational Data Store). Because now-a-days the success of the business is heavily influenced by these business intelligence applications for better and informed business decisions which further rely on data warehouse or ODS for its data feed, it becomes very essential to design a highly available data warehouse or ODS which provides consistent data all the time.
In this video/paper I am going to discuss different approaches (or some of the many available approaches) which you can take to design an ODS for its high availability and data consistency, I will start my discussion with a very basic approach and will list down its pros and cons. Gradually I will move on to the better approach than previous one in terms of its availability and consistency. And finally I provide you some strategic ODS design decision choices and best practices to consider while designing and maintaining it.
Though going forward I will be referring to an ODS design only but same approaches can also be applied for data warehouse as well. You can watch the video or download the deck and article as per your convenience, click here for more details.