Big Data vs. Little Data: Part 2 - The Steps to Data Analytics Mastery

In my previous blog, Big Data vs. Little Data: Part 1 - Structured and Unstructured Data, I discussed the two main types of data that should be top of mind for any organization thinking of becoming truly “Data-Driven.” In the world of data and data analytics, credit unions must leverage ALL the data accessible to them but the journey of mastering data analytics can be very tricky.

Determining the right tools will be critical. When it comes to data and data analytics, the order in which you introduce new tools is extremely important. In order to make each step up the analytics curve effective as the last, credit unions must consider the following steps:

Little Data

Step #1: Data Access – The first step in an analytics strategy is simply getting access to the data necessary for analytics. Although this may seem like a fairly easy task, credit unions may find it difficult to get access to the data they are looking for. It may be due to the level of skill needed to the extract the data, difficulty in making the data useful, and/or due to a vendor’s unwillingness to provide the data. Whatever the challenge might be, data access is an extremely important task in becoming analytically proficient and will need to be tackled right away.

Step #2: Data Management – Data management may be the most difficult step in the process of becoming analytically-driven credit union. It is also the most important step. Data management typically involves a data warehouse or data model. The data model is the foundation of all of the following steps: Reporting, Forecasting, Predictive Modeling, and Optimization. Without a solid data infrastructure, analytics is incredibly difficult (near impossible). Although it may not always be visible to the end users, the data model (middleware) is definitely the most important part of data analytics. Side note: a data model should be as granular (transaction-level) and near real-time as possible.

Possible Tool and Resource Need: Microsoft SQL Server, Data Architect, Database Analyst

Step #3: Reporting – Reporting, also known as descriptive analytics, is the most simplistic form of analytics a credit union (or any organization) can utilize. While reporting is possible with tools such as MS Excel, it becomes increasingly difficult as multiple data sources and data volumes are introduced. Performing trending analysis, from multiple different source systems requires a data warehouse and, more than likely, a more advanced data visualization tool (e.g. Tableau) that can handle larger data sets than MS Excel can accommodate.

Possible Tool and Resource Need: Data Visualization Tool (Tableau, MS PowerBI), Report Developer

Big Data

Step #4: Forecasting – Forecasting is the process of making predictions of the future based on past and present data and analysis of trends. It allows credit union employees to make data-driven decisions that reflect the predictions of the future and ask “what if” questions. With the ability to more accurately forecast and run various scenarios, credit unions will be able to better serve their members and improve business processes by allowing them to tailor their daily activities to the organizational goals.

Possible Tool and Resource Need: Report Developer/Statistician (Data Scientist)

Step #5: Predictive Modeling – The ability to predict the future is one step ahead of forecasting. Instead of running a scenario such as “what if our net interest margin increased by X%,” predictive modeling allows the credit unions to actually increase their net interest margin intelligently. The credit union can look back at years of history and predict future risk of their current loan portfolio and a member’s likelihood to default in addition to what they already know using FICO. Big data tools and other relational database tools must rally around a predictive model to bring predictive analytics and automated decisions to credit union leaders. The new CECL regulation will require predictive modeling.

Possible Tool and Resource Need: Big Data Tool (e.g Hadoop), Data Scientist

Step #6: Optimization – As decisions are made around the predictive analytics, patterns will emerge and the tools will be able to recognize them (machine learning). As the tool and data scientist “studies” the history of trends and patters found in the predictive data model, the model will optimize decisions based on previous results. For example, if a marketing campaign targeted at certain members produces a new set of data, the tool will bring this data into the data model and optimize the predictions it interprets from the data.

Possible Tool and Resource Need: Big Data Tool (e.g Hadoop), Data Scientist

Analytics is a journey. It requires several processes and ordered steps that cannot be skipped. It is often thought that the better the “tool” you have, the better you analytics will be. While this may be true, there needs to be an order in which these various tools are rolled out. Data infrastructure or the data model is the most important step but is often overlooked because it is middleware. Without it, tools such as Tableau and Hadoop cannot assist in their part in data analytics mastery.

After establishing the aforementioned steps for data master, determining the right tools will be critical. It is very important that Little Data tools (SQL Database / Data Warehouse / Data Visualization Tool) are connected to, or compatible with, the Big Data tools (Hadoop, Cloudera, and MongoDB). If the data warehouse does not support the move from structured data to unstructured data there will be a serious loss of value. While both Big Data and Little Data are extremely powerful, the marriage of the two is where the real value lies.