Why the Big Data Swamp Needs to Be Purified to Usher in a Revolution in Lending When used by commercial lenders, a data swamp can prove to be damaging to entrepreneurs and small-business owners.
By Michael Carter Edited by Dan Bova
Opinions expressed by Entrepreneur contributors are their own.
In today's modern world, data has become the new gold. Yet, too much data can cause problems of its own, making it increasingly difficult for businesses big and small to understand how to extract insightful and important information from the large stockpiles of data they've obtained. Even more worrying is how this mass of data, when used incorrectly, affects entrepreneurs who are looking to obtain a commercial loan for their business. To better serve business owners and the 28.8 million small businesses that currently operate in the U.S., according to the U.S. Small Business Association, this excess of data needs to be filtered to improve the loan process.
Related: 3 Strategies for Getting Into Lending Shape
In the past, storing large volumes of data was simply cost-prohibitive and impractical. The introduction of distributed cloud-based storage has meant that businesses from every sector are storing all kinds of data (consumer and commercial) -- including when consumers make a transaction, enter a store or even browse the internet -- just because they can. In 2004, I wrote a whitepaper called "The Death of the Data Warehouse" and it would appear that over 10 years later, that has come. This new, enhanced distributed environment has enabled monitoring to occur for every transaction and online information has created murky, burdensome and difficult to maintain bodies of data that for some, are impossible to turn into value.
This vast amount of data has created what is called "data lake": a method of storing data within a storage system or repository in its raw or native format. In other words, think of it as a large pool where companies drop massive amounts of data to be accessed at a later date for analysis. But, data lakes can create mismanaged data sets, turning what was meant to be an easily accessible storage system into a "data swamp" where there is little governance, data is hard to find, hard to use and is consumed (visualized) out of context.
Related: 6 Tips to Navigate New Online Lending Options
When used by commercial lenders, a data swamp can prove to be damaging to entrepreneurs and small-business owners. As Bloomberg notes, traditionally, the data that is used for credit decisions include such items as the applicant's income, employment history and payments records for credit cards and other conventional debts, as well as public information such as bankruptcy filings and foreclosures. Those are the type of factors that go into a FICO or credit score -- the score that dictates whether individuals are able to get a mortgage, auto loan or credit card -- or a rating from one of the major credit bureaus.
Yet today, lenders have started to delve even further into consumer data to assess credit worthiness, including a consumer's email addresses, social media and educational background, raising questions about how this will affect decisions on lending and credit, including privacy concerns and potential discrimination in decision-making. This means that even one misguided social media post can potentially lead to a business being denied a loan, reflecting the arbitrariness of the credit scoring system.
To declutter this plethora of information and create a fair and accurate process for entrepreneurs to receive a commercial loan, technology should be used more effectively to sift through only the most relevant information when assessing a customer. To do this, organizations should explore their existing data lakes and catalog what's inside, creating business rules to validate, match and cleanse their data. Purifying customer data has the power to provide transparency to both borrowers and lenders, giving borrowers more information on the terms other businesses are receiving, while lenders are able to mitigate risk by having a complete view of the businesses' valuation, revenue, capital and collateral.
Related: Busting 5 Myths About Small-Business Lending
At BizEquity, we believe that more innovation through the use of new and better big data technology and analytics will work to help business owners make informed decisions and for commercial lenders to make improved lending judgments. The existing FICO score has proven to be a lagging indicator of real risk, credit and capacity, and incorporating additional data insights into the commercial lending process, like a business valuation of the individual's lifetime value, will only better protect the creditor and provide the business owner further opportunities to develop the business they've worked so hard to create. As data dependency continues to grow, we must continue to drive innovation at a faster pace so that many of our young and upcoming entrepreneurs aren't left behind.
The Holy Grail for lenders is to measure "capacity," that is, the ability of the business or consumer to take on additional credit or debt to grow and create value into the system. Before things like data lakes, and advanced big data insights like business value, this was only a dream. Now, what is needed are a few revolutionaries on the business and consumer credit side to break this dam and drain the swamp.
Related Video: Time for a Bank Loan? What Lenders Are Looking For in Small-Business Owners.
