September 7, 2016 6:45 pm | Updated 6 months ago.
Big Data Analytics and offshore outsourcing
Over the past decade, with increased adoption of mobile devices and social media especially with increasing multimedia content, the volume of data generated by each organization has simply exploded. Every business is putting in place well defined initiatives to be able to analyze this huge volume of data that is now available and generate intelligence that will help companies make better business decisions. The Data Analysts are, therefore, becoming a key part of the decision-making process, to be able to provide accurate data based intelligence that will help the management in taking critical decisions. Even more sophisticated skills and techniques are needed to be able to analyze and interpret unstructured data from multiple sources to create insights and findings, especially using visualization techniques. The Data Scientists, with knowledge or familiarity with machine learning methods, can design or develop software with artificial intelligence so that the program can respond to things it has never seen before and be able to analyze it and present meaningful insights from huge volumes of unstructured data.
With ever increasing volume of Data and Big Data, not only the demand for Data Analysts and Data Scientists is growing rapidly, but the need for specific skill set for these roles and the depth of these skills is also fast increasing. This growing demand in addition to making finding of right skills extremely difficult is also driving the cost of getting the right team of data analysts or data scientists in place for a large project in the USA or Europe to be prohibitive for any regular business.
The companies in the US and Europe are, therefore, more and more looking at outsourcing these functions to offshore vendors who in turn manage large volumes of data from multiple and often heterogeneous sources and provide insights based on these data to their customers thru visualization and/or mathematical models.
We will now briefly discuss what specific skills to look for while outsourcing analytics and big data analytics projects and what risks are involved and how to manage these risks.
The primary responsibilities of Data Analysts and Data Scientists are to:
– Mine and clean the data
– Analyze and interpret results using statistical tools and techniques
– Determine trends, correlations and patterns
– Provide analytics reports and data visualization
The basic skills the customers should, therefore, be looking while outsourcing Analytics to Business Process Organizations (BPO) are:
1. Technical skills in Relational Databases, Data mining, Data Cleaning and munging, Data visualization and reporting techniques, machine learning techniques and some knowledge of Hadoop and MapReduce
2. Business skills in having industry knowledge and analysts with analytical problem solving and creative thinking capabilities
3. Track record of having done similar engagements in the past.
In D2E’ opinion, for Outsourcing of Analytics, the companies need to consider a few additional factors, in addition to the usual factors they consider for outsourcing of any other business function:
a. As the analytical capabilities of a company determines its ability to capture the market share and run its business efficiently, the customers should consider outsourcing only generic analytics and retaining core analytics in-house, until the company is able to grow robust in-house analytical capabilities.
b. The engagement model with the Analytics BPO Partner needs to be developed giving careful consideration to how sensitive is the data and how sophisticated (competitive advantage based) are the analytics.
c. While outsourcing Analytics to an Analytics BPO Partner, adequate attention needs to be given to protection of IP and firewalling of data.
d. With the growing needs for data analysts and data scientists, the customers need to give careful consideration to analyst retention rate of the BPO partner and, if possible, have a few data analysts and data scientists of the BPO Partner dedicated to the customer who can possibly be transferred to the customer’ company at some point of time in future.