Big Data, Internet and Capital Market
The Indian capital market has transformed over the past two decades with the setting up of SEBI and later NSE and other Exchanges. With over 25 million direct participants, it is one of the largest capital market ecosystems in the world. The market has moved from an outcry system to tech enabled trading system. The capital markets have a wonderful tradition of funding the capital requirement of entrepreneurs. There is an evolved ecosystem existing across the length and breadth of the country that supports the market structure to meet the growing needs of the corporate world. India has been a predominantly bank loan financed economy. The economy of India’s size and potential requires markets to fuel the growth engines. We are already seeing early signs of transformation with close to Rs.6 lakh crore of debt financing being raised through markets over the past couple of years. Read on to know more…
This pace of the capital market growth is expected to gain momentum as the regulators viz, RBI and SEBI, initiate a series of transformation measures to transform the fixed income markets in India as efficient and widespread as the equity markets. While electronic trading took investing to the investors’ homes, the new age tools riding on mobile / smartphone penetration will truly democratise investing for everybody. It is expected that on the back of this new age revolution equity participation levels would surpass those in advance markets even in value terms.
In the background of this ever changing business landscape, particularly after the financial crisis of 2008, wherein regulatory reforms threw up new challenges for the capital market industry affecting the revenues and increasing the cost pressures, the market was grappling for overcoming the challenges. The solution for many of the issues lies in leveraging the available data. The capital market industry has huge piles of data and the processing of these data using Big Data analytics is expected to be a game changer. The solutions that could emerge will help in revenue maximisation and cost optimisation.
Big Data
The capital market industry has a vast pile of data which comprises of structured data like transaction data, reference date and market data. It also has unstructured data like corporate filing, news feeds, social media chatter, macro / micro economic data indicators, etc. The traditional tools cannot efficiently process such large data sets. However, Big Data tools can sift through massive data warehouse that are present in modern business environment. The Big Data strategy tend to relate to speed up the process and the ability to handle large volume of data such as column-oriented or schema-less databases. To this end technologies such as Hadoop, MapReduce, Not only SQL (NOSQL) are most often cited as examples of Big Data in action.
Researchers have categorised Big Data based on the following;
- i) Volume – The amount of data that is processed or managed
- ii) Velocity – The speed of data delivery and processing
- iii) Variety – Types of data like text, audio, video, structured, unstructured etc. and
- iv) Veracity – The quality and integrity of data being processed.
Big Data strategies have now begun to impact a select few areas in capital market over the recent years including sentiment analysis in trading, risk analytics and market surveillance. Data management is now a strategic function within most financial institutions.
In order to conceptualise the scale of a Big Data performance we are talking in terms of Petabytes or Exabytes. One petabyte is 1000000000000000 byte of digital information and it is enough to store the DNA information of the entire population of USA. An Exabyte is 1000 Petabyte.
“Big Data strategies have now begun to impact a select few areas in capital market over the recent years including sentiment analysis in trading, risk analytics and market surveillance. Data management is now a strategic function within most financial institutions.”
Big Data projects can be focussed on:
- a) Client relationship management
- b) Market data
- c) Trading
- d) Post-trade processing
- e) Risk management
- f) Surveillance and
- g) Research for revenue optimisation, cost reduction and reporting
There is an obvious benefit of using cloud computing as a solution to support a Big Data strategy. By using cloud, Businesses are able to scale up or down on a “pay as you go” basis rather than being reliant on existing IT resources. Thus, typically data strategies can be applied to a whole range of functions starting from the front office trading to the back office processing, surveillance, reference data and support.
Trade Analytics
A good example of the revenue generating intention of Big Data analytics is in sentiment analysis. The Big Data strategy can be used to gather and process information surrounding specific markets to create a clear understanding of sentiment that drive front office trading strategies as well as determine the valuation of individual securities. Using this information traders are able to determine whether various market participants and commentators including those on social network such as Facebook, Twitter, WhatsApp, and Blogs etc. are bullish or bearish and can then frame their investment and trading strategies. Further, the sentiments and news surrounding the company can be incorporated into the valuation methodology to produce a fundamental price for the security issued by the company. Comparing this fundamental price to the market price, the investors can effectively gauge whether a security is under-valued or over-valued and thereby open up opportunities for arbitrage trading.
“A good example of the revenue generating intention of Big Data analytics is in sentiment analysis. The Big Data strategy can be used to gather and process information surrounding specific markets to create a clear understanding of sentiment that drive front office trading strategies as well as determine the valuation of individual securities.”
Research
Being able to provide high quality research is a service that can help strengthen the client relationship and generate revenue. Combining natural language and machine learning inputs with unstructured data engineering across economic reports to the monetary policy changes and political events, the quality of research is much improved. The research is converting the manual knowledge-based work to an automated work thereby finding answers in quick time which has helped the organisations to increase their revenue and optimise the cost of research.
Surveillance
Regulators across the world have tightened their oversight and have agreed on key valuables for oversight. However, it is noticed that depending upon the regions, the regulatory designs keep varying. Thus large capital market firms with a cross border presence face a series of regulations starting from Dodd-Frank Act, FINRA guidelines, Basel III, MiFID II, Solvency II, FATCA, EMIR, UCITS IV and FRS9 standards. Thus Big Data is gaining a strong hold due to the increased scrutiny of data quality in regulatory and ad-hoc market report as well as the need for speed and accuracy. For example, the Dodd-Frank regulation requires trade reconstruction report for regulatory investigations within a 72-hour period and this data may be residing in unstructured formats such as voice or text, making Big Data analytics imperative.
“Large capital market firms with a cross border presence face a series of regulations starting from Dodd-Frank Act, FINRA guidelines, Basel III, MiFID II, Solvency II, FATCA, EMIR, UCITS IV and FRS9 standards. Thus Big Data is gaining a strong hold due to the increased scrutiny of data quality in regulatory and ad-hoc market report as well as the need for speed and accuracy.”
The surveillance could also be internal and in areas related to fraud, AML / KYC, unauthorised trading, market and credit risk, employee surveillance and also in continuously monitoring asset performance in businesses that are delivering a higher risk adjusted returns of capital investment. Data analytics can accurately identify risks and exposure across a trade life cycle and the organisation. The trade surveillance uses pattern based analytics to identify front running and insider trading by calling information from various sources and feeds. Similarly, AML and fraud detection can be dealt by developing customised models based on pattern identification, risk score and profile based analysis, etc. Effective reference data management helps in developing “golden copies” or single version of the truth, thus improving the quality and accuracy of data and thereby reducing the need for manual intervention.
Pitfalls and Challenges
One of the biggest reasons for the failure of Big Data projects handled in the past has been the lack of compatibility between capital markets business requirements and the capability of Big Data technology.
Firms also need to prioritise their data storage strategy placing the data sets of most important on faster devices and other sets to be less readily accessible but more cheaply stored.
The vast majority of capital markets is cautious about the use of public cloud technology in commercially sensitive areas due to data privacy issues. Hence some of the firms use private clouds for Big Data analytics.
“The vast majority of capital markets is cautious about the use of public cloud technology in commercially sensitive areas due to data privacy issues. Hence some of the firms use private clouds for Big Data analytics.”
The other factor is that inadequate technical knowledge affects a Big Data project highlighting the need for appropriately qualified staff to implement the Big Data strategy. Further, data privacy is also a major area of concern for Big Data projects. Regulations relating to data privacy varies from country to country and hence this needs to be carefully calibrated before embarking upon the Big Data strategy.
Conclusion
A multitude of transformation and growing complexity of the capital markets is making it imperative for corporates to start using Big Data analytics. When it is implemented in its full scope, Big Data can help institutions to not only go beyond improving risk management, reporting compliance and operational efficiency but also to devise better pre and post trading methodology. This is a win-win solution for all the stakeholders of the capital market viz an investor, the issuer and the regulator.