Big data crushes the regulators
I have repeatedly blogged (for example, here and here) about the urgent need for financial regulators to get their act together to deal with the big data generated by the financial markets that these regulators are supposed to regulate. The reality however is that the regulators are steadily falling behind.
Last week, Commissioner Scott D. O’Malia of the US Commodities and Futures Trading Commission delivered a speech in which he admitted that “Big Data Is the Commission’s Biggest Problem ”. This is what he had to say (emphasis added):
This brings me to the biggest issue with regard to data: the Commission’s ability to receive and use it. One of the foundational policy reforms of Dodd-Frank is the mandatory reporting of all OTC trades to a Swap Data Repository (SDR). The goal of data reporting is to provide the Commission with the ability to look into the market and identify large swap positions that could have a destabilizing effect on our markets. Since the beginning of 2013, certain market participants have been required to report their interest rate and credit index swap trades to an SDR.
Unfortunately, I must report that the Commission’s progress in understanding and utilizing the data in its current form and with its current technology is not going well. Specifically, the data submitted to SDRs and, in turn, to the Commission is not usable in its current form. The problem is so bad that staff have indicated that they currently cannot find the London Whale in the current data files. Why is that? In a rush to promulgate the reporting rules, the Commission failed to specify the data format reporting parties must use when sending their swaps to SDRs. In other words, the Commission told the industry what information to report, but didn’t specify which language to use. This has become a serious problem. As it turned out, each reporting party has its own internal nomenclature that is used to compile its swap data.
The end result is that even when market participants submit the correct data to SDRs, the language received from each reporting party is different. In addition, data is being recorded inconsistently from one dealer to another. It means that for each category of swap identified by the 70+ reporting swap dealers, those swaps will be reported in 70+ different data formats because each swap dealer has its own proprietary data format it uses in its internal systems. Now multiply that number by the number of different fields the rules require market participants to report.
To make matters worse, that’s just the swap dealers; the same thing is going to happen when the Commission has major swap participants and end-users reporting. The permutations of data language are staggering. Doesn’t that sound like a reporting nightmare? Aside from the need to receive more uniform data, the Commission must significantly improve its own IT capability. The Commission now receives data on thousands of swaps each day. So far, however, none of our computer programs load this data without crashing. This would seem odd with such a seemingly small number of trades. The problem is that for each swap, the reporting rules require over one thousand data fields of information. This would be bad enough if we actually needed all of this data. We don’t. Many of the data fields we currently receive are not even populated.
Solving our data dilemma must be our priority and we must focus our attention to both better protect the data we have collected and develop a strategy to understand it. Until such time, nobody should be under the illusion that promulgation of the reporting rules will enhance the Commission’s surveillance capabilities. As Chairman of the Technology Advisory Committee, I am more than willing to leverage the expertise of this group to assist in any way I can.
The regulators have only themselves to blame for this predicament. As I pointed out in a blog post nearly two years ago, the SEC and the CFTC openly flouted the express provision in the Dodd Frank Act to move towards algorithmic descriptions of derivatives. I would simply repeat what I wrote then:
Clearly, the financial services industry does not like this kind of transparency and the regulators are so completely captured by the industry that they will openly flout the law to protect the regulatees.
Posted at 7:51 pm IST on Mon, 25 Mar 2013 permanent link
Comments