One of the most hyped areas in modern IT is in the area of “big data”. The availability of Cloud Computing infrastructures and near universal web access is creating new economic possibilities.

The UK and many other Governments are supporting Open Data Initiatives, making public data available in machine readable form. This has multiple purposes. For instance, mandating Local authorities to publish all their external expenditure makes it possible for media, academics and citizens to search for “wasteful expenditure”, putting pressure to justify all money spent. This also opens up beyond the auditors the chance to find errors and problems.

The other key purpose is to allow app developers to create new tools using the open data platforms for consumers, communities and interest groups. This reduces the need for public bodies to create these applications to meet consumer needs.

So this means that “raw data” as opposed to polished statistical tables can be published.

In media, the Guardian has taken a leading stance in data journalism, opening up large data volumes to readers and interesting visualisations. Instead of publishing to an audience, there is a move to co-creation with the readership.

If these initiatives take off, as they appear to be doing in a number of countries, will this start to spread to regulated industries such as financial services?

Finance already has a lot of data in the public space. If financial regulators make more of their data available in open form, what might this do to the financial system?

For a start, there are bound to be many amateurs trying to spot trends, bubbles and the like.

City folk, being competitive types produce lots of league tables and graphs. These tend to be measures as they wish to be represented. I suspect that the opening up of data could create interesting challenges to city practices and culture.

For instance, in mergers and acquisitions there is always interest to see who has been number 1 in terms of the number of M&A cases and the size of the turnovers involved.

A cynic might point out that given that most M&A activity fails to deliver on promised outcomes, then the number 1 may also be the biggest destroyer of shareholder value. In the Open Data world, it would be easy to create apps that listed organisation performance in other ways.

For instance an interesting ratio might be “millions of pounds of shareholder value created per million pounds fee income”. Imagine an AGM where a shareholder asks why X has been appointed to handle an acquisition when X has the lowest ratio in the top 10?

In insurance, a ratio of policyholder return per pound of administration costs might be of interest. In cases like this a 1, 5 and 10 year rolling average might show the trends.

The investor return per trading cost for actively managed funds would potentially be revealing.

Evidence so far from Open Data initiatives shows a lot of experimentation, many failures but a few get to critical mass and flourish. I think that it is almost inevitable that this will happen in areas like finance, energy and telecoms.

A lot of consumer marketing of financial products is in %s, AER, return over bank accounts or FTSE or the like. These are measures that suppliers choose, often in conjunction with regulators. What the government may well find is that the measures by which it is judged or held to account will no longer be of its choosing. I’m not sure politicians will like that much.

In a sensitive area like annuities, the hidden costs against pension funds will undoubtedly be an early target for open data geeks.

At a time where there is a lot of debate about more/less regulation in Finance, the possibility of new models of “Open regulation” could make the City’s relationship with the wider economy and society very different to current practice.

Suspicion following the credit crunch that “they didn’t see it coming” is contributing to a more cynical attitude to “World Class Financial Services”.

In the long run, the court of public opinion, armed with open data may provide the most profound pressure on current practice. Who would be the winners? Who would lose? When “we the people” are the regulator life could be a lot harder.