The web is abuzz with analyses on Google’s real time search effectiveness assessed in the context of the death of an Hollywood actress today. The question there is how real time is realtime? Google has demonstrated through algorithmic means(without human intervention), the capability to report results in a search query in less than 3 mts(Read Google’s Matt Cutt’s comments therein!) after the reporting of the event! What an amazing progress shown by Google here!
Recently, Morgan Stanley brought out the fact that the mobile Internet is ramping faster than desktop Internet did, and believes more users may connect to the Internet via mobile devices than desktop PCs within 5 years – this is pushing things hard for service providers. Massive mobile data growth is driving transitions for carriers and equipment providers. This is slated to increase over time with emerging markets embracing mobile technologies much faster and pushes the material potential for mobile Internet user growth. Low penetration of fixed-line telephone and already vibrant mobile value-added services mean that for many EM users and SMEs, the Internet will be mobile.
As the MIT journal notes it, both Google and Microsoft are racing to add more real-time information to their search results, and a slew of startups are developing technology to collect and deliver the freshest information from around the Web. But there's more to the real-time Web than just microblogging posts, social network updates, and up-to-the-minute news stories. Huge volumes of data are generated, behind the scenes, every time a person watches a video, clicks on an ad, or performs just about any other action online. And if this user-generated data can be processed rapidly, it could provide new ways to tailor the content on a website, in close to real time.
Many different approaches are possible here to realize this opportunity – currently many web companies already use analytics to optimize their content throughout the course of a day. The aggregation sites , for example, tweak the layout on their home page by monitoring the popularity of different articles. But traditionally, information has been collected, stored, and then analyzed afterward. The MIT journal article correctly highlights that using seconds-old data to tailor content automatically is the next step. In particular, a lot of the information generated in real-time relates to advertising. Real-time applications, whether using traditional database technology or Hadoop, stand to become much more sophisticated going forward. "When people say real-time Web today, they have a narrow view of it--consumer applications like Twitter, Facebook, and a little bit of search," Startups are also beginning to look at different technologies to capture and everage on current data and patterns therein.
Last month, in a very nice article, The Economist highlights, thanks to Moore’s law (a doubling of capacity every 18 months or so), chips, sensors and radio devices have become so small and cheap that they can be embedded virtually anywhere.
Today, two-thirds of new products already come with some electronics built in. By 2017 there could be 7 trillion wirelessly connected devices and objects—about 1,000 per person. Sensors and chips will produce huge amounts of data. And IT systems are becoming powerful enough to analyse them in real time and predict how things will evolve.
Building on the smart grids space – wherein colossal waste in transmission , distribution and consumption could get positively reduced, the Economist notes that it is in big cities that “smartification” will have the most impact. A plethora of systems can be made more intelligent and then combined into a “system of systems”: not just transport and the power grid, but public safety, water supply and even health care (think remote monitoring of patients). With the help of Cisco, another big IT firm, the South Korean city of Incheon aims to become a “Smart+Connected” community, with virtual government services, green energy services and intelligent buildings.
If one were to look at a different place –inside the enterprises, collaboration is enabling people coming together more easily and readily than before – but the upside potential there is very high with possibilities like workflow getting baked natively into documents enabling real time communication amongst the stakeholders.
What is that technology which would enable all this to happen covering data, transaction, content, collaboration, streaming etc inside smart enterprises? It is Complex Event Processing or CEP, a technology in transition- hallmark of any maturing technology and more importantly its potential – as a precursor to enabling enterprises to get ready for more sophistication in autonomous realtime decision making..
At its elements – CEP collects data from various sources and prime raw events & then applies smart algorithms(this learns with time) and invokes right set of rules to decipher patterns in real time – complex scenarios and distills trends therein and churns out results enabling real time decision making. Smart enterprises can ill afford not having such technologies deployed internally. IBM calls this smart planet revolution- all tech majors ranging from software players to infrastructure players want to have a decisive play herein. Every utility service, all competitive sectors like transport and retails etc are looking at adopting this technology aggressively in areas like supply chain management, finance optimization etc.
But smart infrastructure alone won’t be sufficient to make organizations leverage this – it calls for smart thinking at process levels and at decision making rooms. Today information overload is almost choking enterprises and to an extent individuals.
Existing architectures centered around RDBMS and modern web services/SOA are not equipped to handle this nature of data, range, volume and complexity. While processing data from very diverse sources, CEP’s can model data, have native AI capabilities that can be applied while processing event streams. With the right executive interventions and decision makings, smart enterprises of the future may have the wherewithal and power to divide and conquer where needed and synthesise various streams of info in real time in the right way. Such a process would unlock the proverbial hold of central decision making within enterprises and allow decentralized autonomous decision making at real time to maintain competitive edge in the marketplace.
This would lead to more and more of adoption and the resultant complexity would force technology to improve more rapidly –the ideal scenario is that the complex mesh that an enterprise is would get more and more opportunity to respond with real time data at all its nodes making a smart enterprise implementing armed with such solutions to be a veritable force in the marketplace. There is no more the luxury of building cubes and waiting for analyses when real time forces can torpedo any strategic plan with rapid killer response from market forces – that’s where CEP fits in very well – providing real time digitally unified view of various scenarios and their overall effects – enabling much more rapid response in real time. Remember the saying –“the CEO can’t even blink for a moment”
When you look at the key google labs projects (courtesy – CIO magazine) – you cant ignore the Google Goggles project or for that matter, Google’s voice to message to voice translation network -these are all precursors of complex things processed in CEP world –took google as an example as this company wants to solve really big and complex problems as part of its charter!
Real business problems seeking new technology solutions and emerging technology solutions maturing to serve increasingly complex business problems are the perfect way to catalyze growth in new/emerging areas and here we see that playing together well. Forward looking plans, simulation, optimization are all a direct derivatives of good CEP solutions and with such solutions deployed well – we can see the emergence of new digital nervous system inside enterprises and in some cases can trigger creation of new business models itself.