"New Project Thinking"

My personal experience is that many organizations tend to make similar mistakes, which I decided to sum up as 6 Big Data Success Anti-patterns (written a bit tongue-in-cheek). Follow this advice at your peril:

• Replace strategy with a quest to gather more data. “If <insert the name of your favorite successful internet company> is doing great by collecting more data, then the same must apply for us! At the very least, I want us to have more data than our main competitor.”
• Start with the data first. Analyze it for smart answers afterwards. Then set the questions.   “I’m not sure what data we need, so let’s collect as much of it as possible and we’ll analyze it in-depth once we have it. I’m sure it will tell us something that will be useful to the business.”
• More and newer technology is always the answer to past problems. “I know that our old data warehouse project was not that successful. The problem was that it is too small and too slow. If we want to make this work, we must have <insert the name of your favorite Big Data technology, such as IMDB, high-speed storage appliance, etc.>”
• Knowledge is power. Too much of it is a dangerous thing.  “Some management consultants told me that one should have a culture of sharing insight and decision-making throughout the organization. Let’s upload this report on the intranet, but filter it first for sensitive data.”
• Never mind those new data skills:  “association rule learning?, classification cluster analysis?, crowdsourcing data fusion and data integration?, machine learning?, natural language processing?, pattern recognition?, predictive modeling?, sentiment analysis? Bah! Humbug!”
• Speed is not that important. “Yes, I know that the window of opportunity is narrowing but it is overambitious to match the velocity of data to how fast the business processes needs to act on it. Shortening the whole data cycle from collection to insight to action to anything less than a few weeks is impossible!”

Of course, I’m exaggerating more than a little bit, but my point is a serious one: most companies are no longer suffering from a lack of data; they’re suffering from a lack of the right data. It is no longer only about the size of your data, but the real opportunity in the velocity of the exploration methods that will let humans visualize, understand and act faster, or even better, and the methods that will automate the process (for example, one of our customers is analyzing customer sentiment of the support experience, predicting customers expected to be dissatisfied and automatically initiating proactive customer management…all in a very short period of time).

Remember: finding value in Big Data is a discovery process, which requires a different “project thinking” where the journey is as important as the destination.  Make sure your journey is successful!

Often it is less important to know about a document than what a document is about – and like any piece of modern art – you are likely to get as many different answers as the number of document examiners.

You could ask why it is important to know what a document is about. “About-ness” (which is a completely made up word) is a near canonical expression of the information payload of a document. This metadata enables documents to be categorized, inter-related and even found via the services of a search engine. Organizations enamored by the power of Bing and Google often tend to overlook the simple fact that leading search engines have engineered the process of extracting “about-ness” from a piece of content (document, webpage etc.) down to a science – whereas most organizations still carry legacy / irrelevant information in their document titles. A first step to getting better data via search results starts with defining common and relevant business concepts that can be used to define the set of “abouts” for a given document and being disciplined in either identifying these concepts at the time a document is submitted to a repository or enabling automated extraction of concepts that will be used to define the set of “about-s”.

Think about the days before the zip code. How did anyone get any mail to someone across the country … and how long it took! Now look at the USA after the zip code – and mail zips across the continent. One can even tell what other zip codes you are next to (ok – that’s called ambient findability – and more about that on the next post). However, the zip code is a completely synthetic entity, which becomes part of the “about” set for your address and enhances its findability. At Avanade, we have specialized and evolved the art and science of identifying business concepts and entities that could be used to enhance the findability of data and content and finally get search working for the enterprise.

Our experience has shown that the leading search software packages are feature-comparable at scale. What differentiates the end user experience is the level of investment the host organization has made in designing and maintaining the ontology of terms / concepts that define the semantic web of content their information workers must navigate to get their jobs done. While the process is not without some pain – the end results are often more than worth the investment. In my upcoming posts, I shall examine some best practices from the field and the collective learning Avanade has acquired in establishing true search-powered knowledge cultures.

We’re at a tipping point right now where a widespread base of companies and employees can and should be leveraging data for concrete benefits. To obtain true value from data insights, it’s important to remember that big data is everyone’s business. Employees should feel empowered to make real-time decisions.

The good news is we’re seeing progress here. More than half (59 percent) of business executives and IT decision makers indicated that more employees than ever before are involved in making decisions as a result of more widely available data. This finding comes from a global global survey Avanade conducted on the business returns of big data. You can check out my blog last month for more on these findings.

But, as I mention in the Forbes article, there are still challenges that must be overcome. The rise of big data reveals shortcomings in the talent to manage it and the willingness to make educated and informed decisions from it. Tom Groenfeld, a contributor at Forbes, said it best in a recent article, where he asked if big data was destined to flop like CRM. “Big data isn’t going to solve failures to execute on service,” Groenfeld stated. And he’s right. It’s time for companies and employees of all levels to take action.

Big data is a serious and challenging business problem. On May 24, The Wall Street Journal’s CIO Journal wrote a story stemming the MIT Sloan CIO Symposium which said that while the challenges of big data are real, so is the opportunity. Michael Chui from McKinsey & Company said “Big Data will be effective for every segment of the economy … companies will gain or lose market share based on their ability to do this, and those who start earlier will accelerate their advantage exponentially.”

Avanade’s recent global research study confirms Chui’s point. We found that companies are starting to realize tangible business value from big data. In fact, 43% of business and IT executives said that big data helped them create entirely new sources of revenue and the vast majority of executive respondents say they are making better decisions.

Another finding from our research is that access to data is becoming pervasive in organizations. The majority of respondents (58%) to our survey report data management is now embedded throughout their business operations and not just housed within IT. We’re seeing interest from our customers in making data more widely available to all employees. In fact, for SaaS-delivered business intelligence (BI), we just signed a deal with a customer that will roll out analytics-as-a-service to 150,000 users. Truly data for the masses.

Despite the improvement of tools to manage the increased volume of data, enabling employees to find the right information at the right time still remains a challenge. And as CEO, this is something that I often get asked about by our Avanade employees. We have a very current and sophisticated intranet and powerful search functionality; however, it’s not the tools but the volume that makes finding the right information difficult. Over the past 12 years we have been conditioned to make information available in digital form, at our fingertips. So, the data is there and searchable, but that doesn’t make it less challenging to find what you need when you need it.

The phenomenon of Big Data is still relatively new, and as McKinsey’s Chui said, “there aren’t best practices; I’d say there are emerging next practices.” You can read our Executive Summary of our research to see what Avanade recommends companies start to do but I would love to hear from you – what are you seeing and doing to tackle this challenge?

Let me end by fulfilling my promise of an acronym. Given the topic at hand, I offer up to you for consideration, WITWIMD – Where in the world is my data? It’s big and it’s a mouthful. But if you have a better acronym (keep it clean), please share it with me.

• More people have access to technology putting more data in everybody’s hands
• Investments are paying off and big data has begun to create real value
• The benefits of big value come with meaningful challenges

The most forward-looking companies are empowering more people across the enterprise with the tools and skills needed to take advantage of big data. Our survey shows that the technologies and skills used to leverage big data in the enterprise have reached a tipping point.

Beyond tangible business value, evidence shows big data has become pervasive – more types of employees have more access to more technology options to manage and analyze data. Today, 58 percent of respondents say data management is now embedded throughout their business operations. And, the majority of companies surveyed (57 percent) said in just the last 12 months, more technology options became available to analyze and manage data.

Companies are also investing in that technology – almost every company surveyed (91 percent) is using tools to manage and analyze data today. Looking ahead, 75 percent of survey respondents say that their company will make additional investments to improve their ability to analyze data within the next 12 months, including technologies such as predictive analytics, mobile data access and management tools and cloud-based storage.

On the other hand, 85 percent of respondents still report obstacles in managing and analyzing data and 63 percent feel their company needs to develop new skills to turn data into business insights. Further, changes in the way we work – driven by trends such as the consumerization of IT – is only making things more challenging. Our survey showed a trio of trends – employee mobility, cloud computing and social networking – causing companies to rethink their data management strategy.

While there is still work to be done, it’s clear that companies are beginning to see real value from big data. I encourage you to visit our our web page that includes a link to the full report and review our Big Data Infographic.

As big data has risen to the top of the agenda in 2012, there have been many definitions thrown around in an attempt to truly grasp what big data entails. One common definition is around the combination of volume, velocity and variety. Volume refers to the enormous amount of data generated and collected by organizations. Velocity means the speed at which the data must be analyzed. And, variety refers to the array of different types of data that is collected. Many analysts and reporters have claimed that the combination of these three elements produces big data.

While I agree that these three pieces are important when defining big data, I don’t believe it’s a requirement to have all three categories to get what we call “big data.” For example, if you have volume and velocity or variety and volume, etc. you are still dealing with big data. It’s the intersection of these areas that create the big data challenge – and opportunity.

When identifying sources of big data within an organization, I think it’s less about conforming to a particular definition but rather discovering what pools of data you’ve forgotten about. Are there pockets (it doesn’t always have to be massive volumes of data) of unstructured data that your company is not currently analyzing? Have you merged structured and unstructured data to create business insights? Can you pull these smaller data ‘puddles’ into one bigger pool of data to leverage in a new way to solve an old problem?

Think about utilities as an example. As more homes and business move to smart grids, the challenge in optimizing consumption is that everyone needs to know their energy consumption in real-time in order to make decisions. If every meter in a city is sending 200 characters of data from each location every 15 minutes, that’s a lot of data. Most often, CIOs end up storing this data and haven’t yet figured out how to deal with it.

As more tools and technologies become available, the barrier to gaining insights from big data is lowering. Companies need to find a way to expose those forgotten pools of data, filter out the irrelevant information and empower staff with the right analytics to make sense of it. The key is that there shouldn’t be an ivory tower of data scientists any more – business intelligence needs to be distributed throughout the company in order to gain true value.

At Avanade, we are seeing signs that the industry is shifting from a defensive to an offensive attitude in terms of how they approach and respond to big data. Next week, I’ll share key findings from Avanade’s latest global study which shows clear value from big data.

For example, think about the changes taking place in terms of how we work based on these trends:

• As tablets and smartphones continue to permeate our lives, we’ll do more and more work on high-res displays – moving them away from a pure entertainment feature.
• And, these tablets will always be connected securely to cloud applications, which will be delivered via HTML5. And, these applications will be integrated into the fabric of your company and customers, connecting them to enable greater efficiencies, better customer experiences, etc.
• All of the documents and information employees store and share will also be in the cloud. This business-ready storage, along with increasingly mobile workforces, changes the concept of the ‘desktop’ quite radically.
• At the same time, this new fabric of connecting employees and customers will essentially drive enterprises and customers to become their own social networks, which will drive its own type of social analytics.
• With new social data streams and the continued growth of big data, predictive technology will become even more crucial for business decision-making.
• We’ll also begin to see voice (or speech for business) rise dramatically in the enterprise. Computing power will be more accessible and it will allow new types of collaboration and connectivity within businesses.

Even though we’re only a few months into this year, many of these technologies are already in your business in some form. The question is how will your company and IT department manage these changes? As the Inc.com article says, “Whether these trends will force you to change how you do business, or present you with entirely new opportunities depends on how willing you are to be an early adopter and take risks. You’ll have to find room in your IT budget for these things, too.”

My first thought after reading this – wow, this list is long. Today’s IT landscape is filled with numerous technology trends organizations can choose from to develop business priorities. Business success often means leveraging the right technology, in the right order, to move your business forward. But how do CIOs keep up with these long lists of tech trends?

Out of the many IT choices out there, I think you can categorize the myriad of trends into four key areas that are driving the industry:

1. Cloud Computing: When we talk cloud, it is important to note there are several different types of cloud experiences. There are private, public and hybrid clouds. There are clouds for infrastructure, platforms and applications. Lots of choices. Ultimately, companies need to build the strategy and roadmap that helps them focus and delivers the most value to accelerate their business. It’s about finding which deployment model works for you and what pace of change you can tolerate.

2. Big Data: Executives that we’ve surveyed tell us that they have an insatiable desire for more data. And, the media has definitely shown that to be true. Companies are addicted to data. But, whether executives can ride the wave of big data to a more successful business depends on how they use their data.

While creating new data is easier than ever, the challenge is acting on, and generating value from that information. Collecting data from existing and new sources, storing that data, sharing it across networks and continents, searching and analyzing it becomes increasingly more complex as the amount grows exponentially. Unless you’re one of the very few companies who have mastered the lifecycle of moving data into insight, there’s still progress to be had.

3. Consumerization of IT: The enterprise workplace environment is being reshaped as the boundaries between corporate and personal lives blur through the collision of overlapping devices and applications. Avanade recently released the findings of its latest global survey that dispelled several myths around the topic of consumerization of IT – including employee device preferences, hesitance of business leaders to embrace the trend and executive perspectives on Millennials. Despite the notion that business leaders are resisting the shift, we find that companies are investing in staff and resources to enable greater worker productivity that the consumerization of IT enables. In 2012, the focus will be on integrating and optimizing personal technologies for an enterprise environment.

4. Natural User Interfaces: Consider this: the QWERTY keyboard was invented in 1874. Yet more than 100 years later, we are still using this same design. While technology has become faster, cheaper and improved productivity greatly over the past decades, human interfaces have lagged behind. Now, new user interfaces and experiences present the opportunity to improve the last mile between our brains and all this computing power.

Natural user interfaces transform the way that we interact with digital information, collaborate with each other and serve customers. Right now, companies are focused primarily on experiences geared for the consumer market, centered on web, mobile and touch. As the technologies continue to mature, we expect these rich user experiences will move further into the enterprise, tailored to address industry specific and employee needs.

While longer lists are out there, I think most predictions are just a subset of these four major topics. For example, visual communications is a type of natural user interface and social business is really about consumerization of IT. If you focus on these key areas and have a clear point of view for them (even if the point of view is to wait and see), that’s enough for now. As we consider these four trends, what are your key priorities are for 2012?

The Information Technology (IT) landscape is constantly changing and evolving. Before the Internet and social networking, I relied  heavily on computer publications like InfoWorld,  Computerworld,  Digital Review, Network World, PC Week, PC Magazine,  PC World and the ever voluminous  (400-600 pages) in-depth Byte magazine for information on the latest computer  technologies.

During the pre-Internet era, I was very proactive in keeping abreast of the latest technologies. I read every issue of the venerable printed computer magazines from cover to cover, regularly mailed off product literature request cards and routinely reviewed the literature I received. This wealth of knowledge helped me to have a great deal of influence over IT decisions made by my employers and my consulting clients throughout the early 1990s.

Some of these print magazine successfully moved to the Internet  and continue to publish “digital only” versions. A few, in addition to their  digital version, still publish thin incarnations (<50 pages) of the once-thick content-filled magazines of the 1980s and early 1990s while others  like Byte magazine failed to make the transition to digital media.

Internet Era

While all of the online computer magazine and IT communities are useful, they have failed to leverage the true power of the Internet and modern web development, content delivery and search technologies. Where is the engaging, interactive, targeted user experience? Most of digital magazines are poor versions of their former hard copies, overloaded with annoying and distracting advertising, inadequate search capabilities and unintuitive navigational tools.  The content also seems to lack the in-depth coverage once reported in earlier print magazines.

“What we got here is… failure to communicate.”¹

My popup blocker works overtime when perusing these sites.  While I understand advertising generates revenue, which in turn funds these sites, does it make sense for it to be so intrusive that it drives potential viewers away? But, the worst offense of these sites is the article that ends the first page with, “To continue reading please register …” Really? Imagine, reading an article in a printed magazine on a plane and at the end of the first page you read, “Call 1-800-The-Rest to receive the rest of this article.” With many of these sites, once you submit your registration information you unknowingly, unless you read the fine print, agree to receive email communications from the  site and its advertisers. If registration is a requirement then at least allow me to create a detailed profile that could then be used to target the content displayed to my specific areas of interest.

Information Overload

Today more than ever, IT professionals are inundated with unprecedented change from all directions. Some of these are quite sweeping and pervasive like cloud computing, virtualization, mobile computing, social networking and the consumerization of IT. Others like Big Data, along with data and application integrity, reliability and security will continue to be major concerns in the coming years. These  technology trends and major shifts (e.g., NDS to Active Directory, Lotus Notes to Microsoft Exchange,  Blackberry to iPhone and Android, etc.) can be career altering for the IT professional that falls behind on the trends or is reluctant to embrace a new technology. Even with the Internet, it has become ever increasingly difficult to keep up with the pace of IT innovation and the latest trends.

The Internet has allowed IT professionals to become more lackadaisical and less proactive when it comes to keeping up with the ever-changing IT landscape. Why stay well-informed when you can conduct a Google search and quickly learn about the latest products, trends or the latest buzzword your boss asked you about? Today’s IT professionals have a plethora of information instantly available via the Internet yet accessing the most  relevant information is still a conundrum. A Google query of “Hyper-V” returns 36,500,000 results (Bing returns over 7,000,000). A more specific Google query of “Windows Server 8 Hyper-V” reduces the count to 9,700,000 results. So, how does one stay current and keep up with the technological changes that are most relevant to their role within their organization without spending endless hours perusing the frustrating online IT magazines, communities and blogs?

Social Networking Era

For me the solution has been a combination of methods. Like most, I do rely on Internet searches for information, but these are more ad hoc type queries. I still read a few printed magazines like Windows IT Pro and Processor. Over the past few years, I also used a variety of methods and resources like Google Alerts and email newsletter subscriptions to keep abreast of IT trends, but found these to be little improvement over regularly perusing the digital magazines noted above. My current favorite method of filtering the IT information overload down to the technologies, vendors, products, blogs, pundits and thought leaders relevant to my job is to use Twitter and an iPad app called Flipboard.

Flipboard is a social magazine application that was awarded Apple’s “App of the Year” award in 2010. The application collects the content of social networks (e.g., Twitter) and other websites (i.e., CNET) and presents them in an “elegant” magazine format, that puts to shame all online digital magazines, and presents it on the iPad, iPhone or iTouch. Using my Twitter account, I simply created several Twitter Lists that follow the Twitter accounts relating to specific technologies relevant to my job. Most digital magazines, communities, bloggers, thought leaders and vendors have Twitter accounts and some have several that are specific to particular technologies or groups within their organization. Then, I associated my Twitter Lists with my Flipboard and voilà, instant personalized IT magazines showing just the content from the resources that I’ve chosen to follow.

If you’re not using Twitter or other social networks to monitor IT trends, I would strongly recommend you consider taking the plunge. Twitter is not about Tweeting your personal status, but a method to connect with digital media, vendors and other IT professionals with similar interests. I also use Hootsuite, a social media dashboard to view and manage my social networking accounts. I would love to hear from readers how you are leveraging social networking to keep pace with the ever-changing Information Technology landscape.

¹ Gordon Carroll (Producer), Stuart Rosenberg (Director).  (1967). Cool Hand Luke. [Motion  Picture] United States