Finally leading edge Vendors are seeing the real challenge in BI

Excellent article in Wired this week

In discussing deliverables from BI solutions, the article discusses the requirement in BI deliverables to allow people to then interpret the results with not factual (read best out-of-box thinking) but interpretive approaches

“However accurate the data, however good the visualizations, however well laid out, the dashboard will be incomplete. Yes, benchmarks can provide a very useful high-level synopsis. But you should also provide your decision makers with tools that enable them to explore the data themselves. If they can try out the effect of alternative selections they will feel much more confident in the data itself. If the tool enables them to pick up good “information scent,” suggesting new trails of reasoning to follow, they will lead themselves to useful insights.”


Follow the link at “”


Why Apple, eBay, and Walmart have some of the biggest data warehouses you’ve ever seen

By Derrick Harris

Summary:Teradata has been around forever, and its customer base full of huge companies suggests it will probably for a while to come. Here’s how some of its customers use the company’s analytics software. 

In an age of Hadoop and a general analytics revolution, it’s easy to poke fun at legacy data warehouse vendors such as Teradata. Some people might even call it fun. After all, they sell expensive appliances and weren’t built from the ground up to handle the unstructured data that most people think of when they think of “big data.”

But whatever you think about Teradata’s approach to handling big data workloads, make no mistake about the company’s clout: It has been around for decades, and it’s still analyzing boatloads of data for some of the biggest names in business. I spent a day in February touring the Teradata Labs facility in San Diego, and although I heard all about the technology and the company’s vision for a Teradata-Hadoop-Aster analytics super-environment, the thing that stuck out most were the users. Walmart, eBay, Continental … Apple.

Here’s how they’re all using Teradata and at what scale (try not to faint when you think of the bill):

  • Apple: Apple is operating a multiple-petabyte Teradata system (that became apparent during its iCloud launch in 2011) and, I learned, was Teradata’s “fastest ever customer to a petabyte.” Apple uses the data warehouse to get a better understanding of its customers across product groups. Now every piece of identifiable information — and those iTunes interactiona generate a lot of data — goes into the system so the company knows who’s who and what they’re up to.

Rows of Teradata appliances.

Rows of Teradata appliances.

  • Walmart: The retail giant deployed Teradata’s first-ever terabyte-scale database in 1992, and it has grown, uh, a bit since then. Its operational system was at 2.5 petabytes as of 2008, and is certainly leaps and bounds bigger by now — likely well into the double digits when you consider it operates separate ones for Walmart and Sam’s Club as well as a backup system. The analytics efforts have essentially helped Walmart become a massive consignment shop. It tells suppliers, “You have three feet of shelf space. Optimize it.” And then it gives them any data they could possibly need to determine what’s selling, how fast and even whether they should redesign their packaging to fit more on the shelves.
  • eBay: eBay (e ebay) has two systems in place, and they’re both big. Its primary data warehouse is 9.2 petabyes; its “singularity system” that stores web clicks and other “big” data is more than 40 petabytes. It has a single table that’s 1 trillion rows. Yes, this is smaller than the 50 petabytes worth of Hadoop capacity eBay added last year, but Teradata is quick to point out that all of its systems support data into and out of Hadoop, so it’s not as if eBay is operating two entirely distinct data environments.

Of course, Teradata has lots of other petabyte-scale customers, with Verizon, AT&T and Bank of America among them. Here are a few more interesting use cases:

  • Harrah’s (now part of the Caesar’s Entertainment casino empire) understands how much money particular gamblers can afford to lose in a day before they won’t come back the next day.
  • Disney is rolling out new bracelet tickets equipped with GPS and NFC that track everything visitors do while inside Disney’s amusement parks. The New York Times detailed the privacy implications of this move in a January article.
  • A manufacturing customer generates 20 terabytes of data per hour while testing products, although that volume is ultimately reduced to about 1 terabyte after the valuable data is filtered out.
  • At some point, Continental Airlines decided it wanted to keep its customers happy and began assessing them by lifetime value (which, it turns out, is often inversely related to frequent-flyer status) and began making alternative arrangements for them as soon as the airline realized flights would be delayed.
  • A luxury car company used Aster Data to analyze the pattern of failures for various components inside its cars. It found out that lighting, seats and infotainment often failed together (they’re on the same circuit) and began inspecting all three when a customer comes in for service on any of them.


None of this means Teradata is destined to continue being a huge name in analytics (Scott Yara, co-founder of rival EMC Greenplum, recently called data warehouses this generation’s mainframe), but it’s still interesting to learn how big companies are analyzing their data, regardless what they’re running on. And with exabytes worth of data no doubt residing in customer systems across the world, Teradata isn’t going anywhere soon

Business Intelligence key terms

intelligence business; business. Intelligence; business intelligence consultancy; business intelligence software; business intelligence solutions; software business intelligence; business intelligence tools; business intelligence solution; business intelligence services, business intelligence companies; business intelligence system; business intelligence tool; Continue reading

Distributed knowledge pool to the frontlines

Distributed knowledge pool to the frontlines.

The most successful business especially from a customer perspective are those that make information available to all personnel (within reason – eg HR information) because any one person can respond to a customer situation – and more importantly resolve it.  Continental Airlines when it restructured after the second bankruptcy under Gordon Bethune (From Worst to First) was so heavily controlled with rules and regulations, neither customers nor employees could get answers.  Hidden information breeds suspicion and distrust. One of Bethunes’s early moves after the restructuring was to hold a big bonfire and burn all the rules books – then told the front line employees to do what is right to resolve customer challenges – the empowerment made for better employee relations and was the first major airline to actually go out of its way to inform customers of what was happening improving customer relations significantly.  The availability of information AND the empowerment of employees made continental one of the best to fly. Continue reading

Reset the Enterprise – the Goal of Business Intelligence

Distributed knowledge pool to the frontlines

The most successful business especially from a customer perspective are those that make information available to all personnel (within reason – eg HR information) because any one person can respond to a customer situation – and more importantly resolve it.  Continental Airlines when it restructured after the second bankruptcy under Gordon Bethune (From Worst to First) was so heavily controlled with rules and regulations, neither customers nor employees could get answers.  Hidden information breeds suspicion and distrust. One of Bethunes’s early moves after the restructuring was to hold a big bonfire and burn all the rules books – then told the front line employees to do what is right to resolve customer challenges – the empowerment made for better employee relations and was the first major airline to actually go out of its way to inform customers of what was happening improving customer relations significantly.  The availability of information AND the empowerment of employees made continental one of the best to fly.

Continue reading

Measuring BI Value

It is very  difficult to place a value on the benefits received. How well are we using the information to make better decisions? There are metrics, such as comparisons of operational efficiencies, before and after, that are relatively standard for some of the most simple of BI applications. Projecting and calculating tangible values for returns, especially on more complex BI investments, is not simple. The process can be frustrating and has often seemed impossible and, perhaps, pointless. But if we can assess these benefits and provide some tangible ratings, we can provide a basis for management decision-making about BI investments for the organization.

Think about it this way:

Good BI is the fusion of the right information, the right time, the right format, and the right human and/or system resources. If we wish to improve BI, we ask these questions:

  • Do the people (or intelligent systems) have the information needed, when they need it, to make decisions?
  • Do those people have the expertise, training and mindset to use      that information in the best way for the good of the organization?
  • Are they doing their job better because of the information being delivered?
  • How much difference does that information make to them?

A Benefits Audit for BI

The most effective way to assess the benefits of BI is to ask the people who are involved. We can use questionnaires and surveys administered at regular intervals. We need them however, to make a realistic assessment of the dollar benefit of doing business differently. A benefits audit must also be structured to give tangible, realistic assessments of the relative benefits of BI projects, BI day-to-day use, and the total organizational BI investment.

Using well-constructed metrics and patterns, we can turn the opinions of experts into a trustworthy assessment of the benefits of BI for the organization. Those experts are the people who use the information and the managers who daily assess the value of the performance of those people who use this information. We may also wish to add competitive analysis and opinions from customers, marketing teams and consultants. Thus, we can turn expert evaluations into a practical, meaningful assessment tool for management.

There are a number of valuable features to be gained from such a benefits audit. We can:

  • Make better decisions regarding commitment of resources to BI. The relative values of BI projects can be identified before spending the money on development. Opportunity costs and risks can be evaluated up front.
  • Improve management planning both strategic and tactical. Competitive, legal and regulatory requirements can more easily be assessed and planned for, and associated risks can be reduced.
  • Encourage better decision-making.
  • Improve the quality of BI and BI projects through a better understanding of the results. We will be able to feed results back into the processes for BI design, development and use.
  • The organization’s culture can evolve and improve through an increased awareness of BI, integration and communication.
  • Provide encouragement and support for our people. Just administering such audits will send a message that the organization cares about them, respects their opinions and is open to new ideas.
  • Encourage creativity and broader use of BI through recognition and associated education, training and awareness.

The questions, formats and process of the BI benefits audit will provide increased awareness, understanding and training for those who develop and implement the audit, as well as for those who participate.

© Trevelyan Group LLC 2012

Humans inherently ask the wrong questions

Humans inherently ask the wrong questions

People in business today are trapped within their own frame of reference. The result of poor hiring practices?  More likely the challenges that companies put in front of employees but without giving them the capability to search for answers to complex problems – with nuance and experience major contributors to the answers set. Nuance is as much of the corporate knowledge base as is hard financial numbers.  The unknown is similarly a corporate asset.  But even the best companies today provide lots of data but limited ability to determine meaning.

Business (also schools) has trained people to stay within a sphere of reference – some (not all) in-house data and limited external information.  The approaches to Business Intelligence applications haven’t helped as they are often limited to relating “like” information – like financial data. The result is that business has contributed to the perspective that humans don’t ask the right question because we have never given them the opportunity, training and empowerment to succeed.

Usually pushed off as too expensive to acquire or too complex to bring together, we have worked hard to justify the limited information available to people making corporate decisions.  We have provided ‘answers’ when supporting additional questioning is required. The limited foresight that exists related to corporate information use.  If Sam Walton had not had the foresight that information on all transactions going through cash registers, deep knowledge of customers and local market bias, and the power of sharing thee resultant knowledge with suppliers, he would have built just another big-box retail outlet.  However Walton, armed with a far reaching vision of the power of knowledge, created a new retail concept.  He enabled his buyers to ask span-crossing questions and supported the answer determination with some of the largest databases in the world.  Each buyer has access to petabytes of transactional information to base decisions.

Corporate decisioning has also contributed to the myth that people ask the wrong questions.  Decisions have often been made on direction and approach.  When employees, armed with knowledge see answers that don’t fit the mold, management is loathe to listen.  People begin to fear the answer. If it is not the expected or wanted answer, retribution is often the result.  People are boxed into a world that even if they have the knowledge base, and the capability of questioning to get ‘answers’ the answers are not ‘right’.

I have always asked companies to value their customers from most to least profitable.  The ranking then begs the obvious question – why don’t we fire the non-profitable customers.  The answer should be to fire them and put the resources (product development, marketing etc) to bear on the profitable customers.  Too often the answer is unpalatable to (especially middle management) management because it forces the realization that business as usual is not as good as it could be if hard decisions are made.  Great examples are the US airline industry and US Banking.  In major banks, profitable customers generally are less than 20% of the base and yet the banks continue to promote expansion of that base – acquiring four non-profitable customers for every profitable one and often a high cost.  The airlines try to be everything to everybody.  With the exception of focused air carriers (Southwest, AirTran), most airlines would probably be better off firing all unprofitable customers – but when management looks at the situation, they realize that their company might be highly profitable if they act, but the business model will be different – instead of a $10 Billion revenue airline, it might be only $1Billion – but highly profitable.  “Business as usual” is always the decision maker.

Facts versus trends. It is easier to state that ‘we sold 100 widgets this month’ vs. looking a trend that says “worldwide widget sales are down and we should evaluate whether we should continue with this product”

Why don’t we ask the right questions???

Technology is transforming innovation at its core, allowing companies to enable new ideas at speeds and costs that were unforeseeable a few short years ago. They can see results from promotions, process change and productivity improvements quickly. Initiatives that used to take months and huge expense to coordinate and launch can be performed at low cost. Innovation becomes more efficient and cheaper.

Innovation in the enablement of knowledge creation and use will bring big changes for corporate culture—making it easier to challenge business as usual, and force managers to give more personnel throughout the company the capability, encouragement and reward for asking the hard questions that give answers that enable profitability and transformation.  The provision of the real corporate knowledge base and the tools to use it will eliminate the perception that people ask inherently ask the wrong question.


©Trevelyan Group LLC 2012

Information democracy. Broader corporate asset with broader access

Information democracy.  Broader corporate asset with broader access

Who can afford not to know what is outdated, ineffective or threatened in their business model or market??

Current economic crisis has made effective BI adoption a critical imperative – who knows what aspects of their markets, business model operating practices are no longer viable. At the same time, information overload is a reality.

We are not even tackling the biggest challenges yet.  We are not capturing and mining voice communication.  We have limited mining of email data. Competition in an information-intensive world demands we fix the problem.  Exponential explosion in information, evolution and acceleration of information types and economic crisis has permanently changed markets/business models, information is increasingly the value-add differentiator in products & services

We are seeing organizations and their people wrestling with the information problem.  IT organizations have been doing a reasonable job of facilitating access to structured data – rows of numerical data and parsed textual data, organized in databases, spreadsheets and reports. However, information required for decisioning often relies on subtleties of meaning contained in unstructured data – text files, memos, emails, pdfs, blogs, voice and video. Add to that are evolving voice-to-text capabilities to ‘mine’ conversations with contact centers.

To many suppliers – and their customers – it makes a lot of sense to bring these two worlds of data together under one, unified information management architecture. It means that a call centre operator, for example, can see in one screen a customer’s sales history, perhaps his profitability to the company – and his tendency to complain about service to the ombudsman. It means that a series of revealing emails between customers and staff can help to explain a sales manager’s poor figures.

Both types of information contained in files are packed full of meaning and informal structures – but virtually none have been available to business decision-makers at any level of organizations.  IT bases its BI information as facts and figures because that is what they have always worked with and delivered as reports and evolving knowledge bases.  What about text?  How does IT deal with it?  It is a fundamentally different and challenging environment without the foundation of traditional corporate lore.  There are great tools in the market for gaining understanding form text and verbal communications but are not generally considered in the mainstream of business applications.

The evolving social media phenomena places even more challenges to corporations  and their IT ‘knowledge” enablers.  Whereas the human can quickly ascertain substance and nuance form the interplay of multiple media types and characteristics, corporations have struggled to deliver even fundamental knowledge creation capabilities – the foundation for BI –  for decision-makers

The core problem in business is that the business continues to run with inadequate tools and approaches – like it has for the last 50 years, but the world is changing rapidly around us.  Information amounts, availability, acceleration of information types are all challenges to individuals trying to make intelligent decisions.

We see that despite the availability of almost any type of information for knowledge creation, BI has been generally the tool select groups in companies.  Pervasive BI, enabling decision-making personnel throughout companies is the fundamental basis of competition in the future.  The failure of BI tool makers, corporation IT departments and executive management to recognize that knowledge’ and it’s use to transform business processes’ relies on ready access to the information asset. We can no longer allow front line personnel, middle management and product development personnel to be basing decisions based on BI almost as a shadow – using Excel spreadsheets and the like to support their decision making.  Especially since shadow BI has significant challenges – info is not shared, not current nor vetted by business process owners (eg the CFO).

A holistic perspective to managing strategic, operational and analytical BI is required. While technical issues are extensive, management must come face to face with cultural, operational and business aspects as well. Organizations need to embrace the cultural challenges preventing knowledge creation and employee success.

We need to move beyond the complexity. We need management vision to support the cultural challenges facing all companies in this competitive environment.

  • Having the vision to enable corporate IT enablers to provide tools, access and capabilities to unlock the asset
  • Sharing of corporate and cross-departmental information
    • Information as a corporate asset is not recognized
    • Eliminating lack of trust
    • recognizing that users view information as a “fact in time” that then drives new questions – usually in multiple layers of detail and / or broadening to       include other information that is or is not related in the traditional (read IT sense)
    • Removing secrecy
    • Eliminating fear of decision making by giving workers at all levels knowledge
  • lack of foresight
    • IT budgets squandered on fads, applications or crisis of the moment

We need to embrace the new competitive reality and resolve it with access, use, and delivery of the knowledge inherent in the entire information asset of the corporation – and enable its decision makers

©Trevelyan Group LLC 2012

Invest in Meaning to create Business Value

Focus on meaning will lead out of the quagmire

Shift Information Management to Understanding Meaning. 

You must think of business decisioning and problem solving as an informational problem, not a process or application challenge.  Data plus more data gives you information.  Information plus intelligence gives you knowledge.  Knowledge plus action gives you transformation.  The fundamental underpinning of business value creation is to approach things differently – to approach each and every business issue as an opportunity to leverage the corporate data asset – to really understand the business – to uncover the meaning in what the accessible knowledge base tells us.   Leveraging the corporate knowledge asset provides the knowledge base for experimentation, modeling varied approaches to change and then providing the roadmap to business transformation. All those things lead to the understanding of what the inter-related element ‘mean”

We have discussed why people may not want the answer to a question – but beyond that, enabling decisioning to uncover the meaning of events and information is good for the business and for all members of the team.  Most IT departments are a major cause of limitations through their lack of enabling business knowledge because of their results-based approach to knowledge dissemination – “ask us for what you want and we will give it to you”.  We need to change that reality and move Information Management to enabling “meaning creation.

But even if IT were to make all information available and accessible – what then?  What about the accelerating volumes and types available within the corporate knowledge pool?

What about the traditional ideas in business makes little of no facility to capture the nuances in dealings with customers, prospects and stakeholders?

The huge efforts by business application software companies to convince the world that the deep understanding of data within their (very expensive) applications will benefit business at large.  What is missing is the ability to “relate” that view of the business to the more subtle views – what are customers saying about our products, what is the competition doing?  What is happening within the sales force? What web traffic are we generating to make one sale? In other words – what the information means so we can transform the business.

The most successful business especially from a customer perspective are those that make information available to all personnel (within reason – e.g. HR information) because any one person can respond to a customer situation – and more importantly resolve it.  Harrahs has invested heavily in knowledge creation about all aspects of the business, employees, customers and competition.  Harrahs arms its employees with the information that they need to empower the use of the interpreted meaning with the nuance of the situation to do what is best for the business.

Hidden information breeds suspicion and distrust and disguises the meaning of what is happening. At Continental Airlines, Bethunes’s early moves after the restructuring was to hold a big bonfire and burn all the rules books – then told the front line employees to do what is right to resolve customer challenges – the empowerment made for better employee relations and was the first major airline to actually go out of its way to inform customers of what was happening improving customer relations significantly. He also challenged IT with delivering the knowledge base for their use. The availability of information AND the empowerment of employees made Continental one of the best to fly.

I believe that we do not need a new form of analysis.  We need to support the human intellect in dealing with business and informational issues (volume).  We need to get off our historic ways of handling information and deliver knowledge making capabilities to all business users.  As we have stated throughout this treatise, we need to surround the human intellect with the best knowledge that computers can derive to enable decisioning.  We can no longer restrict information from people.  We should never write another report not write an application that fixes data into a rigid structure.  We need to forget that storage is costly – it isn’t.  We need to enable the information within a business or organization.  Information about the business is the greatest asset a company has – let’s let it loose.

Decision makers self limitations need to be dealt with.  IT capabilities need to be expanded.  Outdated approaches to information protection instead of wide-spread use in decisioning need to be shelved. We need to approach the problem differently.  We need to enable people to push their limits.  Using our technique called MindPlus(SM) enable people to extend their capabilities. MindPlus is about approaching the problem differently.  Then the application of the answer to the new questioning process needs to be different. Our experience has shown that if people follow the 6 basic steps they will not only thing better but also significantly improve and optimize the business.

The steps

  1. What am I trying to do?  Think about the      problem.   Your reason for acting needs to be looked at from a holistic perspective.  Don’t allow limitations to refine or define the problem
  2. What do I need to know to do it? – look at the problem holistically – what do you need to know. Are there analytical functions required, is the timing appropriate
  3. Is the information available?  Be realistic – but if you don’t have the information to answer the question that it must be acquired.  Else your limitations are rekindled
  4. If it is available, how will I use it to do what I am trying to do? – making a      decision is only one part of the challenge.  How will this knowledge be applied?
  5. When I actually do it, what value will it bring? – at every stage look at the value – if no value – why are you doing it?
  6. If it brings value, then what am I going to do different to enhance the value/process/company?  Always look at the value – if it brings value, how will that knowledge transform the decisioning and related results. What will you do differently in the future based on that knowledge? What does the knowledge mean to the business?  Then act.

©Trevelyan Group LLC 2012