Question:  Working with a data set consisting of 13months of data with one row for each day (so max 380 rows) and needing to show only monthly detail/summary data in the dashboard, which of the following would you consider to be the better approach to yield the best performance?

1.  Pull 380 rows in cache with date in one column -> pass STARTDATE and ENDDATE from the dashboard and pull every month’s data by writing a cache query.   When the user selects a month for display in the dashboard, Excel will calculate the start and end date of the month and pass those to the cache query with a WHERE clause specifying BETWEEN ‘@STARTDATE’ and ‘@ENDDATE’

Or,

2.  Load 380 rows in the dashboard Excel backend -> calculate the month from the date column within Excel then use a hidden filter to filter the month’s row based on the calculated month in the previous step and calculate the sum and average within excel?

Answer:  As a general rule, if the data set is 200 rows or less, I would say to go with Option 2.  If it is more than 200 rows, I would say you will get better performance with the cache query.  Of course, this also depends on the cumulative total # of rows being loaded into the dashboard.  For example, if you are planning on adding 10+ additional 200 row data sets, then I would definitely go with Option #1.

With too much data in the backend, Xcelsius performance suffers.    InfoBurst allows us to cache a large set of data, and then use a Cache Query to extract just the portion we need to view on the dashboard, as the user makes selections.  This is a tremendous help to the developer who wants to boost Xcelsius dashboard performance.

However, a successful IT Director does need to keep abreast of current and new technologies and IT innovations and compare their processes and ideas with their peers in other companies and organizations.  Unfortunately it is often a challenge to find both the time and the forum to do this effectively.

At best, most IT Directors will only have time to attend one or two conferences or technical seminars a year.  Most such forums typically last three days and will run multiple tracks to provide more choice to a wider audience.  Some events try to be more generic in terms of technology but most will be specific to particular applications, software products or technologies.  However, these types of conferences tend to be put on or sponsored by software application or technology companies, resulting in a disproportionate number of marketing oriented sessions to promote products rather than to educate.

It was with these factors and IT managers, directors and executives in mind that InfoSol created a series of highly informative executive seminars at this year’s InfoSol Business Intelligence Seminar (IBIS).

IBIS is well known for its hands-on immersion training boot camps on different BusinessObjects products, but there has been an increasing demand for concentrated and effective seminars on key  new Business Intelligence technologies as well as how to effectively utilize the latest and greatest BusinessObjects solutions.

There are three Executive Seminar tracks at IBIS 2011.  One covers the new and enhanced offerings with the latest release of the SAP BusinessObjects suite, namely XI4.0.  The sessions in this track will both explain and demonstrate live the new capabilities and educate attendees on their key benefits.

A second track (The Next Generation of BI) covers new and emerging BI technologies with particular emphasis on mobile BI which is of great interest to all these days.  There are also sessions in this track on geographic BI solutions, in memory databases for BI and integrating with internet data sources and applications.

The third executive track (Business Intelligence for Executives) is designed for managers and executives to have a better understanding of all the facets involved in planning, managing and executing a successful BI project.

In addition to the information packed seminars, there are case study demonstrations, customer and expert panel discussions, and plenty of opportunity to have real quality one-on-one information exchanges with peers.

The entire event is limited to no more than 250 attendees, ensuring that it remains a productive and quality gathering for everyone.

I hope you will join us at this year’s InfoSol Business Intelligence Seminar.

The table next to us was a major airline company and I used them as an example, citing that in the recent economic downturn, airlines had to make some tough choices in terms of reducing costs.  They needed to know which aircraft cost the most to maintain and which routes were the least profitable.   That information allowed them to make informed decisions about reducing aircraft and routes with minimal impact to their business and their customers.

I could see the lights go on in their young faces which encouraged me to tell my current favorite Business Intelligence story.

At last year’s InfoSol Business Intelligence Seminar (IBIS 2010), one of the winners of the prestigious Xcelsius Dashboard Awards was a manufacturing company from the Netherlands.  They explained that they were deploying a new manufacturing execution system and wanted to provide some engaging and compelling business intelligence from this system to both encourage usage and provide useful information that was previously unavailable across the company.  They decided to create a business intelligence user interface using Xcelsius.  It would provide in excess of 120 different metrics, deployed graphically and dynamically, about the different production line processes broken down to individual shift level.  The dashboard was dynamically refreshed through the unique intelligent caching mechanism from InfoSol and the impact of this application exceeded the expectations of this company.  For the first time, production, and issues encountered on the production line, was viewed by the entire company and the results were quite revealing.  The dashboard showed that one particular shift was measurably behind all the other shifts.  Since this was visible throughout the company, the workers on this particular shift felt they needed to do something about this and took it upon themselves to correct the issues and improve productivity so that they were on pace to do better than the other shifts. 

What a totally amazing side effect, but one that truly exemplifies not just the power, but the inspiration that can be derived from Business Intelligence.  It was with this example in mind that the theme of this year’s IBIS event “Inspiring Business Intelligence” was born.  There are so many fascinating and inspiring business intelligence stories like this and IBIS 2011 will be based on sharing some of these stories as well as offering an incredible line up of immersion boot camps, executive seminars, training workshops and specialized certification programs. 

IBIS is unique in so many ways as it goes beyond the regular SAP BusinessObjects training to teach best practices that go beyond the conventional ways of designing, developing and applying business intelligence.  The individuals leading these boot camps, seminars and workshops are not just teachers; they are highly experienced consultants, mentors and gurus in their field.

The knowledge transfer that occurs at IBIS is intense and the opportunity to exchange ideas and experiences unparalleled.

IBIS 2011 features hands-on immersion boot camps in Xcelsius, Web Intelligence, Crystal Reports and Xcelsius Data Caching.  The executive seminars cover best practices for success in business Intelligence projects, what’s new on BusinessObjects XI 4.0, and emerging business intelligence tools and technologies.  The main seminar is three full days and the registration is fully inclusive of four nights accommodations at the jaw dropping Ritz Carlton Laguna Niguel on the southern California coast (inspirational in itself!).

New to IBIS this year are some very special certified boot camps which will be held on the Thursday and Friday directly after the main event.  These include the renowned FleXcelsius course from Centigon Solutions, teaching you how to create your own Flex components for Xcelsius.  The Roambi Black Belt certification and the InfoBurst Certified Expert (ICE) boot camps will also be offered. 

Most importantly, there will be the coveted Best Xcelsius Dashboard Awards, which are quickly becoming legendary.  The awards are open to everyone and the deadline for submitting your inspirational dashboard entry is April 15^th.  You can enter your dashboard in one of the following four categories:

-          Best Business Dashboard

-          Most Innovative Dashboard

-          Most Valuable Dashboard

-          Most Xcellent Dashboard

To submit your dashboard entry, click here.

I know that some of the students who stopped by our table left inspired by what they heard about business intelligence just based on the slew of follow-up enquiries.  In fact we have invited two interns to join us starting today. 

I also know that anyone attending IBIS 2011 will undoubtedly leave more inspired about business intelligence than before they came – don’t miss out!

Register today – Early registration discount expires March 31^st.

This question comes up often when I’m teaching an Xcelsius training class.  Often enough that I’ve decided to discuss the solution here in a tech tip on our blog.

This scenario can happen with different data sources (XML, Live Office, Qaaws) when a number string is interpreted by Xcelsius to be numbers rather than string data.  As many people have tried, it is not enough to format the cells as text in the Excel spreadsheet of Xcelsius.  We need to use a trick to add those leading zeros back in. 

For example, say my Order numbers are all 3 digit numbers (for simplicity sake).  My order numbers are loaded into Column D, but then converted to their proper 3 digit format in Column F with the use of the Length formula in Excel, LEN.  By simply combining an if statement and the Length function to determine the length of a value, we can add the appropriate number of zero’s in front of a value, and voila, properly formatted Order numbers! 

See example below:

using the LEN function in Excel

We believe that Business Intelligence is changing our lives in so many amazing ways.  It inspires companies, institutions and people to make timely decisions with confidence and is one of the most exciting happenings on the planet right now.  

InfoSol is inspired to see beyond your data by using Business Intelligence to its ultimate effectiveness.

The SAP BusinessObjects suite of solutions delivers the most leading edge and powerful business intelligence solutions in the marketplace and INFOSOL’S BUSINESS INTELLIGENCE SEMINAR (IBIS),  is the most effective forum for learning how to use these solutions in new and inspirational ways.

IBIS 2011 offers an incredible line-up.  Hands-on  PreSeminar workshops, immersion training boot camps, executive seminars … visit the IBIS 2011 website.  You can view conference details and register.

They had seen Xcelsius dashboards and were impressed by the dynamic visualization and felt this could make both an attractive and appealing way to present the information to their clients. However, they were faced with several challenges regarding automatically populating the dashboards initially and then automatically distributing them as standalone offline entities that could be used by both their Account Managers and clients without being connected back to the originating system.

They did not want to run separate jobs for each client to populate each dashboard. They wanted to run one job that would provide all the statistics for all customers and then split or burst that data by customer to feed each customer’s unique data into the same dashboard template. The amount of data that would be initially retrieved was large (many tens of thousands of rows) and even some of the individual customer dashboards had a fair amount of data (several thousand rows). Once the data had been burst for each customer it would need to be populated into the dashboard template, published as a shockwave file (SWF) and attached to an email that would need to be sent to the right group of people for each dashboard.

While this may seem like a tall order, the new InfoBurst solution was able to achieve all this relatively simply and quickly. InfoBurst is a scheduling, bursting and publishing solution for both Business Intelligence reports and dashboards . It has a built-in scheduler that can be either time or event based triggered. The event can be the result of a SQL statement which makes it very versatile for starting a job only when data has changed or other dependent events have occurred. InfoBurst can accept data from SQL, a Stored Procedure or a Business Intelligence query or report (Web Intelligence, Crystal, Desktop Intelligence, Microsoft Reporting Services etc.).

When handling Xcelsius dashboards, InfoBurst uses unique mechanisms for both ‘OFFLINE’ and ‘CONNECTED’ distribution requirements.

For a ‘CONNECTED’ scenario, the Cache can be accessed from the Xcelsius dashboard in a variety of ways so that only the data required for a particular component of the dashboard is retrieved when required. We provide a number of components that we collectively call ‘INTELLIGENT CACHE’. The CACHE QUERY mechanism in InfoBurst allows you to use the full power and capability of SQL to selectively retrieve, aggregate and sort the data required for your Xcelsius dashboard from the Cache. The CROSSTAB mechanism allows you to generate crosstab tables in advance or on-demand. The FLIP mechanism allows you to PIVOT an existing Crosstab, switching the X and Y axis data. These features save an enormous amount of time and effort over other methods of pulling data into an Xcelsius dashboard and it’s fast! For ultimate flexibility, it is possible to have multiple caches and to populate a cache with data from multiple data sources.

For an ‘OFFLINE’ scenario, InfoBurst is able to create and manage separate ranges within the Cache, and Burst these ranges to a common Dashboard, embedding the data inside of the SWF file and delivering the same Xcelsius dashboard with different data for each individual client.

InfoBurst also has intelligent distribution capabilities where it can read from an LDAP compliant address book, a database table or an Excel spreadsheet to select the relevant email addresses or target destinations for the dashboard or report. Since this customer required the offline dashboards to be sent via email , the information was read from a database table containing all the individual’s information and is able to determine the customer name and criteria and select just the email address for the relevant recipients.

This company is now using InfoBurst to distribute offline Xcelsius dashboards automatically , intelligently and swiftly to its customers and is saving a lot of time and money in the process.

For more information or a free demonstration and trial on InfoBurst visit www.infosol.com/infoburst

In this blog I will discuss 2 methods, both of which are valid and are useful ways to organize dashboard data.  Deciding as an organization which approach you adopt can help with the process of taking up someone else’s project. 

 Some Xcelsius developers like to use a method of having all the data that is being used in the dashboard on one (or more) tab/s of the Excel spreadsheet, and then as you need to use data for display, using a series of vlookups or match & index functions for displaying data on charts, which they layout on other tabs of the spreadsheet.

My preferred method is to spread the layout in such a way that the names of the Excel tabs generally correspond to the functional area on the Xcelsius dashboard, perhaps even the same name of the Xcelsius  tab (Either the Tab Set Container or the Label based menu, is what I’m thinking of), so that all the data being expressed on that “page” of the dashboard is on one tab in the Excel spreadsheet behind the scenes. 

I like to have the data insert & any manipulations I have to do for expressing that data nearby the loaded data, so that it is all in one place & I can find it easily.  The only difficulty with this is when you have many components on one “page” or you have to create several aggregate levels to display.  Spreading it out & spacing it can become an issue, and this is where again, labeling the spreadsheet is so important.  (But aren’t we glad that if we move data around on the spreadsheet that the links follow, for those of you who have been using Xcelsius since the 4.5 days!). 

This approach, of course, assumes you have the aggregated data to work with!  Which leads back to my previous blog regarding data connectivity options.   If I have to use QaaWS, I end up having to do a lot of manipulation in the universe to get the data to look or be ordered the way I want to use it in Xcelsius.  I usually have to end up making many of what I call, “dimensionalized measures” when I want to end up with a crosstab result.  So I have a measure for Jan Sales, Feb Sales, etc.  Very tedious.  Sorting is another  issue that is often easier addressed in the universe, but also can be labor intensive.  

Live Office gives me the option to create crosstabs and sort the data in any way I wish and I can also have multiple tabs to aggregate the data at different levels (YTD, MTD,  past 30 days, etc).  The problem that I so often bump into with this approach is the amount of data when we get down to the detail level.  We have all experienced poor performance any time we go over the 512 row default and horrible performance when we approach 1000 rows. 

All of these are reasons why I enjoy using the InfoBurst XDS or XDM option, as it addresses all these issues, plus gives me many more options.  The Intelligent Cache syntax of this tool is by far the most exciting feature, in my opinion.  Using their syntax, I can pass parameters to the XML url call and only return the rows of data I wish to display.  For example, within my xml url call, I can add syntax that returns …. YEAR=2009;DIVISION=HPD;DEPT=23.  This ability to extract just the rows of data I want to display helps keep my dashboards very responsive, since I am not loading it down with too much data.  It also gives me the ability to go to a much more detailed level than I would have been able to with QaaWS or LiveOffice.

XDS or XDM also has syntax to return unique values of a particular column from an xml data set which I can use for  my selector.  My selector will always have accurate values, as it is loaded from the current xml cache, rather than from a hard-coded list in my Excel spreadsheet which may not be up-to-date.

And, I have the flexibility to create these large XML data sets from a Web Intelligence document or by writing a query, directly to the database, or even from an Excel spreadsheet, and any combination of all three!

 Whether you find yourself having to update one of your own dashboards after a period of time of not working on it, or having to hand off the maintenance of the dashboard to a colleague, the initial effort of good documentation will save valuable hours and ensure continuity and success.  In this post some of the graphics will speak for themselves – you know the cliche of ‘a picture paints a thousand words’. 

The graphics show examples of: 

• good labeling,
• use of color to enhance visual cues
• and the use of comments within Excel. 

Use of Color and Labeling

Below are samples of Excel spreadsheets that incorporates the use of visual cues like color coding with simple, clear and easy to follow labels showing where components derive their inputs.

Using Excel Comments

Another helpful option is to use the built in Excel feature of Inserting a Comment.  This leaves a small red triangle in the top right corner of a cell to indicate that a comment has been added to that cell. When the mouse hovers over the cell, the comment pops up. 

Helpful tips can be included within the comment.

Comments can be added to any cell by right-clicking on a cell and selecting “Insert Comment”.  Once a comment has been added to a cell, the small red triangle will appear.  A comment can be edited or deleted by right-clicking on the cell.

Using a Legend

There are many ways to document your Xcelsius Dashboard Excel Spreadsheet. 

I would love to hear about your “Best Practices”.

There is an Xcelsius Bootcamp being offered in Track 4 of this year’s IBIS 2010 event June 6-9 - Seeing Beyond Business Intelligence.