I am glad that I have chosen Business Intelligence as one of my electives. It will definitely help me along in my future career. It has also opened up to me another perspective of both business and IT side.

In conclusion, for the whole 16 weeks, I have enjoyed taking this module. The group project has not only helped me to build up my personal skills but as well as knowledge relating to Business Intelligence.

Rock On BI!(;

This was the last lecture for Business Intelligence. It sums up the whole entire module and what is ahead for data mining!

Here is an article that i have found explaining the future of data warehouse.

Data Warehousing: What the Future Holds

Data Warehousing Horizons

Information Management Magazine, November 2001

John Onder

Within a year, the overall information technology industry has gone through a major upheaval. How has this affected our marketplace, and what does the future bring for those that specialize in the data warehouse (DW) and business intelligence (BI) space? This article is a series of observations and thoughts based on the experiences with Headstrong’s customer base in our Digital Intelligence Practice.

Many changes are looming on the horizon for our industry. First is a focus on the return to basics – drive revenue, cut costs, and acquire and retain customers. C-level and board applications are moving to displace departmental applications as the starting point for data warehouse development. The reasons are many and varied and include the maturing of departmental applications, the overall return to sound business basics after the dot-com boom/bust and the ever-important focus on the customer and on shareholder value.

Other trends indicate that the future will bring a level of complexity and business importance that will raise the bar for all of us. The real-time implementation of a business action, decision or change of direction that is based on the results of strategic data analysis is now the reality. The data issues surrounding this trend aren’t getting any easier or smaller. Combine the need for real-time data warehousing and increased data size and complexity, and we set the stage for a new type of warehouse – the “virtual” enterprise data warehouse. This virtual DW or private hub for both operational and informational needs will begin to drive new demands on the ability of organizations to assimilate vast data assets stored in merged/acquired companies or divisional enterprise resource planning (ERP) and legacy environments. The time and/or dollars needed to integrate all the operational systems will make the traditional method of data integration impractical. This intersection of Web channels and data warehousing has the potential to become the standard architecture for large, complex organizations.

Next, it is without a doubt that the entire customer relationship management (CRM) explosion is driving a large portion of the current data warehouse projects in the industry. But is this really new? I’ve always stated that those of us who are veterans in the DW world were building and running CRM environments before there was an acronym for CRM. Furthermore, we are definitely seeing a reverse evolution in the CRM space, which accentuates the importance of analyzing and measuring the effectiveness and efficiency of operational CRM capabilities. Continuing on the theme of measurement, remember the executive information system (EIS)? The EIS was the easy-to-use, show-me-the-numbers application directed at the senior business management of a company. The intent of an EIS was to make the key performance indicators essential to running a business available at the touch of a button. These applications were the rage in the mid-’80s. The EIS lives again as the digital dashboard. All the leading online analytical processing (OLAP) software vendors have developed or are in the process of developing key performance indicator modules; and many dashboard-specific vendors are starting up and/or growing rapidly.

Lastly, the complete scope of the organization, structure and processes needed for a successful customer deployment of a large, complex data warehouse continues to be a major problem for most organizations. Making it happen successfully takes a unique focus and team of people. Mainly, the ability of an organization to change or dispose of the conventional wisdom regarding the correct mix and role of staff is important. Regarding this topic, I have the following observations. Your customer deployment team structure must mimic your desired business goals; and the team should reside within the same space – business operations, business analysts and IT professionals need to learn to cohabitate. Training must go beyond basic tool training to include pure business solution analysis tied to the desired outcome. Tight tracking and measurement of the expected business return has to be easily understood and continually revisited against expectations. Include in your engineering of the customer deployment shop not just what the team members will do, but how they will do it. What process can be made repeatable, scalable and flexible enough to change with the business?

The next article is about the future of data mining by IBM.

IBM’s recent announcement that it is to acquire SPSS not only has implications in terms of the completeness of IBM’s analytics offering but it also has ramifications for the future of data mining as a whole. It is the latter upon which I will focus.

There are two aspects to this: the first is with regard to the choices available to users wishing to use data mining tools and the second is with respect to in-database data mining. I will discuss this second issue first.

Traditionally, in order to perform data mining operations you extracted the data from the data warehouse or mart, into the mining tool where you processed it. The problem with this is that there is a significant performance hit from the extraction of all that data. Of course, you could reduce that impact by sampling the data, but then you lose accuracy. As a result, the trend recently has been to perform mining processes “in-database”. And one of the first announcements we can expect after the acquisition is complete is that IBM will be implementing SPSS functionality in DB2.

Now, both Oracle and Microsoft have the ability to process mining algorithms in-database (as indeed does IBM) but neither of these has a tool like SPSS, so this will give IBM a significant advantage over these two. However, almost the entirety of the rest of the warehousing market has no such capabilities, the exceptions being where SAS has been working with Teradata, Netezza and others to implement SAS capabilities in-database. Also notable (and to be commended for their foresight) is Netezza, which acquired NuTech Solutions last year, though this is more of a tool for building predictive applications than a data mining tool per se.

All of this means that most of the warehousing community is going to be utterly dependent on SAS. Those warehousing vendors who do not persuade SAS to invest in the effort involved in implementing in-database capabilities are going to be at a significant disadvantage compared to those that do. This is good news for Tibco.

Historically, there has been one major data mining vendor (SAS), one middling player (SPSS) and several tiddlers. The tiddlers used to include Angoss, Kxen and Insightful, amongst others, and there has never been any indication that any of these smaller players could break the industry stranglehold exerted by SAS and SPSS. However, Tibco acquired Insightful last year following a previous acquisition of Spotfire. And Tibco is not a tiddler. The company also has a leading event processing engine. Put these facts together with Tibco’s focus on the predictive enterprise and you have a company that has the potential to fill the gap left by IBM’s acquisition of SPSS. Whether it will do so or even attempt to do so is another matter but the potential is there. Personally, I hope they do: however much I may like SAS I would prefer it if they had some real competition.

Future of Data Visualisation (an overview):-

Data Visualization Past, Present, and Future

Data visualization, the use of images to represent information, is only now becoming properly appreciated for the benefits it can bring to business. It provides a powerful means both to make sense of data and to then communicate what one has discovered to others. Despite their potential, the benefits of data visualization are undermined today by a general lack of understanding. Many of the current trends in data visualization are actually producing the opposite of the intended effect, confusion rather than understanding. Nothing going on in the field of business intelligence today can bring one closer to fulfilling its promise of intelligence in the workplace than data visualization. But this will happen only if one understands it and use it properly.

Credits

http://www.information-management.com/issues/20011101/4212-1.html

http://www.it-director.com/technology/applications/content.php?cid=11451

http://www.zdnetasia.com/itlibrary/data-management/0,3800009948,44199545p,00.htm

Below is an article related to implementing business intelligence system:-

Implementing Business Intelligence Systems: An Organizational Learning Approach

Information Management Magazine, November 2003

Jerry Kurtyka

Can organization theory inform the designers and implementers of business intelligence (BI) systems to help increase the likelihood of their success? The link between BI and the organization that uses it is that an organization collectively is a cognitive system: it senses the environment, makes a representation of it, acts on the basis of the representation and learns from the results of its actions, storing its experience as institutional memory. A BI system supports this process at various levels of organizational learning.

This article, a follow-up to a previous article published in the June 2003 issue of DM Review, “The Limits of Business Intelligence: An Organizational Learning Approach,” looks at how designers can reference organizational theory to more effectively implement BI systems. There are two key areas where designers can exercise leverage over BI implementation to help assure their success:

1. Correctly assessing the level of organizational learning and intervention that the BI project requires to be successful.
2. Integrating change at the correct level of the organization’s institutional memory or knowledge structure.

Levels of Organizational Learning: Single and Double Loops

There are many ways that learning occurs in an organization. Chris Argyris and Donald Schon, scholars at Harvard and MIT respectively, have articulated a useful learning framework.¹ They describe organizational learning as comprising single-loop and double-loop modes of change. Single-loop learning occurs when an organization encounters a situation that it can resolve using its current systems, ideas and values. Double-loop learning requires a revision of these artifacts and assumptions. Argyris and Schon also define a third construct, the organization’s action theory that describes the collective task knowledge of the organization that it and individuals within the organization access to respond to events. The action theory consists of both explicit knowledge that everyone sees and knows about and tacit knowledge that describes why people actually do what they do in the organization (i.e., it is the organization’s collective “moxie”).

In the single-loop learning mode, an organization inquires into a situation by referencing its current action theory or repository of institutional knowledge and memory. This can include the collective systems and information resources of the organization, its espoused (public) values, and its basic underlying values and assumptions – the components of institutional memory. In single-loop learning, members of the organization know these things and they are not questioned or changed as the situation is resolved. Figure 1 illustrates how single-loop learning works.

Figure 1: Single-Loop Learning

Here, a single feedback loop, mediated by inquiry into the current organizational knowledge base, connects an outcome of action mismatched by expectations (i.e., negative feedback) that is, therefore, surprising to the organization.² This is similar to how a room thermostat works. The temperature deviates from some preset level, and it triggers the furnace to kick in until the room temperature rises to the preset level again. In a business example, perhaps sales drop for a quarter and management responds with the usual tactics: cut prices, increase advertising, etc. When sales respond, the intervention is deemed successful. No major changes have been incorporated into the business as a result. The situation has responded to tactics dictated by the current action theory. Single-loop learning thus accommodates and responds to negative feedback from the business environment.

My experience has been that many BI efforts are intended to address a single-loop learning/change situation. Users and technicians design a data warehouse (DW) that feeds data into functional data marts and/or “cubes” of data for query and analysis by BI users. The functional data marts represent business domains such as marketing, finance, production and planning. These are used by functional analysts and managers to inquire into current operating issues and to facilitate incremental improvements and plans. This is seen in Figure 2, a diagram similar to those used by BI vendors to illustrate how BI informs the overall organizational learning process. Figure 2 is a single-loop model of learning.

Figure 2: BI Single-Loop Learning Model

In contrast, a double-loop learning event results in a change to the organization’s action theory and its knowledge base. This is shown in Figure 3.

Figure 3: Double-Loop Learning

An additional learning loop connects the observed events with the strategies needed to formulate action (i.e., positive feedback). The action theory and even the knowledge base can undergo revision, as shown in Figure 3. In BI terms, this means that the business intends to integrate the BI technology in such a way as to change its operating model, not just to seek incremental improvement.

In the BI world, examples of double-loop implementation are more difficult to find and are risky when attempted. I can think of one case that I observed where a company planned to use its BI system (a CRM analytic DW) to enable it to migrate to performance-based pricing with its customers. In this case, the BI suite was a technical success but the company did not sell its customers on the pricing idea, nor did its management follow through with the organizational changes needed to accomplish the goal. The end result was that the BI suite was an expensive tool that did not give the business the strategic competitive edge it sought from the technology. It is a lot easier to talk about BI technology changing the business operating model than it is to attempt it when the time comes!

Impact of Learning Modes for the BI Designer

Single-loop methods are workable when inquiry and corrective action take place within the framework of values and norms dictated by the current action theory. However, when events require the organization to revise its action theory so that a new theory is available to members, double-loop learning is needed. The process of inquiry between the organization and its environment, including double-loop type events that result in changes to the action theory, puts a lot of stress on managers and technical staff who must support this mode with BI. It is much more risky. Double-loop organizational learning is becoming a necessity for organizations when environmental change is rapid and disruptive.

For the BI designer, this means that he/she has to assess the level of change and learning that the BI suite is intended to support and then to factor that into the BI project risk. If the BI suite will support a change in the organization’s action theory (double-loop change), then there are a lot of factors that will be outside of the BI designer’s scope of control but will be necessary to assure the success of the project. These might include changes in organization structure, metrics, compensation formulas and so forth. For example, an enterprise CRM system that includes an analytic DW component integrated with channel contact applications is a project that is likely to be double-loop in scope. According to the industry statistics I have seen, such projects do not have a good track record of success. The rubric about people, processes and technology needing to be in alignment has never been more true when it comes to double-loop change situations!

A Framework for Implementing BI Change

An organizational learning framework for successfully implementing BI change will have to consider both: the type of change that the BI system is intended to support (single- or double-loop) and, how the components of organizational knowledge (institutional memory) are affected. Before presenting this framework, I want to review how organizations store and process knowledge.

Earlier, I described the construct of the action theory that consists of both explicit knowledge that everyone sees and knows about and tacit knowledge that describes why people actually do what they do in an organization. In my prior article, I discussed the structure of organizational knowledge as seen in Figure 4.

Figure 4: Action Theory Encoded in Organizational Knowledge

Figure 4 illustrates that the action theory is “encoded” in a schema of three levels of knowledge that comprise the organization’s institutional memory.³ Artifacts are the visible and tangible products of culture in an organization and include items such as the physical layout, reward systems, important business processes, information systems (e.g., the BI suite), symbols of status, logos and dress codes. Espoused values are what people in the organization say that they believe; they are the organization’s official view of itself and usually come into being as a result of management action (e.g., a mission statement or promotion of people who exemplify some desired behavior). Finally, basic assumptions are the real operative principles that underlay a culture. Basic assumptions tell the members of the organization what to pay attention to, how to react emotionally, and what to do in various situations; they are embedded in the organization’s unconscious knowledge and are less visible. All of these are the repository for explicit and implicit rules that comprise the organization’s action theory and, depending on the level of change that a BI project entails, need to be considered.

Putting It All Together

Figure 5 is a framework for BI designers to assess the level of change and risk that a project requires and to plan accordingly for implementation. The framework shows the scope of the learning or change effort for a BI implementation (single-loop and double-loop) matched to the levels of organizational knowledge that the effort entails.

Figure 5: Framework for BI Change

For example, a data mart that is used for financial analysis by a few users would likely involve only single-loop change; it is not changing the organization’s business model or impacting many other users. Thus, its impact is mainly at the artifact level of technology and business processes. My guess is that most departmental BI projects fall into this category and are represented in the lower left-hand corner of the diagram in Figure 5.

In contrast, BI projects that initiate double-loop learning impact the more nebulous levels of organizational knowledge and are represented by the right-hand side of Figure 5. An enterprise CRM system that connects an analytic DW to e-business contact channels such as a user Web site, a contact center and a sales force automation application is likely to involve a lot more organizational change. This type of system could provide both the information and functionality that changes the organization’s view of itself and its environment. It will impact the levels of artifacts, espoused values (there might be quality standards involved, especially with customer service functions) and possibly even basic assumptions (e.g., moving to a relationship pricing model with customers). Thus, the project that implements this level of change will need to consider the impact on the higher and more nebulous levels of institutional knowledge. This is represented in the upper right-hand corner of Figure 5. Working at this level of organizational change is sometimes referred to as organization development or OD. The main issue for BI implementers working at this level is to help senior managers to understand what their basic assumptions actually are and how the BI project is likely to influence them (and vice versa).

In short, the BI implementer is faced with the task of assessing the degree of change that a project will require in the target organization and factoring this into his/her plans. The more that a BI project impacts knowledge levels above that of artifacts, the more change effort is required of the organization and on the part of BI project leadership to assure success. Planning and budgeting for the organizational impact of BI, especially when it involves double-loop learning, can greatly increase the likelihood of success of the BI project as a business initiative.

In this article, I have attempted to explain how designers can plan for successful BI implementation using the constructs of organization theory. There are two key areas where designers can exercise leverage over BI implementation to help assure success:

1. Correctly assessing the level of organizational learning and intervention that the BI project requires to be successful.
2. Integrating change into the right components of the organization’s institutional memory or knowledge structure.

This is accomplished by assessing whether the change that is needed for BI can be accommodated within the organization’s current schema of knowledge (single-loop learning) or requires a revision of the schema (double-loop learning). Subsequently, BI designers have to focus change efforts at the right levels of knowledge within the organizations’ collective or institutional memory.

References:
1. Argyris, Chris & David A. Schon, Organizational Learning II. Reading: Addison-Wesley, 1996
2. ibid, p. 21
3. Schein, E. Organizational Culture and Leadership. San Francisco: Jossey-Bass, 1997

Credits

http://www.information-management.com/issues/20031101/7610-1.html

This lecture is one of the easier understanding topics. Both text mining and web mining are used in different scenarios. The above mindmap gave a brief overview of what I have learnt from this lecture.

Besides the lecture notes, I have also research on both text and web mining. From there I have got to know more about it. I will explain text and web mining further below.

Text Mining

The purpose of Text Mining is to process unstructured (textual) information, extract meaningful numeric indices from the text, and, thus, make the information contained in the text accessible to the various data mining (statistical and machine learning) algorithms.

Information can be extracted to derive summaries for the words contained in the documents or to compute summaries for the documents based on the words contained in them. Hence, you can analyze words, clusters of words used in documents, etc., or you could analyze documents and determine similarities between them or how they are related to other variables of interest in the data mining project.

In the most general terms, text mining will “turn text into numbers” (meaningful indices), which can then be incorporated in other analyses such as predictive data mining projects, the application of  unsupervised learning methods (clustering), etc.

There are 3 types of Web Mining:-

Web Structure Mining

Web structure mining, one of three categories of web mining for data, is a tool used to identify the relationship between Web pages linked by information or direct link connection. This structure data is discoverable by the provision of web structure schema through database techniques for Web pages. This connection allows a search engine to pull data relating to a search query directly to the linking Web page from the Web site the content rests upon. This completion takes place through use of spiders scanning the Web sites, retrieving the home page, then, linking the information through reference links to bring forth the specific page containing the desired information.

Structure mining uses minimize two main problems of the World Wide Web due to its vast amount of information. The first of these problems is irrelevant search results. Relevance of search information become misconstrued due to the problem that search engines often only allow for low precision criteria. The second of these problems is the inability to index the vast amount if information provided on the Web. This causes a low amount of recall with content mining. This minimization comes in part with the function of discovering the model underlying the Web hyperlink structure provided by Web structure mining.

The main purpose for structure mining is to extract previously unknown relationships between Web pages. This structure data mining provides use for a business to link the information of its own Web site to enable navigation and cluster information into site maps. This allows its users the ability to access the desired information through keyword association and content mining. Hyperlink hierarchy is also determined to path the related information within the sites to the relationship of competitor links and connection through search engines and third party co-links.
This enables clustering of connected Web pages to establish the relationship of these pages.
On the WWW, the use of structure mining enables the determination of similar structure of Web pages by clustering through the identification of underlying structure. This information can be used to project the similarities of web content. The known similarities then provide ability to maintain or improve the information of a site to enable access of web spiders in a higher ratio. The larger the amount of Web crawlers, the more beneficial to the site because of related content to searches.

In the business world, structure mining can be quite useful in determining the connection between two or more business Web sites. The determined connection brings forth a useful tool for mapping competing companies through third party links such as resellers and customers. This cluster map allows for the content of the business pages placing upon the search engine results through connection of keywords and co-links throughout the relationship of the Web pages. This determined information will provide the proper path through structure mining to improve navigation of these pages through their relationships and link hierarchy of the Web sites.

With improved navigation of Web pages on business Web sites, connecting the requested information to a search engine becomes more effective. This stronger connection allows generating traffic to a business site to provide results that are more productive. The more links provided within the relationship of the web pages enable the navigation to yield the link hierarchy allowing navigation ease. This improved navigation attracts the spiders to the correct locations providing the requested information, proving more beneficial in clicks to a particular site.

Therefore, Web mining and the use of structure mining can provide strategic results for marketing of a Web site for production of sale. The more traffic directed to the Web pages of a particular site increases the level of return visitation to the site and recall by search engines relating to the information or product provided by the company. This also enables marketing strategies to provide results that are more productive through navigation of the pages linking to the homepage of the site itself.
To truly utilize your website as a business tool web structure mining is a must.

Web Content Mining

This type of mining uses the ideas and principles of data mining and knowledge discovery to screen more specific data.  The use of the Web as a provider of information is unfortunately more complex than working with  static databases.  Because of its very dynamic nature and its vast number of documents,  there is a need for new solutions that are not depending on accessing the complete data on the outset. Another important aspect is the presentation of query results. Due to its enormous size, a web query can retrieve thousands of resulting webpages. Thus meaningful methods for presenting these large results are  necessary to help a user to select the most interesting content.

The below figure shows 3 types of Web Content Mining Techniques:-

Web Usage Mining

Web usage mining is the type of Web mining activity that involves the automatic discovery of user access patterns from one or more Web servers. As more organizations rely on the Internet and the World Wide Web to conduct business, the traditional strategies and techniques for market analysis need to be revisited in this context. Organizations often generate and collect large volumes of data in their daily operations. Most of this information is usually generated automatically by Web servers and collected in server access logs. Other sources of user information include referrer logswhich contains information about the referring pages for each page reference, and user registration or survey data gathered via tools such as CGI scripts.

Analyzing such data can help these organizations to determine the life time value of customers, cross marketing strategies across products, and effectiveness of promotional campaigns, among other things. Analysis of server access logs and user registration data can also provide valuable information on how to better structure a Web site in order to create a more effective presence for the organization. In organizations using intranet technologies, such analysis can shed light on more effective management of workgroup communication and organizational infrastructure. Finally, for organizations that sell advertising on the World Wide Web, analyzing user access patterns helps in targeting ads to specific groups of users.

For more information on Text and Web Mining you can visit these websites:-

http://www.statsoft.com/textbook/text-mining/

http://www.scribd.com/doc/19492815/web-structure-mining

http://docs.google.com/viewer?a=v&q=cache:Y1KrfNwiyAQJ:www.softcomputing.net/jikm.pdf+web+usage+mining&hl=en&gl=sg&pid=bl&srcid=ADGEESh3FudVNVMtw8GB7usywCe0glFmtiWSJrb1VbZfUwzLQS89JjsLmBzXFK3ZTW1qJB1qHFIgWuphCUOeula6KDOscf0vI7LkEck7pb0DXD-sWnvNosP3rZx5o6upMMQIgkd6Dolr&sig=AHIEtbTy8Z7OCF2EeVBDGYBKXt1tEzec-Q

Credits:-

http://www.statsoft.com/textbook/text-mining/

http://www.web-datamining.net/structure/

http://maya.cs.depaul.edu/~mobasher/papers/webminer/survey/node6.html

This lecture is different from any other lectures as most is based on numbers and calculation. I have learnt that there are different types of regression models. Through doing the excel exercise during the lab lesson, I have learnt how to calculate R(square), F-statistics, F-dist, and the comparison of t-value and t-test prob to see the accuracy of the value.

Here are some of the types of regression:-

Linear Regression

One of the simplest and most popular modeling methods is linear regression. Linear regression fits a straight line (known linear function) to a set of data values. The form of the function fitted by linear regression is:-

y = a0 + a1*x1 + a2*x2 + …

Where a0, a1, etc. are parameters whose values are determined so the function best fits the data. Linear regression is a popular modeling technique, and there are many programs available to perform linear regression. However, linear regression is appropriate only if the data can be modeled by a straight line function, which is often not the case. Also, linear regression cannot easily handle categorical variables nor is it easy to look for interactions between variables.

Nonlinear Regression

Nonlinear regression extends linear regression to fit general (nonlinear) functions of the form:-

y = f(x1,x2,…,a1,a2,…)

Here are few examples of functions that can be modeled using nonlinear regression:

y = a0 + a1*exp(x1)
y = a0 + a1*sin(x1)

As with linear regression, nonlinear regression is not well suited for categorical variables or variables with interactions. The other challenge involved in using nonlinear regression analysis is that the form (model) of the function must be specified. For engineering and scientific problems, the function model may be dictated by theory, but for marketing, behavioral and medical problems, it can be very difficult to develop an appropriate nonlinear model.

Logistic Regression

Logistic regression is a variant of nonlinear regression that is appropriate when the target (dependent) variable has only two possible values (e.g., live/die, buy/don’t-buy, infected/not-infected). Logistic regression fits an S-shaped logistic function to the data. As with general nonlinear regression, logistic regression cannot easily handle categorical variables nor is it good for detecting interactions between variables. Classification trees are well suited to modeling target variables with binary values, but – unlike logistic regression – they also can model variables with more than two discrete values, and they handle variable interactions.

Neural Networks

Neural networks (also called “multilayered perceptron”) provide models of data relationships through highly interconnected, simulated “neurons” that accept inputs, apply weighting coefficients and feed their output to other “neurons” which continue the process through the network to the eventual output. Some neurons may send feedback to earlier neurons in the network. Neural networks are “trained” to deliver the desired result by an iterative (and often lengthy) process where the weights applied to each input at each neuron are adjusted to optimize the desired output.

They are often compared to decision trees because both methods can model data that has nonlinear relationships between variables, and both can handle interactions between variables. However, neural networks have a number of drawbacks compared to decision trees.

Binary categorical input data for neural networks can be handled by using 0/1 (off/on) inputs, but categorical variables with multiple classes (for example, marital status or the state in which a person resides) are awkward to handle. Classifying a result into multiple categories usually is done by setting arbitrary value thresholds for discriminating one category from another. It would be difficult to devise a neural network to classify the location of residence into the 50 U.S. states. Classification trees, on the other hand, handle this type of problem naturally.

Neural networks do not present an easily-understandable model. When looking at a decision tree, it is easy to see that some initial variable divides the data into two categories and then other variables split the resulting child groups. This information is very useful to the researcher who is trying to understand the underlying nature of the data being analyzed.

A neural network is more of a “black box” that delivers results without an explanation of how the results were derived. Thus, it is difficult or impossible to explain how decisions were made based on the output of the network.

If a challenge is made to a decision based on a neural network, it is very difficult to explain and justify to non-technical people how decisions were made. In contrast, a decision tree is easily explained, and the process by which a particular decision “flows” through the decision tree can be readily shown.

For more information, do visit the links below!

http://databases.about.com/od/datamining/g/regression.htm

http://www.scribd.com/doc/6420211/Advanced-Data-Mining-Techniques

Credits

http://www.dtreg.com/othermethods.htm

Stacked bar/ column graphs

This is good if you want to display multiple instances of whole and its parts. It offer similar complexity to clustered bar charts by adding together component value items within chart bars. Through stacking items and assigning a different color to each item, you can effectively display trends among comparable or related items, or visually emphasize a sum of several indicators.

Bar and line graphs

The advantage of this type of graphs is that some data can be displayed using bars and some using lines especially for data which users would want to see the trend. For this type of graph, you can also use 2 scales so as to save space.

There is also Pareto charts where individual values are being displayed in bars and cumulative total of values displayed as a line. The steeper the slope of the line is, it means the greater the amount of the certain person/object is.

Line Graphs

This is useful when users wants to see trends. It is best for time-series data.

Box plots

This is used when the data with the same median and data range but different distribution. Box plots are useful to display differences between population without making any assumptions of the underlying statistical distribution. The spacings between the different parts of the box help indicate the degree of dispersion(spread) and skewness in the data, and identify outliers. Boxplots can also be drawn either horizontally or vertically.

Scatter plots

It involves more than 1 set of data . Data are being shown as a collection of points, each having the value of one variable determining the position on the horizontal axis and the value of the other variable determining the position on the vertical axis.

Arranging items on a dashboards

• Organise information to support its meaning and use
  It is important to organise groups according to business functions and use. Meaningful comparisons should be supported.
• Maintain consistency for quick and accurate interpretation
  Since a dashboard’s main purpose is to allow managers to view at one glance, it is important to maintain consistency for quick and accurate interpretation. The consistency of appearance as well as the display of the media is crucial as well. The information has to support its meaning and use too if not it would defeat the purpose of a dashboard.
• Making the viewing experience aesthetically pleasing
  You would have to choose colours appropriately as the choices of colours would affect the effectiveness of the dashboard. The right text should also be used, hence you should be careful when choosing appropriate words.
• Design as a launch pad
  Interactions are necessary, some examples would be drilling down and slicing.
• Test your design for usability
  You have to show prototype to users and allow them to test it. Let the prototype be the starting point for discussions and observe the users as they look over, so that you can improve it.

Credits

http://portfolios.knopf.com/portfolios/single-source/brio/ahtml/Charts16.html

http://en.wikipedia.org/wiki/Box_plot

Here are some useful websites:

This shows a winning entry of a  good dashboard design where we can reference to.

http://www.information-management.com/issues/20050901/1035522-1.html

This website shows ten key elements that is essential for creating an effective dashboard design.

http://searchdatamanagement.techtarget.com/generic/0,295582,sid91_gci1362216,00.html

In this lecture, I have learnt about data warehouse, data marts and online analytical process. The above mindmap is an overview of what I have learnt (Click to enlarge). There are 3 types of schema, star, snowflake and constellation.It depends on the situation of the organisation in order to decide which to use.

Data Warehouse

It contains a collection of tools that are used for decision support where large historical databases are involved. This would enable end-users to make quick decisions. All enterprise’s data are being stored there and also normalized in order to minimize redundancy.

Data Mart

Each data mart is a collection of tables organized according to the particular requirements of users. Sometimes retrieving a collection of different kinds of data from a normalized data warehouse can be complicated and time consuming.

Schemas

• MOLAP: OLAP storing the data in the multi-dimensional mode. To put it in a simplistic manner, there is one array for one combination of dimensions and associated measures. In this storage method there is no connect between the MOLAP database and data-warehouse database for query purpose. It means that a user cannot drill down from MOLAP summary data to the transaction level data of data-warehouse.
• ROLAP: OLAP storing the data in relational form in dimensional model. This is a de-normalized form in relational table structure. ROLAP database of OLAP server can be linked to the Data-warehouse database.
• HOLAP: Data is stored in the multi-dimensional model in the OLAP database and the transactional level data is stored in the relational form in the data-warehouse database. There is a linkage between the summary MOLAP database of OLAP and relational transactional database of Data-warehouse. This gives you the best of both the worlds.

Here are some websites which are useful:

http://msdn.microsoft.com/en-us/library/aa902672(SQL.80).aspx

This website explains in detail about data warehousing such as the users of data warehouse, designing the dimensional model and more.

Credits

http://www.essentialstrategies.com/services/warehouse.htm

http://searchoracle.techtarget.com/tip/0,289483,sid41_gci1169348,00.html

Dashboards consists of key performance indicators and metrics.

Metrics

When we use the term metric we are referring to a direct numerical measure that represents a piece of business data in the relationship of one or more dimensions. An example would be: “gross sales by week.” In this case, the measure would be dollars (gross sales) and the dimension would be time (week.) For a given measure, you may also want to see the values across different hierarchies within a dimension. For instance, seeing gross sales by day, week, or month would show you the measure dollars (gross sales) by differenthierarchies (day, week, and month) within the time dimension. Making the association of a measure with a specific hierarchal level within a dimension refers to the overall grain of the metric.

Looking at a measure across more than one dimension such as gross sales by territory and time is called multi-dimensional analysis. Most dashboards will only leverage multi-dimensional analysis in a limited and static way versus some of the more dynamic “slice-and-dice” tools that exist in the BI market.

Process mapping is to allow a better understanding, analysis and documenting processes and activities in an organisation. It also helps in assisting in identifying opportunities in order to improve the organisation’s business process.

Process mapping is a technique for graphical representation of work processes in a form of a process map. The process map shows all the process associated activities, including volumes of input and output, approvals, exceptions, and cross-functional hand-offs. The basic goal of the map is to provide a unifying vision of business processes, so that participating organizations and individuals can have an understanding of their specific role in the overall system.

Creating Process Map

When creating a process map, you need to identify the key process objectives and main activity steps. While collecting and analyzing information about the process, the factors to be considered are process complexity, number of organization and individual teams involved and time and cost issues. The process map should highlight bottlenecks and delays, rework, unnecessary steps and the authority ambiguity of the process. A good map would show the process flow from a customer’s point of view, showing whether each step of the process helps create a clear value for the customer.

Here are some of the useful links I have found:

http://www.scribd.com/doc/13268201/Simple-Process-Mapping-Techniques

This website is useful as it provides a clearer view of the whole entire process mapping. It also teach what are the things to be included in a simple process map.

http://www.visitask.com/key-performance-indicators.asp

This website, explains more about KPIs.

Credits

http://www.businessintelligence.com/article.asp?id=159

http://www.mind-pad.com/solutions/process-mapping-and-reengineering.htm

Firstly, Business performance management is about creating and sustaining processes for successful strategy execution, connecting processes, people, and performance. It encompasses a closed-loop set of processes that links strategy to execution in order to optimize business performance. Companies can then determine what is important to them and what they should monitor in order to sustain an effective and efficient organisation.

For the business performance management(BMP) closed-loop process, it consists of 4 steps:-

• Strategize

This is to create a strategic plan to analyse the current situation that the company is in. Also to identify the objectives and goals in order to know where they want to head to. The gap between strategy and execution is due to four causes, vision, people, management and resources.

• Plan

This refers to operational planning. An organization needs to carefully plan the allocation of resources in order to optimize the usuage to archieve strategic success.

• Monitor

This refers to monitoring the situation of the process and make sure that everything is heading to the right direction. This is where balanced scorecard would be used to align all the strategies together.

• Act & Adjust

In this stage, the organisation would predict future results or situations in order to be prepared for possible risks.

What is Balanced Scorecard?
Balanced scorecards are helpful to an organisation.  It is a strategic planning and management system that is used in business and industries to align business activities to the vision and strategy of the organization which would then improve both internal and external communications, and monitor organization performance against strategic goals.

The balanced scorecard is a management system as well as a measurement system that enables organizations to clarify their vision and strategy then translate them into actions.It provides feedback around both the internal business processes and external outcomes in order to continuously improve strategic performance and results. When fully deployed, the balanced scorecard transforms strategic planning from an academic exercise into the nerve center of an enterprise.

The four perspectives that are involved in a balanced scorecard:-

• Financial

Under financial perspective, timely and accurate funding data will always be crucial. Managers will do anything that is necessary to provide it. Often there is more than enough handling and processing of financial data. With the implementation of a corporate database, it is hoped that more of the processing can be centralized and automated.

• Customer

Under customer perspective, there has been an increase realization of the importance of customer focus and customer satisfaction in any business. These are leading indicators: if customers are not satisfied, they will eventually find other suppliers that will meet their needs (churning). Poor performance from this perspective is thus a leading indicator of future decline, even though the current financial picture may look good. In developing metrics for satisfaction, customers should be analyzed in terms of the different kinds of customers and processes for which they are providing a product or service to those customers.

• Internal business process

This perspective refers to internal business processes. Metrics based on this perspective allow the managers to know how well their business is running, and whether its products and services conform to customer requirements (the mission). These metrics have to be carefully designed by those who know these processes most intimately; with our unique missions these are not something that can be developed by outside consultants.

• Learning and Growth

This perspective includes employee training and corporate cultural attitudes related to both individual and corporate self-improvement. In a knowledge-worker organization, human resources are the main resource. In the current climate of rapid technological change, it is becoming necessary for knowledge workers to be in a continuous learning mode. Metrics can be put into place to guide managers in focusing training funds where they can help the most. In any case, learning and growth constitute the essential foundation for success of any knowledge-worker organization.

From what I have learnt about a balanced scorecard, there is a difference between the scorecard of a non-profit organisation and a profit making organisation. The sequence of perspectives would be changed due to the organisation as well as the theme would affect it. At the same time, before one starts to create a balance scorecard, they should develop a strategic foundation which consists of the organization’s mission, vision, values and assessment of both internal and external environment and the formulation of strategic themes. For strategic themes, it is very crucial to be focused and able to select only themes that will be able to close the strategic gap to organization’s destination. Other than that, it has to be execuable.

Here are some of the benefits of using a balanced scorecard :-

What is a strategy map?

It is a diagram of what the execution team believes will drive its strategy and it also shows a chain of objectives that will lead to successfully executing the strategy.

For additional information on balanced scorecard, you can refer to the following links:-

http://www.isixsigma.com/library/content/c011008a.asp

http://www.sas.com/solutions/bsc/

More information of strategy map:-

http://www.valuebasedmanagement.net/methods_strategy_maps_strategic_communication.html

Credits

http://www.balancedscorecard.org/BSCResources/AbouttheBalancedScorecard/tabid/55/Default.aspx