The position of analytics in trade-off decision-making

“We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.”

– Pierre Simon Laplace, A Philosophical Essay on Probabilities. Whether the Daemon is disproved, or supported by, by Chaos theory, Quantum mechanical irreversibility or ‘Free Will’ in some guise (which itself may be theorised to be an emergent property of the two listed theories) is a fascinating discussion though the answer may not address why we find the Daemon so appealing. Personally I recall having Laplacian thoughts as a ten-year-old merrily considering a grand equation. Perhaps I was influenced by the following exchange in the 1999 Sitcom Spaced: Brian : Chaos Theory! Tim : Eh? Brian : The predictability of random events. The notion that reality as we know it, past, present and future is actually a mathematically predictable preordained system. Daisy : So somewhere out there in the vastness of the unknown is an equation… for predicting the future! Brian : An equation so complex as to utterly defy any possibility of comprehension by even the most brilliant human mind, but an equation nonetheless. Tim : Oh my God! Brian : What? Tim : I’ve got some fucking Jaffa Cakes in my coat pocket!

“The results show that failure avoidance was negatively correlated with self-efficacy, goal commitment, and task performance. The relationship between failure avoidance and performance was mediated by relationships with self-efficacy and personal goals. Goal commitment moderated the relationship between personal goals and performance. The results of this study are discussed in terms of Locke’s motivational sequence, suggesting that failure avoidance motivation, although overlooked, has important consequences in goal-setting situations.”

Failure Avoidance Motivation in a Goal Setting Situation (Heimerdinger & Hinson’s 2008). This extract is taken from the abstract of the paper. Hearteningly there is a rich literature on the topic of failure as motivation. This essay is aimed at the negation of uncertainty through analytics and the associated dysfunction of organisations that over-index on this heuristic whereas my brief survey all of the psychological studies evidences the presence of the motivation and its influence on capacity rather than the second-order issue of emergent dysfunction within organisations.

“Fear of failure is examined from a need achievement perspective and in the context of research amongst high school and university students. Theory and data suggest that fear of failure can be separated into two camps: overstriving and self-protection. Although each has yields in terms of achievement or in terms of self-protection, they render the academic process an uncertain one for students marked by anxiety, low resilience, and vulnerability to learned helplessness.”

Fear of failure: Friend or foe? (Martin & Marsh 2006). Another excerpt from the abstract of a paper. There is something a little cheeky about this form of referencing as the presentation of academic rigour in combination with a thrifty scan of the first paragraph seems slightly duplicitous… don’t you think?

“This analysis restricts risk seeking in the domain of gains and risk aversion in the domain of losses to small probabilities, where overweighting is expected to hold. Indeed these are the typical conditions under which lottery tickets and insurance policies are sold. In prospect theory, the overweighting of small probabilities favors both gambling and insurance, while the S-shaped value function tends to inhibit both behaviors.”

Prospect Theory: An Anlysis of Decisons Under Risk (Kahneman; Tversky 1979). Though pithier descriptions of Loss aversion are available this references the paper that introduces the theory and is therefore deserving of a mention. Kahneman is of course the Nobel prizewinning Kahneman of Thinking Fast and Slow fame amongst much else. The Nobel prize is not awarded posthumously and Amos Tversky was excluded from the 2002 cash windfall by six years. Kahneman said of the situation “I feel it is a joint prize. We were twinned for more than a decade.”

Organisations are comprised of functional components with specific inputs, transformations, outputs and networked influences. The transactional and functional information comprises the operational model and could therefore be optimised or changed through rational, objective, study.

Pure rationality is, regrettably, fleetingly rare given the necessary human actors and perspectives. Negation of our natural fallibility is achieved through the deployment of thinking models, personal tendency, governance and so on. These controlling elements serve more masters than singular address of this issue but do play a central role.

The transformative development of organisations requires evaluation and attention. This activity is polluted by multiple forms of biases that ultimately conspire to create a distance between that which should be worked on and that which is. This is a form of suboptimal change and different elements of organisations are subjected to this distance more than others to the extent to which organisational components are variously sensitive and susceptible to bias.

This bias is fairly predictable given a decent understanding of our nature. Activities that are perceived as artistic and creative are highly prized. Marketing, product and graphic design attract the attention of those in search of self expression and joy. Sales and other means of direct revenue generation pull the spotlight towards themselves due to there obvious capacity to create personal wealth, mastery of conversation and opportunities to travel. Cyber-security is attractive to readers of spy novels and people who are enthused by developments in technology.

The effect of bias on research teams is an interesting and subtle area of dysfunction and it’s explanation is the central discussion of the article. Research teams are those elements of enterprise that supply truth and disambiguation to their audiences. The contribution of value to organisations is realised through two mechanisms. The first is through the advantages offered in decision making (efficiency, scope and results) and the second is through the accumulation of insight-as-an-asset or Intellectual property (IP hereafter). This essay focussed on the role of insights within decision making though IP accumulation adheres to similar principles but is perhaps deserving of its own, private, analysis.

Where and how do our biases manifest around research? The discipline has received a recent boost in its sex-appeal due to the apparent opportunities of artificial intelligence though we, in the field, are hamstrung by the difficulties of data engineering in what seems to be a roughly 3:1 investment of preparation:AI in the analytic space. This limitation is likely to be negated through the increased adoption of blob storage (which is to say centralised unstructured data) as a standard. Still though this slight buff to the glamour of the industry is likely here-to-stay and seems to me to be deserved.

Beyond the zeitgeist the charms of business intelligence surround the topic of uncertainty. Uncertainty is most plentiful and potent in the future. Well developed organisations are in a constant state of scanning their history to develop better models becoming, with enough investment, Laplacian Daemons. Mechanistic understandings of organisational and market behaviours to offer advantages to organisations. Contact centres are able to plan for demand within a cost-effective corridor of predicted volume. Bridgewater Associates offers a consistent return on its portfolio based on constant modelling.

It would be wrong to suggest that there is not advantage available to organisations through the study of the past though it does seem that these models cannot cope with the established and demonstrable noise created through emergent complexity. Somewhere between the mechanistic model and the chaotic reality lies the analytic satisficient (a portmanteau of satisfaction and sufficient coined by Herbert A. Simon in 1956) middle that is both true enough and cheap enough to be useful.

We often cling to the need for research because it alleviates the reputational risks of decision-making. It seems that this disposition, frequently found within the gas-lift-swivel-chairs of England, is a manifestation of irrational loss-aversion. Put more plainly, people (especially those with office jobs) seem to want not to lose more than they want to win and are prepared to emphasise research activities asymmetrically to this end rather than in strict pursuit of advantage.

For some instinctive evidencing of this tendency to value loss more than gain, we might imagine that the same proposition posed in language relating to either loss or gain may have differing uptake even though the game is essentially the same. The literature (most prominently D. Kahneman’s work) on loss aversion seems to lead to the same conclusion. Please consider the following manipulative statements, their positive or negative phrasing and how this polarity influences the relative persuasiveness.

  • “Your fine is £100 if paid within the next 10 days it will be reduced to £5o.”
    • The language of loss is invoked as the offer will expire after the initial period. The opportunity not to lose is expiring and the recipient will ‘fail’ to take advantage more than they will exploit the opportunity.
    • A gain based formulation might be to say “Paying this £100 fine within the next ten days will save you £50.” This seems likely to be less compelling to most people even though the proposition is exactly the same.
  • “Quitting smoking will allow you to keep playing football.”
    • Here the language is positive and arguably weak. It seems that some of the impotence of these gain-based articulations is related to what we feel that we deserve given our homeostatic nature.
    • Consider the stronger, loss-based, formulation: “If you don’t quit smoking you will be unable to play football”.
  • “Stocks are limited, get yours before time runs out.”
    • Once again the language of loss is apparent here in the common marketing copy. The focus here is on the word “yours” which suggests that the target is already in possession of the item and need only perform the mere administrational effort of maintaining ownership.
    • The gain based articulation may be something “Stocks are limited, get one before time runs out.” his phrasing lacks the essential statement of pre-existing ownership and might, therefore, be considered to be a little weaker.

The power of this loss-emphasising-language serves as evidence of the intuitive sense of the potency of this polarity of argument. This tendency for us to avoid loss more than we seek to gain is present in everything we do and is especially visible within organisational decision-making. Though plenty of data exist to evidence this phenomenon it is instructive to acknowledge the presence of it within our intuitive responses to language as it is within these moments of reaction that dysfunctional decisions are commonly made.

The demonstrable loss-aversion bias has a powerful effect on many areas of organisational behaviour but particularly on the investment granted to business research. Optimal resourcing of this operation can be achieved through appropriate contextualisation of the enterprise within its larger aims. Too often though research is commissioned asymmetrically in service of preventing loss rather than to achieve advantage in its purest terms.

Given that people are likely to fear loss more than is strictly rational it follows that organisations may overemphasise loss-mitigation-based operations beyond what is optimal. Analytical functions within businesses are good candidates for this over-attention as the more effort that is poured into analysis the safer the decision that must be made is likely to be. Furthermore, it is hard to find the moment that the answer or analysis provided by analytical functions is accurate enough as the acceptance of the accuracy may well be determined, to some extent, by the organisation’s appetite for risk.

Unfortunately, this hyperextension of analytical functions in pursuit of ‘very-correct’ answers is doubly inefficient as gains in accuracy usually diminish over time and the misappropriation of the analytical resource limits the potential analytical gains that could be generated in other, less explored, areas.

Just as other attentional biases are mitigated through rationality; analytical functions must be controlled. In order to assist in this endeavour the following principles are humbly suggested:

  • Data analysis should be proportional to an organisation’s aims.
    • The organisation must consider its positional advantage. Is there more to be gained or lost? Is the organisation ahead of the competition in an exhausted market with everything to lose or is it the entrant into an expanding landscape? Furthermore, what is the organisation’s capacity to lose and desire to win? Given these considerations (and more) an organisation might consider it wise to conduct analysis in accordance with these parameters.
    • In order that analytical activity can be commissioned within the appropriate boundaries, a high-functioning organisation may wish to track the purposes of its analytical undertaking and adjust the strategic resourcing in accordance with some custom-weighted framework.
    • Before moving on from this we should note that as ever, models are wrong. Purely rationing analytical endeavours in accordance with organisational aims can produce sub-optimal results given other variables. We might imagine that business advantage is less attainable through loss-prevention than gain-seeking and therefore an asymmetry in the emphasis placed upon the analytical activity, This, and other, compromises are important considerations within organisational design.
  • The sufficiency of the analysis should be determined.
    • The commissioners of the analysis should work with the analysts to create a deliberate end-state so that it will be clear what level of accuracy is needed to be generated. This is not something that can be created without collaboration between these parties as the end-state must be satisfactory to the decision-makers and be analytically robust.
    • The format of the end-state conditions is worth dwelling on briefly. If the organisation is prepared to be wrong in one in ten similar decision-making situations the analysts may be inclined to aim for accuracy in their work that matches this statement. The way that organisations choose to think about what is good enough will shape the criteria itself and should, therefore, be a consideration.
  • Decision parameters should be understood.
    • Just as the sufficiency of the analysis should be approached deliberately the exact parameters of the tabled decisions need to be explored and stated. Without a full understanding of the choices that are intended to be made a full analytical operation cannot be designed.
    • There are always limits within which organisational choices can be made. If an organisation is not capable of launching a differentiated campaign across its markets then the analysis should be similarly constrained within this condition.
    • Though it is clearly sensible to invest analytical effort in those areas that are within the scope of the decision-making activity this must be approached deliberately as the pull towards the unalloyed good of having more information is a powerful distortion.
    • This is not to say that there shouldn’t be some additional analytical stretch made beyond the pure parameters of the choice when such additional information may have other beneficiaries; just that this should be deliberate. It is often the case that analytical activity generates knowledge that can be gainfully deployed in other areas. Long term strategy,
  • Modes of analysis should be deliberate.
    • When an organisation deploys analysis to increase certainty within a decision-making framework it should be understood that as the analysis progresses and the levels of certainty rise the mode of the analysis may change. It is quite common for a quick analysis and compilation of the known facts and parameters to be completed at an early moment of a large analytical project. From this point, deeper threads of understanding can be pulled at and the levels of certainty will rise. These distinct phases of an investigation are, necessarily, different and should be treated as such as there may be advantage available to organisations that prioritise certain elements of these analytical processes. Please see the diagram below for a worked example that demonstrates how this idea might usefully be deployed.

This essay demonstrates a key weakness in our ability to rationally commission analytical work as well as a few principles that may be gainfully deployed to strength our flawed reasoning. Though analytical endeavours provide powerful instruction in uncertain conditions the temptation to over-invest in a single question is alluring and may lead to inefficiency, or worse perhaps, missed opportunity.

The ultimate cure for the dysfunctions described here must be the construction and parameterisation of decision frameworks that can be optimised openly. The overindulgence of analysis in pursuit of unnecessary uncertainty can only occur in the dark and it is hoped that by allowing these endeavours to be treated like functional parts of a larger machine they can be exposed to beneficially supportive forces.

How to Meaningfully & Comprehensively Analyse Data in Three Visual Models.

“If you torture the data long enough, it will confess to anything.”

Ronald H. Coase – Essays on Economics and Economists. I also found this phrase listed as an idiom rather than a direct quote. For a full dive into Coase I can recommend reading into Coase theorem.

“Because our educational system is hung up on precision, the art of being good at approximations is insufficiently valued. This impedes conceptual thinking.”

Ray Dalio – Writing in Principles: Life and Work; a book of tremendous value and boundless wisdom. In my past year commuting on the London Underground I have seen only one person immersed in a copy which seems at odds with my perception of the book’s value. Perhaps I’m on the wrong train.

“War is ninety percent information.”

Napoleon Bonaparte – French Military and Political Leader. I’ve struggled to find a direct attribution for this popular quotation though I have happened upon a wonderful collection of his misquotations including the following sentences that are often wrongly attributed to the man “An army travels on its stomach.“, “No plan survives contact with the enemy.“, “An army of sheep, led by a lion, is better than an army of lions, led by a sheep.” and “Never ascribe to malice that which is adequately explained by incompetence.“.

“Errors using inadequate data are much less than those using no data at all.”

Charles Babbage – Inventor of the difference engine. This genius and father of computing also has the following, equally suitable, quotations “At each increase of knowledge, as well as on the contrivance of every new tool, human labour becomes abridged”, “The successful construction of all machinery depends on the perfection of the tools employed; and whoever is a master in the arts of tool-making possesses the key to the construction of all machines… The contrivance and construction of tools must therefore ever stand at the head of the industrial arts” and (brilliantly) “On two occasions I have been asked, — ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question“.

“The only people who see the whole picture are the ones who step outside the frame.”

Salman Rushdie – From The Ground Beneath Her Feet. A rather excellent expansion on the ‘think outside the box’ and ‘the spectator sees more of the game’ line of thinking that is beautifully pithy. I was lucky enough to spend a couple of hours listening to Mr Rushdi talk on his most recent book. The Q&A was mostly fatwarian in nature as you might expect.

Throughout much of his career, he led a double life: as an intellectual leader in the ivory tower of pure mathematics and as a man of action, in constant demand as an advisor, consultant and decision-maker to what is sometimes called the military-industrial complex of the United States. My own belief is that these two aspects of his double life, his wide-ranging activities as well as his strictly intellectual pursuits, were motivated by two profound convictions. The first was the overriding responsibility that each of us has to make full use of whatever intellectual capabilities we were endowed with. He had the scientist’s passion for learning and discovery for its own sake and the genius’s ego-driven concern for the significance and durability of his own contributions. The second was the critical importance of an environment of political freedom for the pursuit of the first, and for the welfare of mankind in general.

I’m convinced, in fact, that all his involvements with the halls of power were driven by his sense of the fragility of that freedom. By the beginning of the 1930s, if not even earlier, he became convinced that the lights of civilization would be snuffed out all over Europe by the spread of totalitarianism from the right: Nazism and Fascism. So he made an unequivocal commitment to his home in the new world and to fight to preserve and re-establish freedom from that new beachhead.

In the 1940s and 1950s, he was equally convinced that the threat to civilization now came from totalitarianism on the left, that is, Soviet Communism, and his commitment was just as unequivocal to fighting it with whatever weapons lay at hand, scientific and economic as well as military. It was a matter of utter indifference to him, I believe, whether the threat came from the right or from the left. What motivated both his intense involvement in the issues of the day and his uncompromisingly hardline attitude was his belief in the overriding importance of political freedom, his strong sense of its continuing fragility, and his conviction that it was in the United States, and the passionate defence of the United States, that its best hope lay.”

Marina von Neumann Whitman on John Von Neumann.

What data analysis is

Data analysis has been given an elevated position amongst the contemporary commercial disciplines. Though the analysts, data and questions are numerous the field itself seems oddly ill-defined. If two people were is asked to ‘analyse data’ it could not reasonably be expected that the outputs of their labour would be similar or comparable. This seems a strange state given the self-evident opportunities present in the conversion of data to information (and perhaps to insight) and the legions employed in this amorphous endeavour.

The ambiguity within the commercial ‘analytic’ world can be evidenced with only a brief search for ‘analyst’ job postings in, say, London wherein the analytical undertaking can mean anything from clerical work to large-scale strategic overhaul. Naturally there is a conflation between the convenience of the job title “analyst” and the actual act of analysing. These two elements do not necessarily overlap though the disparity and variation in the space seem to support the notion of widespread dubiety.

Given these starting parameters it seems reasonable to make an assertion of the primary models of data analysis within commerce (excluding R&D possibly) so that organisations might begin to share a common understanding of the work. The models proposed in this essay are all original and aim, in three parts, to address the role in generic terms. Just as a bricklayer might expect to arrive on site and contribute in a similar pattern regardless of the structure, substrate or supervisor these models should allow for a data analyst to turn up, fill herself with coffee, and work uninterrupted with the benefit of a common language of expectations between her and her commissioner.

In order for these models to function we must establish a quick summary of what we mean by ‘data analysis’. This list is shown below and has been kept as generic as possible to facilitate a wide-ranging application:

  • Data analysis is the conversion of the unknown into the known.
  • Data analysis is the aggregation of information to facilitate understanding.
  • Data analysis is a support structure to decision-making.

The three models

The models presented here each provide a different view of data analysis and are to be treated as overlays of each other as opposed to alternatives to each other. Each model presents the task of analysing data from a different perspective and therefore the adoption of all three should provide a comprehensive approach to the domain.

Each model is independent and users are free to deploy and discard models as they fit. Though each one takes aim at a different element of data analysis it is natural that situational necessity will dictate the utility of each model. These models are, furthermore, likely incomplete and should be included within strategy as part of an array of methods and expectations.

The three models are as follows:

  1. The Machine Understanding Model
    • This provides the analyst with a sense of purpose and reduces the total uncertainty experienced throughout the whole process.
  2. The Analysis Pathway Model
    • This model provides the analyst with a checklist-style output register that will assist in the provision of complete analysis.
  3. The Data Defence Model
    • This model asks the analyst to consider the output within the context of its presentation to the business in order to maximise robustness and utility.

Each model is explained in detail below with reference made to the diagrams provided. The diagrams are generic and serve as illustrations and aids to communication rather than strict, technical, blueprints.

The Machine Understanding Model

The Machine Understanding model proposes a five-step deepening of analytical effort that transports the analytical endeavour through the commissioning of analysis to the monitoring of the altered-state. In this case we imagine that there exists an element within an organisation, market or space that can be thought of as a machine with inputs and outputs. This machine may be improved in some way and the analysis is deployed in service to this improvement.

The ‘Investments’ shown here are simply the system inputs. This may be informational or financial but ultimately serve as a commitment of resources to an end. The ‘Outcomes’ are the desired outputs of the system. This may be revenue, customers, savings and so on. The diagrams show a flow of activity from left to right as the investments are transformed into desirable outcomes. The first diagram shows the initial conditions and the subsequent steps are labelled A through E.

A) Find the machine

Organisations are incredibly complicated. First and second order emergent properties are not only common but form the very bedrock of what we would usually perceive as simple. As a result the data analyst must first work to find the machine that she has been asked to study, understand where the edges of the machine are and understand the elements of the organisation or broader system that are not within this particular machine. This is, of course, a difficult and demanding occupation and should not be under appreciated; without a clear definition of the mechanistic scope a practical causal analysis cannot be undertaken within the practical limits of limited resources.

B) Identify known and unknown components

Within any complex system it is entirely possible that there exist multiple elements that must be considered by the analyst in order to fully understand the mechanism of the machine. It is, furthermore, quite likely that not all of these subcomponents will be fully explorable through data. This then establishes a plane of explorability throughout the subcomponent mix. Expression and quantification of this landscape is crucial at this stage as it will directly influence the viability of analysis and confidence in the remaining analysis.

C) Quantify component interaction

We now enter the exploration of causal influence and dependence. Provable, quantifiable interactions exist between subsystems and it is the analyst’s responsibility to define these relationships. Much of the analytical workload will focus here as the demonstration of influence is fundamental to an understanding of the mechanism in totality.

D) Model creative and exploitive alterations

Now the machine is understood and its mechanisms are quantified the analyst works creatively and collaboratively to model potential futures given certain manipulations of the machine. This process may be explorative or directed but will ultimately converge into an assertion of optimum state given potential alterations and known operation.

E) Measure machine performance

The final step assumes adoption of the most advantageous alteration. Within this element of the model the analyst is asked to monitor the actual activity of the mechanism against the forecast to both derive the efficacy of the implementation and the verisimilitude of the model. Within this model the analysts work is considered complete as an evidence-based change has been enacted within the organisation.

The Analysis Pathway Model

The Analysis Pathway Model charts an evolving list of outputs that the data analyst could reasonably be expected to produce. Each element of the model A through H is necessary to consider the analysis to be in any way complete. This is a model that has been in development for some time and should form the basis for most Commerical analyses.

Just as a house of cards collapses with the withdrawal of a single strut the model proposes a list of interdependent elements that span most analytical activity and the removal of any element allows for an entirely theoretical product. To remove a single point here is to introduce a potentially destabilising nature that may be unaddressed by the remaining elements.

The analyst deploying this model may wish to use this structure as both a backbone for the project plan and a delivery format for the final product, depending on the environment and audience, as the information flow has natural divergent, convergent format that facilitates productive communication.

A) Describe the data

The data landscape must be articulated by the analyst so that subsequent deductions are conducted within a clear and common Field of understanding. It is crucial that the available, and unavailable, data are described so that the following analytical effort can be fully understood. The analyst might, at this point, describe the, say, four data sets that will be utilised, where the data come from, the flaws within the data and the eccentricities of the information. Through this activity the remaining analytical effort is grounded within a known inertial frame of reference.

B) Show the aggregate

This stage represents the total analytical effort in too many cases. The statement of the mean, mode, harmonic mean… and so on, is the culmination of the thrifty analytical glimpse. Though the assertion of the aggregate, of one form or another, is plainly not without value this step must be deliberate in its descriptive statistics and it contextualisation within the subsequent steps in the model.

C) Show distribution

So the data collect around some point or another, but to what extent? This stage asks the analysts to describe the form of the distribution. This is, within the context of the model not just the step after C but rather a necessary component of B. Truly, the suggestion of an aggregate statistic must be balanced with some description of distribution in order to be considered useful. The challenge here is not the generation of the statistic but rather the communication of the significance of the measure.

D) Show comparisons

We now evolve the analysis beyond the first set of observations and contextualise our findings within comparable data points. This is where the analyst seeks a dimensionalisation of the initial findings. Examples of these dimensions might be temporal, geographical or segmental in some other fashion. I shall explore the dimensional analytical space in future articles as this topic demands a particular attention.

E) Show confidence

Assertion of confidence is necessary at this juncture as we are about to begin to assemble our findings. This step represents a moment to ensure that the error margins are established and quantified such that subsequent assertions are moderated in their potency given the analytical landscape. Without this step the recipients of the analytical products would be unable to ‘hedge-their-bets’ as is appropriate.

F) Derive meaning

It is at this point that the analyst is asked to extend their capacity beyond the mere presentation of fact but rather to synthesise the information into a coherent narrative. Within many organisations this step is regretfully ignored as analysts present data visualisations and ask for the commissioners of the analysis to draw their own conclusions. This model calls for the analyst to pursue the truth through the presentation of facts and present to their audience an assertion of meaning. It should be noted that these assertions are of course moderated via the confidence assertions made in the previous step.

G) Extrapolate future

Most analysis within commercial (and governmental perhaps) enterprises is deployed in decision-making processes. This then means that the findings produced by an analyst are indulged insofar as they are useful in the prediction of future conditions. The analyst must then take the opportunity to extrapolate the findings into the future so that the organisation is able to make optimal choices.

H) Articulate analytical opportunity

The model calls for, lastly, an articulation of the subsequent analytical opportunity. This is the moment that the analyst can pitch the absences and weaknesses of their own products in the context fo the total business opportunity. If the analysts has evidence of lacking data or transformational capacity then this moment can be utilised to build contextual requests.

The Findings Defence Model

The Findings Dence Model suggests that the analyst reflect on the way in which the analyical product is reveived at the conclusion of the activity. It is quite normal for data analysis to be greeted with a barrage of concerns and questions and this model isolates three areas of these for particular attention.

In this model we imagine that the analysis and findings exist as a central castle walled in by three defensive structures. Each structure is assualted by a particular concern and the analyst is asked to build the product in such a way to either defend against these misiles or deter the attack entirely.

It should be noted that this model posits the combative environment purely as metaphor and the manifest conflict is left to the users to determine. Just as the Human eye finds its resting place between two opposing forces this model hopes to balance the reasonable concerns of the organisation against the reasonable propositions of the analyst.

A) People looking for answers asking “So what?”

These people are keen to find the meaning in the analysis. The need here is to ensure that the synthesis of the data is meaningful and deployable. It is a shame that too many analysts present a selection of graphs to an audience without a series of express conclusions. The model suggests that unfulfilled recipients will require a conclusion from the analyst; this is entirely reasonable if the analyst has not taken ownership of the conclusion and considered the final empirical judgement to e part of their output.

B) People looking for problems asking “What about?”

“What about” is somewhat of a canary on the dysfunctional coal-mine. I would suggest that if you encounter such a conversation you refer to the essay on the dysfunctional measurement. Although this line of question i s normal within most organisations it illuminates either a failing in the analytical scope or a communication problem throughout the organisation. I, at the moment of conclusion, the analyst is presented with a new parameter… something has gone wrong.

C) People looking for action asking “Are you sure?”

Finally, we arrive at a relatively simple yet pervasive issue; the problem of confidence. Simply put, it is unreasonable for organisations to act on data Without some assertion of confidence. As a result, if these questions are being asked of the analyst it is clear that the confidence has not been articulated clearly.


The models provided here may assist in the formation of a general understanding of the nature and expectations of the discipline of data analysis within contemporary commerce. Though these models do not represent a comprehensive articulation of the discipline it is hoped that these three ideas might be humbly offered to the analytical masses and deployed to whatever degree is most appropriate.

The Detection and Quantification of Organisational Dysfunction

“I can prove anything by statistics except the truth.”

George Canning – Very briefly Prime Minister who also said the rather beautifully nuanced “with keen, discriminating sight, black ’s not so black – nor white so very white.” I won’t try to do the man any biographical justice in this small section though my quick reading on the Canning has proved deeply interesting. He is the shortest serving PM of the UK dying in office on his 119th day in office. He’s regarded as somewhat of a ‘lost leader’ due to the swift decline in his health and outstanding oratory.

“In the most dysfunctional organizations, signalling that work is being done becomes a better strategy for career advancement than actually doing work (if this describes your company, you should quit now).”

Peter Thiel – co-founder of PayPal and Author of a couple of books, one more controversial than the other.

“You aren’t a machine with broken parts. You are an animal whose needs are not being met.”

Johann Hari – This is a great summation of his book Lost Connections in a single sentence. I thoroughly enjoyed this book and found its description of and recommended solutions to depression and anxiety disorders highly compelling. I was disappointed to see these ideas rejected by a great number of people who found these ideas to be either a restatement of existing thoughts or ignorant of current academic standards. I’m unable to validate either of these concerns and am to this day convinced that this is an urgent, useful and informative book.

“It is no measure of health to be well adjusted to a profoundly sick society.”

J. Krishnamurti – Highly quotable philosopher, speaker and writer. This is one of those phrases that has escaped its author and has become somewhat of an idiom or maxim more than a quotation. Fundamentally it does not need Krishnamurti to say it as it would be equally profound even if said by other people (try “It is no measure of health to be well adjusted to a profoundly sick society.” – Paris Hilton, or “It is no measure of health to be well adjusted to a profoundly sick society.” – Homer Simpson, perhaps even “It is no measure of health to be well adjusted to a profoundly sick society.” – Kanye West).

Is function different from dysfunction?

How dysfunctional is your organisation? To what extent is this question different from asking ‘how functional is your organisation, and, how do we determine the extent of this dysfunction so that we might measure it over temporal or structural dimensions? These are questions that my recent reading has failed to answer and I am tempted to think that this topic is somewhat underdeveloped. Strategic optimisation and decision-making thought takes a level of dysfunction as a background and moved deliberately away from it.

Measurement and empiricism seem to be often focussed around the benefits achieved given the imposition of optimal tactics and I have not yet observed a committed effort to analyse dysfunction within businesses. This seems a shame as I would suggest that the challenges of measuring how well something works are distinct from how well something doesn’t work rather than presenting chirally. Opposites are sometimes more interesting than reversals just as the rightnesses and leftnesses of our hands are maintained as they are rotated through space.

In order to explore the idea that measuring the extent of dysfunction might be different from measuring the extent of function we may consider the necessities of medical diagnosis wherein we find that measurements of sickness are distinct from wellness. We study the particular manifestations and presentations of certain conditions in a way that we would not be compelled to do for a patient who is not suffering from that disease. When someone is well we do not call for a battery of blood tests as we don’t need to see in which way they are well in the same way that we may want to see in which way someone is not well and is, of course, a product of diagnostic necessity. Though there are some tests that are administered to people who are well and otherwise we can see that there is a need for specific measurement of dysfunction in order to define effective corrective intervention.

Things can exist in different states of function and dysfunction and the measurement of these is not simply an extrapolation of either state into its polar opposite. This is to say that the instinct to measure states along continua may be misleading given some amount of complexity or specificity. Why then in business are we more compelled to measure function as a proxy for dysfunction when there may be particular readings within broken machines that may be of significant use? This may be an artefact of optimism or the self-evident truth that we must fix objectives in place with indicators of performance and the positive assertion of gain is more advantageous than the quantified eradication of dysfunction.

If we were to measure the state of an organisation through deliberate quantification of its dysfunction expressed in terms beyond the absence of function how might we do it? The visual model below shows a possible model that could be used t0 describe the main principles of organisational dysfunction. For the purposes of this essay we will accept a model of distinct elements that must co-operate and co-ordinate to realise goals as our shorthand for organisations. It should be noted at this point that there are of course other models available and reality far more complex than can be captured within this sketch.

The machines above show distinct types of dysfunction. These are provided as a framework upon which to hang a system of measurement of dysfunction. The machine in question is a simple lever-switch wherein a force is applied to the “Tilting part” which then makes contact with the “Sensing part”. This machine is provided for illustrative purposes only though is a useful metaphor in a few ways such as the distinct operation of its components and the fragility of its dependencies which can easily be manipulated into failure.

The machine has a simple set of three components that conspire to connect the ‘Tilting part’ to the ‘Sensing’ part. The functional machine is shown in the first illustration with the different types of dysfunction laid out in figures A through G. I doubt that this is an exhaustive list of the dysfunctional permutations, and I am not convinced that the list does not include some duplication or conflation of types. We can see that there are many more ways in which the machine can be broken, to some extent, than working and some of those broken states are more broken than others in that the machine’s goal may be met in a sub-optimal (or differently optimal) manner than intended.

Each of these types of dysfunction can be considered archetypes of dysfunction that could be detected, quantified and addressed in their own way. This taxonomy of dysfunction may be a more optimal route to functional organisations than a focus only on the extent of functionality. These archetypes are explained below with possible forms of measurement and detection. Please note that the quantification notes that appear here are, broadly, abstract to account for organisational differences. I should imagine that the realisation of these principles within dysfunctional organisations would require the substitution of vague terms (say ‘output’) with concrete metrics (say ‘billed-orders’).

Dysfunction of organisation (fig A)

The machine is unable to operate as the way in which the component parts are organised (in this case ordering is used as the metaphor) means that any functional subcomponent is unable to produce a meaningful result. This may be considered to be something of a leadership or strategic failing. Each component is functioning in a manner that is desirable but the net of all activity is ultimately ineffective due to some high-level failure of design.

Measurement of dysfunctions of organisations may be achieved in the following ways:

  • Net-output over combined-component-output.
  • Leadership understanding of component purpose.
  • Organisational similarity to best-practice or comparable high-performing organisations.

Dysfunction of alignment (fig B)

The machine is unable to function as its ‘Pushing part’ is pushing in the wrong direction. An insufficient review of this component may deduce that it is functional insofar as it is indeed pushing though a contextual understanding of the total purpose of the machine is needed to understand whether the component is really contributing. I expect that this is a common form of dysfunction and is has an intuitive attractiveness to most analysts of this issue. As an example we might consider a research team that is pursuing its own intellectual curiosity rather than the necessary components of marketable products. The research unit may be highly functional though disastrously misaligned to its necessary function.

Measurement of the dysfunction of alignment might be measurable in the following ways:

  • Sub-component understanding of organisational goals.
  • Sub-component contribution to organisational output.
  • Organisation adoption and deployment rates of sub-component output.

Dysfunction of absence (fig C)

The machine is unable to function due to a missing part. In this case the perfectly functional “Pushing part” and “Sensing part” are unable to affect total functionality due to there being no “Sensing part”. In organisations, this type of missing capability can render the activity of component parts to be strategically worthless. This seems to be common where organisations are asked to translate creative outputs into marketable products. The variability of creative output can present a challenge to even the most capable organisations wherein there is no guarantee that the creative products can be taken to market.

Measurement of dysfunction of absence may be achieved through:

  • Organisational similarity to best-practice or comparable high-performing organisations. (This is perhaps less useful than earlier as missing components may become more apparent given rapid change, upscaling or adaptation to unfamiliar or unknown environments.)
  • Capability-blocked initiatives over all failed-initiatives.
  • Processes with operating models over all processes.

Dysfunction of method (fig D)

The machine is suffering from a single component carrying out its action in a way that is counter-productive as a whole. The ‘Pushing part’ is aligned to the organisational goals in that it knows that it must provide a downward force to the ‘Tipping part’ though the way in which it does this is ultimately catastrophic. Examples within organisations may include phasing issues, technological barriers, communication issues and hierarchical problems. This is an interesting and challenging manner of dysfunction as nearly everything is working as it should it is simply the interaction between components that is flawed.

Measurement of this methodical dysfunction may be acheived as follows:

  • Sub-component understanding of dependent sub-components
  • Negotiated sub-component interactions over all sub-component interactions.
  • Variability of sub-component interaction duration

Dysfunction of bypass (fig E)

The machine is sub-optimally functional (as is machine G: Dysfunction of optimisation) as the goal of the machine is achieved at the expense of operational harmony or utilisation. In this case the role of the “Tipping part” is being carried out by the “Pushing part” directly. This has made the “Tipping part” essentially redundant even though the dipping part may be better suited to the function of transferring pressure to the “Sensing part”. This machine is dysfunctional right now as the redundancy is not addressed. The design of the machine may mature into a more efficient mechanism over time, suffer on with the vestigial component or collapse revealing the inadequacies of the over-reach. These sorts of bypasses are common in organisations that contain malleable, helpful, components as well as expansionistically-minded individuals.

Measures of bypass dysfunction may include:

  • Shared sub-component objectives over all sub-component objectives
  • Capability auditing of subcomponents
  • Information flow multiplication rates between sub-components

Dysfunction of action (fig F)

The machine is unable to function as the “Pushing part” is unable to generate a downward force on the “Tipping part”. This machine is unable to operate a single element has had a catastrophic failure of some kind that has resulted in an ineffective whole. Within organisations this occurs when work is not executed by those responsible for doing it, this might well be a cultural, managerial or capability-derived issue.

Measurment of this dysfunction of action could be handled in these ways:

  • Sub-component objective failure rate
  • Capability auditing of subcomponent and cultural measurement of some kind (Atrophy, E-Sat e.t.c.)
  • Investment in sub-component over planned investment

Dysfunction of optimisation (fig G)

Lastly we can see that this machine is functional though it seems that the “Pushing part” is generating more force on the “Tipping part” than the machine requires. This may be seen as a bottleneck and may be representative of an over investment in one area or an under-investment in another. This may or may not be the opposite to the ‘dysfunction of action’ depending on the reason for the over or under performance. The ‘dysfunction of optimisation’ machine is dysfunctional not because is incapable of producing the desired outcome but rather because it does so inelegantly and inefficiently.

Detection and measurement of the dysfunction of optimisation may be measured in the following ways:

  • Backlogged initiatives over completed initiatives (and rate of change)
  • Investment in sub-component over planned investment or advised investment
  • Sub-component output variation and total output over time dependent subcomponent output variation and total output.


The simple diagrams shown in this essay are merely abstractions that exist to enable a description of an element of dysfunction. In real organisations the mechanisms that govern input, output and performance of sub-component systems are far more complicated. These small illustrations allow for the generation of principles that appear in reality as emergent phenomena subtly nested within functional and dysfunctional processes. Each of these dysfunctions may manifest to some extent, in collaboration with, obscured by or presenting as some other dysfunction. The real machine may in fact look more like some grand fractal network existing in some form of n-dimensional space. Perhaps more like the image at the header of this essay wherein each connection may be functional or dysfunctional as is every combination of every connection.

The world can only every be ’tile-over’ with models like the ones in this essay. Reality is too complex to submit to these crude taxonomies though it is my hope that this examination of dysfunction may be of some use in the future. It seems clear to me that there exist divisions between organisational dysfunctions that can be usefully articulated through distinct patterns of measurement. I plan to build on this work in the future, refine this model and see what can be learned about organisations through this understanding.

Providing context to further posts and solving structural and formal writing problems.

“If you read only one book per month, that will put you into the top 1% of income earners in our society. But if you read one book per week, 50 books per year, that will make you one of the best educated, smartest, most capable and highest paid people in your field. Regular reading will transform your life completely.”

Brian Tracy – Self-development writer and entrepreneur

“Since brevity is the soul of wit / And tediousness the limbs and outward flourishes, I will be brief…”

Polonius – Chief counsellor of the king, and the father of Laertes and Ophelia. Hamlet may have popularised the phrase more than created it perhaps.

If I had a nickel for every person who ever told me he/she wanted to become a writer but “didn’t have time to read,” I could buy myself a pretty good steak dinner. Can I be blunt on this subject? If you don’t have time to read, you don’t have the time (or the tools) to write. Simple as that.

Stephen King – This quote seems to come from his book ‘On Writing: A Memoir of the Craft’ and is much repeated on the internet. Naturally, I have not yet read this book.

Language is the tiling-over of experience

Words to the effect of the above came, I think, from the neuroscientist/philosopher/podcaster/author Sam Harris though I’m presently unable to find the exact source.

“Talking is thinking; the externalisation of our internal cognition and the application of the tools of vocalisation should be understood to be part of thought and not as distinct from it.”

Daemon Towndrow – Musician and data analyst. I visualise this concept comfortably as an expansion of the area in which thought happens beyond the skull by a couple of feet.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

Antoine de Saint-Exupéry – French writer, poet, aristocrat, journalist, and pioneering aviator… what a wonderful list of accolades. He has many other useful quotes “Language is the source of misunderstandings“, “A goal without a plan is just a wish” and “What makes the desert beautiful is that somewhere it hides a well” stand out.


In order to create a consistent product (and indeed to create consistent productivity) I will use this first post to work through a few of the problems I suspect that I am soon to incur in the writing of this blog. I will take a brief moment to establish purpose and clarity and in doing so facilitate an easier creative process for myself. I hope to unburden this blog to some extent so that it needn’t become all things to all people and can instead occupy its own comfortable, useful and appropriate niche. This post will therefore cover the following ground:

  • Purpose of the blog
  • Editorial focus
  • Citations and references
  • Formal elements
  • Structural considerations
  • Creative concerns
  • Language, proofing, standards and revision

These seven elements will form the basis of the first post. This will, with luck, provide context for the new reader and a series of hooks from which to hang the project in general.

Purpose of the blog

A number of the books I have read recently make arguments for purpose ahead of much else. It is a common and self-evidently useful insight that highly functioning individuals and collaborative enterprises should have a declared purpose. It is, therefore, regrettable that so few activities are guided by such an explicit statement from their outset. Just as a toothpaste manufacturer helps their customers lead fulfilling romantic lives (through confident bright smiles, long-lasting fresh breath and hygienic kisses [and not simply a lightly abrasive paste with fluoride and detergents]) I aim to give the reader…

A more successful working-life through improved decision-making.

This purpose is then approached through the following mechanism.

Short articles exploring problem solving, cognition and analysis with a primary focus on the application of these principles to commerce.

Naturally, there are smaller goals and purposes nested within the two larger thoughts above. In the interest of the exorcism of these demons these are laid out in the following list.

  • Capturing and storage of thoughts that are usually left to my own, inadequate, recollection.
  • The forced synthesis and appraisal of my own reading into informationally dense artefacts.
  • The creation of a ‘self-portrait’ or diary of sorts.
  • Scratching the creative itch and the need to deploy my reading and thinking into an immediate product rather than manifesting it opportunistically in my working life.
  • The development of my own linguistic faculty.

Editorial focus

The blog will focus on the following areas:

  • Psychology and its application to commercial activity. I suspect that this element will veer towards what people call “Pop Psychology” insofar as the author has no formal expertise in this area.
  • Commercial disciplines such as project management, change management, technical development, data analysis, selling and so on. These office skills are, I believe, an aggregate concern.
  • Personal development, creativity, emotional growth etc. Are these perhaps nested within Psychology? Potentially so, though I feel that calling out these will help the new reader.

Citations and references

Citations will be given where possible. It is not the intention of the blog to obtain intellectual credit for the work of others and effort will therefore be taken to provide references. That said, this is not an academic exercise and it is freeing to be able to write in a journalistic, opinionated, fashion and I will therefore seek to balance these approaches.

It seems to me to be abundantly possible that I may write a short article based on ideas and research that have disintegrated in my own memory to form a fragmented backdrop to my thinking. I apologise in advance to the original authors of what seem, to me at least, to be my own thoughts.

Formal elements

The form on this blog will be short articles supported wherever possible with visual aids. Language will be prosaic and readable, business jargon and cliché are to be avoided wherever possible.

Structural considerations

Articles will be short. I tend towards the verbose in general; I will try to limit the length of the output in favour of readability and frequency. This will make articles less impressive in isolation and more impressive, potentially, in aggregate.

There will be a list of quotations in the header of each article. This quick assembly of thoughts allows for the creation of an abstract conceptual space within which I can create the article. I feel that this is a useful shortcut for us both, the author and the reader, to assemble a common understanding and background. I will attempt not to directly reference these quotations understanding that they are connected tangentially to the line of thinking taken in the article.

Articles will contain a summary at the end in a simple list format. This will increase the retention of the reader through the simple repetition of the article’s main points. Furthermore, these summaries will provide a quick reminder of the contents of the article if the reader is rescanning the content.

Creative concerns

This project will, at times, create some elements as well as simply synthesising the work of others. In these moments I gently ask for the readers’ patience and confidence. It is all to clear to me that whatever education I may be able to bring to a topic will be deeply inadequate when compared to the experiences and expertise of the professionals working within the domains that I intend to cover.

Clearly, there must be a moment in the assembly of information on a topic where the working knowledge is ‘good enough’. I hope to use this blog to explore a number of concepts and feel strongly that I will need to acclimatise myself to the imperfect and bold act of writing about something before all the available material has been consumed.

This experiment will require balance. I must carry out sufficient research confidently report back to the reader whilst not paralysing the process through imposing the restriction of impossibly high standards.

Language, proofing, standards and revision

No editors, proof-readers or consummate literary professionals are employed in the service of this blog. Typos, inadequate communication, syntactical and grammatical errors are anticipated to be commonplace. These will be corrected when discovered without notation.

The blog may make changes as required over time particularly in the case of conceptual revision. Effort will be made to ensure that such revisions are clearly shown.


  • The blog seeks to help people make good choices in their vocation
  • The blog will reliably contain references though is not an exercise in academic writing
  • The blog opens with quotations that serve as a backdrop to the article
  • Blog posts are closed with bullet-pointed summaries