The Papers as a Collection

The papers present a variety of theoretical underpinnings including actor network theory, narrative as performance, and the panopticon concept.

Pollock and Crawford use a multidisciplinary theory set as a background to their research. The notion of ‘biography’ comes from material culture and describes the adaptation of artefacts as they move from place to place and the relationship the artefact has with the actors involved with it. This concept is coupled with actor network theory where artefacts are now seen as non-human ‘actors’ and also have a history. The claim is that these concepts allow the researcher to see relationship dynamics between various actors (both human and non-human) more clearly. This seems to be a valid theoretical underpinning as these concepts have been used before by other researchers and are accepted concepts within research.

Sia, Tang, Soh and Boh use the concept of the Panopticon to investigate the control aspect of ERP systems. Against this they use the concept of Empowerment to help them to investigate the positive aspects of the system. They supply several references to literature to support their use of both of these concepts. Again, the theoretical underpinning appears to be legitimate.

Wagner, Galliers and Scott use actor network theory alongside narrative research methods to investigate how the concept of ‘best practise’ is inscribed into a systems. This may well be a novel combination of methods, but both considered individually are accepted concepts to use in research and so the combination should also be acceptable.

The theoretical underpinnings of Orlikowski and Hofman and more difficult to pin down. They reference Argyris and Schon’s opinion and suggest that there is a discrepancy between how people think about technological change and how they implement it, and claim that this reduces the likelihood of success. This may well be true but it just doesn’t seem to have much substance to it, and whilst not having a theoretical underpinning may not necessarily be a bad thing, to a wary reader it does little to engender trust. Their proposed model which encourages ‘ongoing and iterative experimentation, use and learning’ is very similar to rapid prototyping, a concept used in systems design, however there is no mention of prototyping (rapid or otherwise) in the references.

The papers’ research periods, where it is given, span 6 months to 3 years. A variety of research methods are used including interviews, observations, and artefact collection.

Pollock and Crawford gathered their research material over 3 years, by sitting in on meetings and presentations, semi structured interviews with users and developers, document collecting, focus groups and observation of system testing. They haven’t mentioned if the implementation process only extended over those 3 years, so it is unknown if they were there for the first three years of the process, the last three years, three years in the middle or the entire thing. The exact time period they were able to collect material for is crucial to know as this will affect their findings and perhaps their validity.

Sia et al use a combined qualitative / quantitative methodology. The authors mention that the research began at the projects last phase of implementation. This may well limit the validity of their findings as they would have missed a lot of the political negotiations and concessions which occurred at the projects inception  and which may not have been formally documented. Six months were spent on data collection including a review of archival documents and undertaking interviews. Data was interpreted by researchers’ consensus. I can find no problem with the qualitative findings other than the fact that they started gathering the data quite late. Interviews are treated as truth, not as a performance as with Wagner et al.

However, there are relatively serious problems with the quantitative data presented in this paper. They use a 5-point Likert scale which is not further elaborated on in the paper but usually implies: Strongly Agree, Agree, Neutral / No Opinion, Disagree, Strongly Disagree. They then appear to have converted these into numbers on a scale of 1-5, to do statistical analysis. This is where the problem lies. If Strongly Agree = 1 and Strongly Disagree = 5 you get a certain set of results. However, if you code them in the reverse order, where Strongly Agree = 5 and Strongly Disagree = 1, you get a completely different set of results, with different numeric means. Unfortunately the statistics, notwithstanding the fact that they compensated for education, are meaningless: what is the average of 1 Strongly Agree and 3 Disagrees? – 2.75? 3.25? most people are kind of neutral? - all of which are inadequate representations of the truth (such as it is with statistics).

Wagner et al conducted their study over 14 months making five 8-week long visits to conduct interviews. Again we do not know how long the actual implementation lasted. The interviews are treated as a performance, not necessarily as truth, which then needs to be interpreted by the researchers. It is not clear however, how the researchers validated their interpretations.

The paper by Orlikowski and Hofman presents no methodology that I can find. They present the case study as a given, with no mention of how they got that information, how they verified its authenticity or how they compensated for potential bias from their source. Coupled with the vague theoretical underpinnings offered, the findings of this paper are starting to look very suspicious.

The papers differed in their consensus of what a success was. Wagner et al lean towards an unsuccessful interpretation of events as the grant accounting system  necessitated many workarounds and dual support of two systems.

Pollock and Crawford end their paper with a warning regarding adequate organisational conceptualisation in advance of implementation, user workarounds subsequent to  implementation and how the fit of the system to the organisation may correlate to the fit of the organisation within the sector.

Sia et al highlighted the fact that that the ambivalent potential of the ERP system, with the possibility of user empowerment, may be suppressed by management in favour of increased control.

Orlikowski and Hofman on the other hand, have noted extra employees resulting from the change, more paper work and relatively large amounts of organisational change, but the tone of the paper is that of a successful implementation.

Which leaves the question – what is a successful outcome? That the system works completely as planned and expected? That the system works at all? That there is no need for manual workarounds? That no extra staff were required? That the management are happy? That users are happy?

The notion of user resistance to ERP systems was revealed by Pollock and Crawford, Sia et al and Wagner et al, each independently of the other. The occurrence of user resistance therefore seems to be a strong trend and as a result the lack of it in the Orlikowski-Hofman paper is doubly suspicious. In fact, the people who work for Zeta are suspiciously ‘nice’ - ‘I don’t care who grabs credit for my work’. Unfortunately, no methodology has been presented which a reader can use to verify the authenticity of these claims and quotes.

In conclusion, the installation of ERPs is rarely simple or straightforward and often complicated, expensive and incomplete. ERPs often result in user workarounds, user dissatisfaction, extra staff and potentially, a system redesign.

Management are often let off the hook where the system is seen as being responsible for increased control and/or increased work.

ERPs are sold as being standard products, however organisations are invariably non-standard, with many locally negotiated practises which cannot be captured by the system. These standard products are espoused with the notion of best practise which may never have been rigorously and objectively tested for validity.

The success of a system depends in part on user buy-in. Users who don’t feel valued will invariably resist change. This may be true even if the system is technically suitable, however as no system in any of the papers was actually technically efficient at the process it was installed to do, this claim is unverifiable on the basis of the research in these four papers.

While the above is an outline of the important findings of the papers as considered together it must also be said that all of the papers, to greater and lesser extents, are limited in some way. Sia et al had a relatively short research period, began late into the project and incorrectly used Likert scales with statistics. Neither Wagner et al, nor Pollock and Crawford state whether they were able to cover the project from its actual inception. Orlikowski and Hofman have no defined theoretical underpinnings, no methodology, and suspiciously different findings

Doolin, B. (2004). Power and Resistance in the Implementation of a Medical Management Information System. Info Systems Journal, 14, 343-362 [Electronic Version]. Retrieved October 28, 2004 from http://www.emerald-library.com/ft

Gallacher, S., Williams, R. & Procter, R. (2001). The Politics of Intranet Usability: Can One Size Fit All? Usability, Politics and New Media. Springer.