282 Flawed evidence : a case study of misquoting and inaccurate referencing

The need to quote and reference accurately is crucial to academic writing and debate. While small errors are excusable, obvious interpretations, alterations, additions and deletions are not, unless indicated. In addition, primary texts are preferred: if secondary texts are used, this should also be clearly indicated. This article details a case in which these rules have been ignored. It begins by identifying the scope of the particular case: the mis-quoting of Michaels Gibbons. It then traces the route of the inaccurate quotation as it moves from its source through various influential documents and journal articles, until it is firmly established, and easily cross-referenced, but as a vague resemblance of the original, not recognised by the original author. Although this article deals with this one case, it is obvious that the circumstances are not unique, and that there is little reason to suppose that these events cannot be replicated elsewhere. This case illustrates the need for writers to always, where possible, refer to the original documents when citing them.


Introduction
In academic writing, the need to quote and reference accurately is a given.To ensure accuracy, reference systems are designed in great detail, and, from researchers' days as students, the methods are enforced.It is accepted that small errors occur (and may even exist in this paper).While plagiarism is often the main concern (Lampert 2004: 347), the overall goal of accuracy goes further than giving quoted authors due credit.It includes the need to ensure that they are not mis-quoted to the extent that information obviously not intended is mis-attributed to them, and that other readers can source documents for their own research.
There is a concurrent need to reference primary rather than secondary sources, where possible.(Historically, 'primary documents' are items such as letters and private journals.In this instance,this paper usesthe term more broadly, and includes any document that becomes a source of information for other, secondary documents.)Referring directly to these primary documents, rather than to the secondary document, lessensthe chances of errors' being compounded, and encourages the original context and meaning to be retained.In practice, if the primary source is not available, then the fact that the reference is through a secondary source should be stated explicitly.This paper examines the caseof quotations from and references to a document written by Michael Gibbons.We begin with an overview of the problem by comparing the primary document to a document apparently quoting from it four years later.We then attempt a trace of the relevant sections from the primary document to secondary, influential documents, all of which cite the primary document astheir source.Finally,we view the quoted sections in journal articles and conference presentations.In addition to the small typographical errors that occur in these documents (which are not of great concern here), one seesthe impact of the initial error compounded by the fact that most of the documents imply that they are citing from the primary document while it is obvious that they are not.Furthermore, becausethe process continues today, one can surmise that it will extend beyond the sources quoted in this paper.

The problem
In order to achieve an overview of the problem, we shall begin by looking at the material in the primary document, and compare it to material in a document published four years later.

The primary document
In 1998, Michael Gibbons, Secretary General, Association of Commonwealth Universities, published a World Bank text titled Higher Education Relevance in the 21 st Century.This is not the place to summarise Gibbons' work; it is sufficient for the purposes of this paper to point out that, in this text, as elsewhere (Gibbons 2000a;2000b), Gibbons describes a change in Higher Education which he labels 'Mode 2' (contrasting with 'Mode I' which describes the current dominant approaches in higher education institutions).Amongst other things, Mode 2 advocates that higher education institutions I. Ken Masters is a senior lecturer in IT Education, in the Faculty of Health Sciences, University of Cape Town.& Info Sci 2005, 71 (3) need to concern themselves with an approach that is guided by knowledge produced in the context of application, is transdisciplinary, heterogeneous, socially accountable and reflexive (Gibbons 1998: 5;2000a: 35;2000b: 133-139).

SAJnl Libs
In Gibbons' 1998 document, there is a discussion about the skills required in the 2\ st century.One section is written by a guest author, William Saint, who asks: Are development assistanceefforts right to place so much emphasis on increasing enrolments in science and technology, or should they be emphasising curriculum reform to develop the skills needed for effective knowledge generation and problem-solving under the new paradigm: team-building, group creativity, information management, networking, communication, negotiation/mediation, and social sensitivity?(Gibbons 1998:57) Later, Gibbons himself writes: What kind of citizens will by [sic]  Setting academic protocol aside for a moment, one might be tempted to argue that the Broere and Kruger article is a small conference article, and therefore of little consequence.One isolated incident is of little concern.Although it is difficult to defend such a position, even if it were defensible, we would have to be sure that the Broere and Kruger article is an isolated incident.
In addition, this position assumesthat Broere and Kruger summarised the Gibbons text, added their own ideas, and, through redrafts, created the errors.A second possibility exists, and, it will be argued, is more likely.Broere and Kruger did not consult Gibbons at all.They consulted secondary texts, and those texts had made the changesearlier.

Methodology
A search for journal articles, abstracts and other documents, using a combination of Academic data bases (accessed through EBSCOHost), and general search engines (Google, Alta Vista) revealed several documents with references to the Gibbons skills.All these documents, including the original Gibbons text, are available on the Internet, so verification is relatively straight-forward.
The documents were then arranged according to: • whether or not they referred to one another, and • date of publication In order to ensure that Broere and Kruger's skills list had not been sourced from a different Gibbons text, the author of this paper contacted Michael Gibbons directly.In personal correspondence, Michael Gibbons confirmed that the list of skills described was not his (email dated 6 October 2004).

Results
Document 1: '1998/1999 Annual Report of the Council on Higher Education (South Africa)' (1999) It would be difficult to find a definitive and original source, but one early document, from 1999, is a strong contender.It is the 1998/1999 Annual Report of the Council on Higher Education (South Africa) (CHE), which states: Ultimately, higher education institutions are challenged to generate the competencies that will be required from all graduates during the 21st century -computer literacy, knowledge re-configuration skills, information & Info Sci 2005,71(3) management, problem solving in the context of application, team building, networking, negotiations/mediation competencies and social sensitivity (no page numbers; p.25 of pdf document).This document does not cite the source of this statement in the text.It does, however, list the Gibbons article in its sources.

SAJnl Libs
Document 2: 'Increased Government Support for Higher Education ' (2000) The list is picked up in May 2000, by Nico Cloete, writing in 'Increased Government Support for Higher Education.'This is a publication of the Centre for Higher Education Transformation (CHET), and was commissioned by the [South African] Council on Higher Education.He writes Gibbons in his World Bank paper (1998) summarises these skills as: computer literacy, knowledge reconfiguration skills, information management, problem solving in the context of application, team building, networking, negotiations/mediation competencies and social sensitivity (HTML page 6).
Cloete gives Gibbons' 1998 document in his list of references.
This list is identical to the CHE list to the point that the probable typographical error in 'negotiations/mediation competencies' is duplicated, and the hyphen in 'team building' has been dropped in both documents.Apart from the chronology, the flow of the copy from CHE to Cloete is supported by the fact that the hyphen introduced in the CHE document ('re-configuration'), as a result of the word's over-running a line, is retained in Cloete's document even though the pagination does not require it.
There is no suggestion in either document that the list is a subset of Gibbons' skills, or that any editing has occurred.
There is also no mention of William Saint in these two documents.A small but important difference is that, in Cloete's document, the concept of the summary is introduced, and it is attributed directly to Gibbons.
At this point, it is worthwhile noting the major changes to Saint's and Gibbons' writing.These documents • introduce the concepts as a single and coherent summary or list of terms • introduce the term 'computer literacy' • adapt variations of the reconfiguration and re-configuring of knowledge elsewhere in Gibbons' document to 'knowledge re-configuration skills' and introduce a hyphen into 're-configuration' (probably as a result of the word's over-running a line in the CHE text) • omit these terms: group creativity, communication, cross-cultural communication, cultural adaptability, problem analysis (although possibly subsumed in 'problem solving') and self promotion.
• simplify 'problem-solving under the new paradigm' to 'problem solving in the context of application,' ignoring the other facets of the new paradigm.
It is worth noting that the closest Gibbons comes to advocating computer literacy is the single sentence reading 'In the future it is likely that [students] will be also taught through computers, videos, television, and at a distance' (16).If this statement is taken as advocating computer literacy, then it must surely also advocate video literacy, television literacy and distance-learning as skills to be essential to the graduate.This is ignored by both the CHE document and C1oete.
It might be argued that C10ete did not refer directly to the CHE document, but that both documents used a common third source.No other document has yet been found, and in any case, this does not alter the strong link between the two documents and the fact that Saints' and Gibbons' work has been mis-represented.
Document 3: 'National Plan for Higher Education in South Africa' (2001) In February 200 I, the South African Ministry of Education published its 'National Plan for Higher Education in South Africa' ('the Plan.')In Section 2.7, it states: The skills that all graduates will require in the 21st century have been aptly summarised by Michael Gibbons as computer literacy, knowledge reconfiguration skills, information management, problem-solving in the context of application, team building, networking, negotiation/mediation competencies and social sensitivity (Gibbons: 1998) (Section 2.7).
The Plan lists the Michael Gibbons' text in its list of references.Because, however, the list is identical (with minor, tidying corrections) to that in the CHE and Cloete documents, it is unlikely that the Gibbons text was consulted.It was certainly not quoted.
From this point on, there is a shift in the dynamic of the quotation.Because the Plan is an influential document in South Africa, it is widely read by educators, and will be referenced in many educational articles.The chief difference between these articles and the previous documents is that the Plan becomes the primary source, although Gibbons The Plan goes on to quote Michael Gibbons who saysthe skills that all graduates in the 21st century require are "computer literacy, knowledge reconfiguration skills, information management, problem-solving in the context of application, team building, networking, negotiation/mediation competence and social sensitivity" (Ngara 200 I :46).
There is a vital change in Ngara's document not found before: the list has now been placed in quotation marks.From the sense in the Plan, it is understandable how Ngara would have interpreted it as a quotation.Ngara, however, does not place Gibbons in his list of references, making it clear that he is not claiming to be referencing Gibbons' text.
Document 5: 'From School to Higher Education?Factors Affecting the choices of Grade 12 Learners' ( 2002) In a Human SciencesResearchCouncil (HSRC) publication, Michael Cosser and Jacquesdu Toit write: In this regard, the National Plan makes reference to Michael Gibbons's list of the skills required by graduates in the 21st century: computer literacy, knowledge reconfiguration, information management, problem-solving in the context of application, team building, networking, negotiation / mediation, and social sensievity (Gibbons, 1998) (Cosser andDu Toit 2002: 129).
In comparing this list with a list from the South African Qualifications Authority (SAQA), the authors speak of 'the Gibbons and SAQA lists' ( 129).The authors give the reference as 'Gibbons, 1998', and  There is a final document.Writing in the journal For Engineering Educators, Zola Mbanguta refers to the list in the Plan, saying: The Plan adopts Gibbons' skills requirements (1998), which must underpin all graduate education in South Africa.The skills required are computer literacy, knowledge reconfiguration skills, information management, problem-solving in the context of applications, tearn building, networking, negotiation or mediation competencies and social sensitivity (Mbanguta 2003: 16).
The placing of 'South Africa' in that sentence leadsone to the next level of interpretation -that when Gibbons created his list, he was talking about South Africa.This might simply be an unfortunate juxtaposition of words, but it certainly supplies a new and unintended context.While Mbanguta does not place the list in quotation marks, in the same vein as other writers, he references the Gibbons text, leading the reader to believe this list is taken from the primary source.

5Discusion
At this stage, it might be useful to summarise the differences between the Saint/Gibbons list and the list in the Plan.
Becausethere is no such thing as 'the Saint/Gibbons list,' this is speculation at best, but would look something like the

contents of Table-I (see below).
A comparison of the Gibbons list as commonly quoted reveals it is, at best, based loosely on some of the terms found in Gibbons' document; at worst, it is a construction with adaptations, additions and omissions.The changesare far too large to be explained as typographical errors -they represent significant alterations to an author's writing.It is obvious that this list, as a comprehensive summary of skills required by 21st century graduates, has not been created by Michael Gibbons, and doesnot exist in his document, or in any of his writings.When first referring to Broere and Kruger's article, there was the possibility that the issuewas of minor importance, as it may have merely been a single error in a single conference paper.By examining other documents, however, it becomes clear that the error is widespread.There is also the probability that it will expand in future.
The significanceof the error (particularly regarding the influence of the documents in which it occurs) and the range of the error (the sheer number of articles in which it occurs) hasimplications that require some reflection.
Significance and range of the error Many of the publications cited here are influential, as they collectively represent views from the following sources: -Council on Higher Education -Centre for Higher Education Transformation -South African Ministry of Education -South African Human SciencesResearchCouncil Surely, it is the very weight and credibility that these institutions carry that leads the list to be unquestionably cited by other writers in the KAS publication, the CITTE conference paper and the Engineering Education journal article.There are additional examples where either the full list or the summarised list from the Plan's Executive Summary is quoted without reference to Michael Gibbons or William Saint (e.g.Waghid 2003: 93;DOE 2002) emphasising that the Plan's credibility stands by itself.We now have a strange situation of 'fact' verification through error repetition.

Credibility
The errors in these documents do not necessarily imply a lack of credibility of the institutions behind the documents.Similarly, this is not an attempt to discredit the authors or accusethem of bad faith.The errors would, however, lead the Beyond the articles Of course, these articles themselves become sources for further documentation, and they play their role in ensuring that the false 'Gibbons list' becomes firmly entrenched in the minds of educators across the country.
For example, there would be an unknown number of internal institutional policy documents and reports that would not be available easily to the public, but which would shape South Africa's Higher Education, all drawing on the Plan and corroborating documents.This, as Mbanguta notes, would be because the Plan 'adopts Gibbons' skills requirements (1998), which must underpin all graduate education in South Africa.'A rare, publicly available example is Hall (200 I), which quotes and discussesthe relevant section of the Plan and its significance for the University of Cape Town.One might now wish to question some of the arguments put forward in these policy documents if they use proposals in the Planand the false list astheir motivations for arguments.
Beyond the specifics of this case This particular case has concentrated on one error: the mis-quoting of Michael Gibbons' text.In many instances, this might have been an isolated incident.This case, however, is an example of an influential document's being quoted by other influential documents.Because the error is repeated, it becomes the truth, easily verifiable by referring to important documents.Indications are, as authors continue to refer to the secondary texts, the error will continue in the future.It is only when someone refers to the primary text that the error is revealed, but the sheer weight of the incorrect quotations will be difficult to counter.
The documents quoted here deal with education; although the extent to which the error will influence education is something to be researched further, it is obvious that there will be some impact.In other fields of study, an error such as this can just as easily occur, and can also influence policy and procedures.Until, of course, authors return to the primary source.
There are also issues of credibility, alluded to in this paper -quoting from a secondary text while referencing a primary text can damage an author's credibility, or at least encourage a sceptical reader to question the authority of the research.
Finally, there are the moral obligations -to the reader and to the original source.It can be argued that the misquotation is a violation of Michael Gibbons' moral rights under copyright, in the sense that it damages the integrity of his original text by attributing the false list to him.This is largely a matter of individual conscience, and is not for discussion here.

Conclusions
In 1998, Michael Gibbons published a World Bank text titled Higher Education Relevance in the 21 st Century.In that text, he and a guest author, William Saint, discussed skills that might be considered necessary in the 21st century.There was not, however, a single and comprehensive list of skills proposed.
Fours years later, a conference paper cited a list of skills created by Michael Gibbons in his 1998 text.While this list owes a great deal to Gibbons and Saint, it has additions that were not in their text, it has variations of their terminology, and it has also ignored several skills raised by Gibbons and Saint.It differs so substantially from the original work that Michael Gibbons has confirmed that it is not his.
By closely examining two documents, this paper has demonstrated that the false list had been constructed by others shortly after Gibbons' publication, and had been falsely attributed to him.The false list has been repeated in several influential documents, and the problem has been compounded by the fact that most of the documents claim to be citing or even quoting Gibbons directly, while it is obvious that they are drawing on secondary texts ..One of these documents is the National Plan for Higher Education in South Africa, and the false 'Gibbons list' is now formally part of South African education policy.The impact on the credibility of Michael Gibbons is unknown.
While it is probable that the importance of the primary and early secondary sources have affected the pace at which the error has been duplicated, there is nothing unique about the case that suggests it could not occur elsewhere.This case has demonstrated the need for writers to reference and cite accurately, and, wherever possible, to refer to the original documents.
' text is still frequently listed in references.Here is a selection that indicates the scope of the problem.(Although there are minor (possibly typographical) changes in the examples below, attention is drawn only to changes that are significant).SAJnl Libs & Info Sci 2005, 71 (3) Document 4: 'Competence, Character, and Compassion as Values in Education' (2001) In a Konrad Adenauer Stiftung (KAS) publication chapter 'Competence, Character, and Compassion as Values in Education,' Emmanuel Ngara refers to the Plan, and writes: place the Gibbons text in their list of references.Document 6: 'Towards a multimodal teaching, learning and assessment strategy after the NPHE' (2002) And so we return to Broere and Kruger's article.The section quoted above from their article follows a discussion of the Plan'sExecutive Summary,and is prefaced with the words 'These views [in the Executive Summary] are echoing those of .. .' (3).As noted above, however, the list is given as a quotation directly attributed to Gibbons.Unlike Ngara, the implication here is that the researchers are not quoting the Plan, but are quoting from Gibbons and have actually confirmed the existence of the summary in Gibbons' text.As mentioned, moreover, Broere and Kruger's list Gibbons' article in their references.Document 7: 'The Conceive-Design-Implement-OperateInitiative' (2003) reader to query other parts of the documents, particularly those documents that have the Gibbons text in their list of references.(This would include, of course, the National Plan itself.)For example, where Cosser & Du Toit refer to 'the Gibbons and SAQA lists' (129) a reader would now, perhaps unfairly, question the accuracy of the 'SAQA list' as mentioned in their article.One also wonders how a close reading of Gibbons' article might have influenced Cosser & Du Toit's argument.
required by the global village of the 21 st century?What types of skills and values

Table I
Differences between the Saint/Gibbons'lists' and the Current'Gibbons' list, by category