Knowledge Acquisition during Collaborative Writing

Présentation au sujet: "Knowledge Acquisition during Collaborative Writing"— Transcription de la présentation:

1 Knowledge Acquisition during Collaborative Writing
Denis ALAMARGOT Laboratory LaCo-CNRS - University of Poitiers - France Jerry ANDRIESSEN Department of Educational Sciences - University of Utrecht - The Netherlands David GALBRAITH Department of Psychology - University of Staffordshire - England Knowledge Acquisition during Collaborative Writing of Argumentative Text Effect of individual differences in the personality of writers Program of research granted by the CNRS - France CATTW / ACPRTS 2003 Dalhousie University Halifax, Canada

2 Theoretical framework

3 Problematic Under the right circumstances, text production activity can lead to a modification and/or an improvement of the domain knowledge of the individual writer. This effect of writing is called « epistemic effect » (Eigler, Jechle, Merziger, & Winter, 1991; Scardamalia & Bereiter, 1991). Research question: can we obtain the epistemic effect in collaborative writing?

4 Problematic The creation or the modification of new ideas during text writing activity can be due to two different strategies of composition: Knowledge Transforming Strategy Bereiter & Scardamalia (1991) Dialectic between content space and pragmatic/rhetorical space, Retrieval of ideas guided by pragmatic goals, Sensibility to the audience needed Knowledge Constituting Strategy Galbraith (1999) Dialectic between content space and externalised text Development of understanding by articulating implicit ideas in language Pragmatic/rhetorical goals inhibit the phenomenon Less sensibility to the audience needed

5 Problematic According to Galbraith (1996), two interactive factors can modulate the epistemic effect, via the two strategies of composition: The type of content planning required : - organized composition (outline) facilitates Knowledge Transforming (reorganisation of existing ideas) - free composition (notes) facilitates Knowledge Constituting (articualtion of new ideas) The social personality of writers : Snyder (1986)’ personality test. - more sensibility to social audience (high self monitoring) facilitates Knowledge Transforming - less sensibilty to social audience (low self monitoring) facilitates Knowledge Constituting

6 Galbraith 1996, 1999 Low self-monitors produce new ideas when writing full text and these are associated with increases in subjective ratings of knowledge. Interpreted as development of understanding. High self-monitors change their ideas when planning in note-form, but these are not associated with increases in subjective ratings of knowledge. Interpreted as retrieval of wider range of ideas relevant to rhetorical goals, but not with development of understanding.

7 Problematic Outline Organization required O+ Note
Organization not required O- High Self Monitor (H) Identification of rhetorically appropriate ideas Knowledge Transforming Inappropriate situation Low self Monitor (L) Development of understanding Knowledge Constituting

8 Problematic According to the propositions of Alamargot & Andriessen (2002), the results obtained by Galbraith (1996, 1999) can be extended to the collaborative writing situation. The question is then to evaluate the epistemic effect induced by the interaction between the ‘Text content organization’ and the ‘Social personality of collaborators’. Studying the nature of interaction between pairs of collaborators, characterized by different social personalities, should give information about the process underlying epistemic effect.

9 Problematic Outline Organization required O+ Note
Organization not required O- High Self Monitor Pairs (HH) Interaction during collaborative session based on social regulation and search for relevant arguments Inappropriate situation for epistemic effect Low self Monitor (LL) Interaction during collaborative session based on developing understanding

10 Experimentation

11 of the experimental sample
Participants 40 participants All students at the University of Poitiers and in a high school of business (second years of studies) Snyder’s scoring of the experimental sample Snyder’s test Answering « true » or « false » to 25 questions : “I find it hard to imitate the behaviour of other people” “My behaviour is usually an expression of my true inner feelings, attitudes and beliefs.” Mean = 10.59 Median = 10 Std = 4.13 Low self monitor : score from 0 to 10 High self monitor : score from 11 to 25

12 Experimental design

13 Experimental task 1 : to read documentation on GMO
The documentary sources exposes different point of view about GMO (7 pages of texts) Task materials have been developed by Matthieu Quignard for the SCALE project.

14 Experimental task 2 : famine recherche environnement bactérie
To evaluate the semantical relationships between the most frequent words in the sources. Adapting from the pathfinder task (Britton, 1996) : participants were asked to indicate relationships between 16 words by using a 7 points scale : Strong opposition ………….No relation ……………. Strong similarity famine recherche environnement bactérie biotechnologie commerce espèce nourriture danger engrais sécheresse tradition herbicide anticorps eau pesticides

15 Experimental task 3 : To collaborate to compose an argumentative text presenting the positive and negative aspects of GMO. Texts were composed from memory, without any source Participants - composing a pair - communicated together by sending small sheet of papers (of different colors), replicating an er functionning. Only one sheet of paper was provided to compose the text. Participants exchanged the text by sending it as an attached document. They were free to manage their interaction and their work

16 Experimental Procedure
1 - Reading of sources : GMO (30 to 40 mn) 2 – Similarity ratings : Britton (1996)  (10 mn) 3 - Text composition (30 to 40 mn) 4 - Similarity ratings : Britton (1996)  (10 mn)

17 Measures and Analysis 3 kinds of measures and analysis
Characteristics of produced texts during collaboration : Production’s indicators Nature of interaction during collaboration : Classification with the Rainbow method Change in knowledge after collaboration : Similarity ratings and constraint satisfaction analysis

18 Results part 1: Produced texts Production analysis

19 Exemple of a produced text : LL/O-
Les OGM sont selon la définition des organismes génétiquement modifiés. Grâce aux recherches, les chercheurs ont établi de nouvelles espèces, par exemple du maïs, du soja ou encore du coton, capables de résister à certains pesticides. Dans la composition de ces plantes par exemple, des gènes ont été modulés pour devenir plus résistants. Les organismes qui fixent les normes pour pouvoir vendre des produits avec des OGN ont limité la mention obligatoire "OGN" aux produits contenant plus de 1% d'OGN. Les autres ne sont pas dans l'obligation de noter ce produite dans les ingrédients. Les principaux exploitants d'OGM sont les américains ainsi que les Espagnols. Des sondages ont été effectués afin de se rendre compte de l'impact qu'ont les OGM sur les populations européennes. Il s'avère que les Français, les Allemands, les Anglais, ainsi que les Portugais, ne sont pas vraiment favorables au projet. Certains spécialistes envisagent de lutter contre la faim dans le monde ou les problèmes de l'écosystème grâce aux OGM, ce qu'il y a c'est que les avis divergent. Pour les pays tropicaux, où beaucoup de bactéries posent des problèmes quant aux plantes, certains chercheurs souhaiteraient créer des plantes résistantes aux climats difficiles ainsi qu'aux maladies. Mais certaines autres personnes pensent que ces problèmes là pourraient être réglés d'une façon plus économique, comme faire participer les pays les plus riches dans la progression de ces pays en voie de développement.

20 Number of ideas in the texts
HH 23.25 20.80 LL 20.60 26.57

21 Nature of interactions
Results part 2: Nature of interactions Rainbow analysis

22 example : a CHAT interaction
* example : a CHAT interaction Example is from data collected by Lyon University for the SCALE project

23 Rainbow method Activities Task-focussed activity 1. Off-task
(i.e. discussion on topic X) 1. Off-task Non-Task-focussed activity (I.e. all interaction that is not discussion on topic) 4. Task management : coordination, themes, roles, rules… 2. Social relation : greetings… 5. Attitudes, opinions and agreement : opening, clarifying, closing 3. Interaction management : presence, perception, time, turn (coordination), language understanding 6. Provide (counter-) arguments Rainbow functional category Brief definition 1. Outside activity Any interaction that is not concerned with interacting in order to carry out the researcher-defined task, including socio-relational interaction that does not relate to interacting in order to achieve the task. 2. Social relation Interaction that is concerned with managing the students’ social relations with respect to the task (debating about X), e.g. greeting, leave-taking, politeness, expressions of frustration with the way the partner is interacting, etc. 3. Interaction management Interaction concerned with managing the interaction itself: who will speak or not and when (coordination), establishing contact, perception, understanding, attitudes (communication management), topic shifting, time management, … 4. Task management Management of the progression of the task itself: planning what is to be discussed, establishing whether problem solved or not, … 5. Opinions Interaction concerned with expressing, making explicit, opinions (beliefs, acceptances, …) with respect to the topic debated; expression of opinions at opening and closing of phases of argumentative discussion. 6. Argumentation Expression of (counter-)arguments (e.g. GMOs increase pollution) directly related to a thesis (e.g. GMOs should be stopped). 7. Explore and deepen Interaction concerned with (counter-)arguments linked to (counter-)arguments, argumentative relations, and meaning of arguments themselves (elaboration of them, definition, extension, contraction, i.e. any discursive or conceptual operation performed on content of arguments themselves). 7. Explore / deepen question : explanation, grounding, clarification, mutual understanding, … © SCALE happy friends family: Michael Baker, Jerry Andriessen, Matthieu Quignard, Marije van Amelsvoort, Kristine Lund, Timo Salminen, Lia Litosseliti, Lisette Munneke General phases of the task-focussed activity [Task = Debate on topic of thesis X]

24 Explore and Deepen 7a, ‘Talk about exploring and deepening arguments during discussion’ involves interaction in the context of debate itself that, by elimination is neither expression of opinion nor bringing into discussion (counter-)arguments that apply directly to theses. In this category it is about exploring and deepening during the discussion part of the task. There are three ways in which students can deepen and explore (see also figure 2): §         7.1 Deepen an argument involves any expression of a (counter-)argument that ‘builds’ on some(counter)argument that has already been expressed. This is ‘going deeper’ in the argument tree. Most probably, this will involve counter-argumentation. An example is shown in figure 3. §         7.2 Linking an argument involves any discussion with respect to an argumentative link, such as questioning or supporting that link. In terms of a Toulminian model, this could correspond to backing or warrants of the transition from data to conclusions. §         7.3 Meaning of an argument involves discussion of the meaning of an argument, or rather, of a key notion that it is built on, or more generally, discursive operations such as reformulation, conceptual dissociation and association, elaboration of a given argument (Baker, 1999). The difference between 7.1 and 7.3 is that the first is about building on an argument with another argument and the second about building on an argument with examples or specifications.

25 Rainbow categories in the experiment:
Awareness of interaction (3): Social regulation? Awareness of task strategies (4) Task regulation? Attitudes, opinions and agreement (5) New arguments or counterarguments (6): new arguments…knowledge telling or transformation Broadening and Deepening (7): elaborating on existing arguments…knowledge constitution

26 1 : off task 2 : social relation 3 : interaction management 4 : task management 5 : attitude, opinions, agreement 6 : (counter) arguments 7 : explore/deepen

27 N°3 : Interaction management : Awareness of interaction X2 (n=220) = 1
N°3 : Interaction management : Awareness of interaction X2 (n=220) = 1.58 ; p= .20 NS McNemar : (A/D) = ; p=0.001 O+ O- HH 43 51 LL 47 79

28 N°4 : Task management : Awareness of task strategy X2 (n=261) = 4
N°4 : Task management : Awareness of task strategy X2 (n=261) = 4.83 ; p= .03 O+ O- HH 51 73 LL 75 62

29 N°6 : (counter) Arguments :
Creation of arguments X2 (n=100) = 3.08 ; p= .08 (marginal) O+ O- HH 36 25 LL 16 23

30 N°7 : Explore and deepen : Modification of arguments X2 (n=153) =
N°7 : Explore and deepen : Modification of arguments X2 (n=153) = .12 ; p= .72 NS McNemar : (A/D)= 8.01 ; p =.004 ; (B/C)=10.45; p=.001 O+ O- HH 26 23 LL 52

31 N°4,5,6 & 7 : Task focus activities Task Content X2 (n=827) = 17
N°4,5,6 & 7 : Task focus activities Task Content X2 (n=827) = 17.29; p< McNemar : (A/D)= ; p <.0001 O+ O- HH 103 228 LL 226 270

32 Results Rainbow method
Conclusion 1: LL pairs engage in significantly more interaction management in O- Conclusion 2: LL pairs engage in significantly more broadening and deepening of argumentation Conclusion 3: HH pairs generate significantly more (counter) arguments than LL in O+ (outline) condition Conclusion 4 : HH and LL pairs engage in significantly more task management when they work in non- appropriate condition Conclusion 5: HH pairs engage in significantly less task focussed activity than LL pairs, in their appropriate conditions

33 Constraint satisfaction analysis
Results part 3: Change in knowledge Constraint satisfaction analysis

34 Change in knowledge : Constraint satisfaction analysis
New conceptual relationships: Pairs of concepts before interaction which were unrelated (zero rating) but which were related after interaction – new propositions. Coherence of conceptual relationships: Ratings entered into a constraint satisfaction network (the C-I program developed by Kintsch, 1988) Network allowed to settle; coherence measured by number of cycles taken to maximize harmony (Britton, 1996) Score calculated as reduction in number of cycles required after writing: high score = increase in coherence of conceptual relationships.

35 New conceptual relationships
New conceptual relationships are pairs of concepts which had a rating of 0 before collaboration but which had a rating after collaboration. There are two main effects: group type (p=0.26) and organisation (p=0.13), neither are significant, but they combine to make LL O- significantly greater than LL O+ (p=0.03) and close to significantly greater than HH O- (p=.10)

36 Change in coherence of conceptual relationships
This is based on the cycles to settle measure. It is the number of cycles taken to settle before collaboration divided by the number of cycles to settle afterwards. Less cycles means more coherence, so a score of 1 means no change; a score > 1 means an increase in coherence; a score < 1 means less coherence. This is a main effect of organisation : O- > O+ (F (1, 17)+ 6.03, p=.02). For O-, the increase in coherence is significant (p=0.04). For O+, there is a slight decrease (i.e. score < 1) but this is not significant.

37 Correlations between knowledge change and features of collaboration
New ideas Increase in coherence 3. Interaction management .51 (p = .02) 5. Attitudes and opinions .39 (p = .08) .36 (p = .11) 6. Arguments -.56 (p = .01) The negative correlation between arguments and new ideas suggests that the HH O+ discourse, and high production of arguments, is like knowledge telling – it just involves suggesting topics, but does not involve conceptual development. Are attitudes and opinions equivalent to articulating opinion, expressing ideas – knowledge constituting?

38 Conclusions for knowledge change
O- (both LL and HH) develop a more coherent understanding But only LL O- create more conceptual relations than other groups – they change their understanding HH O+ do not change on either measure, despite the wider range of arguments they produce during collaboration. And development of new understanding is negatively correlated with the production of a wide range of arguments.

39 Conclusion, perspectives
Collaborative writing has differential effects on learning depending on the individuals involved and nature of task they are required to do. Producing new arguments in the dialogue is negatively related to increase in understanding Interaction management and elaboration are positively related to increase in understanding The O- scores much better than the O+ assignment in this respect

40 Conclusion, perspectives
Knowledge change in collaborative writing showed similar results as for individual writing Low self-monitors, producing unorganised text developed new, more coherent, understanding. This is associated with a high quantity of interaction management and task focussed activity. High self-monitors, producing organised text, discuss a wider range of ideas but these are not associated with developments of understanding.

41 Conclusion, perspectives
Assuming differential effects of planning processes and text production process helps us analyse these differential effects of learning and can inform better design of collaborative writing environments Exemple of LH Interaction and Knowledge Change R= -.98, p=.004

42 Conclusion, perspectives
These results have to be replicated and confirmed by genuine electronic communication (VCRI, DREW), and larger numbers of subjects