Multiple Choice Questions Five Methods

Types of Questions

2007-09-14T02:30:00.000-07:00

Sorry for the silence over the summer. But the work went a bit quiet then. What we have done recently is to add the WSDL end points for the SOAP part of our service and attemted to test in anger the service.

A number of points we found

We had to correct an oversight. Any attribute in xml cannot have a quotation mark in it (because the quotation mark is the delimiter of an attribute). This was causing problems with questions created with quotation marks. Therefore we are just bug fixing that.
Also, we can't have questions with hash symbols in them either. If they appear in generated qml, QuestionMark refuses to import them.
A third question is what variant of QML (by Questionmark) should we support. For now, we are working on supporting the importation of non graphical questions form QML in Perception 3 and Perception 4 - but only exporting to Perception 4's flavour of QML. This latter point has given us some problems with a question of this type:
---ORDER---
Associate the following numbers with their type:
1@Odd
2@Even
3@Odd
If we export this as a QML "SEL" question, there is no problem - the three numbers appear with dropdown boxes beside each one allowing the student to choose "odd" or "even". However, the SEL is depracated in Perception 4 where a "MATCH" style question is used instead. However, in that "odd" appears twice in the drop down list -and more problematically - in runtime mode - you aren't allowed to choose an item you have chosen previously. Making this kind of question very difficult. However, an easy work around is to switch the question to type "MAT" (or Matrix) - where it will again become possible to make multiple associations of the same term.

All of which shows that translating questions is much like translating languages. There are often 1-1 correspondences which are easy. But also there are occaisions where you need to arrive at the essence of what is being said and then recast it in the terms of the other language. The English "the straw that broke the camel's back" - in Italian becomes "La goccia che fa traboccare il vaso d'aqua" literally - the drop that made the vase of water topple over.

Clarifying the questions

2007-05-22T03:28:00.000-07:00

Sorting out QTEXT questions and their corresponding QTI and QML type of question has required alot of thought. Questions such as ORDER type questions can be represented in different ways and the challenge for us was deciding the best way to represent each for clarity. The following is therefore a clarification on how each QTEXT question is represented and how these correspond to QTI and QML question types.

QTEXT Representation of the five types of questions are shown below along with the question type they correspond to in QTI (R2Q2) and QML (QM perception 4) .

The first QTEXT question is a multiple choice question (MCQ) which has a single correct answer represented by the @ symbol. This corresponds to a "choice" question in QTI and a Multiple Choice question (QTYPE="MC") in QML.

-----------------MCQ---------------------
What is rightthree is right
This isn't
Nor is this
@This is
But this isn't
And neither is this

The second QTEXT question is a multiple response question (MRQ), which has multiple correct answers, again represented by the @ symbols. This question corresponds to a "choice_multiple" question in QTI and a Multiple Response question (QTYPE="MR") in QML.

-----------------MCQ---------------------
What is righttwo,three and four are right
This isn't
@This is
@This is too
@And so is this
But this isn't

The next QTEXT question in a Fill in the blank type question (CLOZE). This question corresponds to a "text_Entry" question for single blanks and a "gap_match" question for multple blanks in QTI. In QML this question corresponds to a Fill in Blanks type question(QTYPE="FIB").

-----------------CLOZE---------------------
This is a fill in the blank or text Entry question. The gaps are placed between [at] signs like @so@. You can have more than one blank.

The next QTEXT question is an order type of question. The answers are typed in in the correct order. This type of question corresponds to an "order" question in QTI and a Ranking question (QTYPE="RANK") in QML.

-----------------ORDER---------------------
Put the following in the correct order (order is implicit)They are only numbers

Uno

Due

Tre

Quattro

Cinque

The next type of QTEXT question is a match or association type of question, where the match is represented by the text following the @ symbol. This type of question corresponds to a "match" type question in QTI and a matching type question (QTYPE="MATCH") in QTI

----------------ORDER---------------------
Match the following numbers with their english meaningThey are only numbers

Uno @One
Due @Two
Tre @Three
Quattro @Four
Cinque @Five

MCQFM on sourceforge.net

2007-05-20T14:22:00.000-07:00

The MCQFM project has now been accepted on sourceforge.net, the Open Source software development web site. It can be found at http://sourceforge.net/projects/mcqfm and the current project code can be found by browsing the CVS repository.

Converting from QML to Text

2007-05-03T07:05:00.000-07:00

While so far we have been exploring translating from the low featured QTEXT to BQL and then finally to QTI or QML, now, when trying to convert the other way, we find we are doing more than just tree conversions, we have to make deductions about what a question is trying to say, and how, therefore, its data might be recast into another format.

Lets for example think of a very complex multiple response question:

---MCQ---
Which of these are valid responses to belonging to unappealing social groups?
A) Leave the group
B) Seek to change the group
C) Remain in the group but complain about it intensely
D) Do passive aggressive things like wearing slovenly clothes whenever the group meets
E) Try to change your attitude to see the good in each member of the group

We might use a question like this to see if someone is coherent in their thinking (regardless of which strategy they use). (A) on its own is valid if defeatest (E) on its own is valid but maybe a bit airy-fairy - (B) on its own is a bit overambitious - but (B) and (E) together seems like a wonderful strategy. (C) and (D) if done together are very negative, but still coherent. However if anyone choses either (C)or(D) AND any of (A)(B)or(E) - then they are obviously a basket case (which is probably overdoing it - since most of us are contradictory at various points in our lives). Based on that reasoning I have tried to produce some QML below with scoring to reflect the best strategy.

<QUESTION STATUS="Normal" DESCRIPTION="Which of these are valid responses to belonging to unappealing social groups?" TOPIC="a" ID="3123191976896635" TYPE="text/plain" >
<CONTENT TYPE="text/plain" >
Which of these are valid responses to belonging to unappealing social groups?</CONTENT>
<ANSWER QTYPE="MR" SHUFFLE="N">
<CHOICE ID="0">
<CONTENT TYPE="text/plain">
Leave the group</CONTENT>
</CHOICE>
<CHOICE ID="1">
<CONTENT TYPE="text/plain">
Seek to change the group</CONTENT>
</CHOICE>
<CHOICE ID="2">
<CONTENT TYPE="text/plain">
Remain in the group but complain about it intensely</CONTENT>
</CHOICE>
<CHOICE ID="3">
<CONTENT TYPE="text/plain">
Do passive aggressive things like wearing slovenly clothes whenever the group meets</CONTENT>
</CHOICE>
<CHOICE ID="4">
<CONTENT TYPE="text/plain">
Try to change your attitude to see the good in each member of the group</CONTENT>
</CHOICE>
</ANSWER>
<OUTCOME ID="simpleDefection" SCORE="1">
<CONDITION >
"0" AND NOT "1" AND NOT "2" AND NOT "3" AND NOT "4"</CONDITION>
<CONTENT TYPE="text/plain">
</CONTENT>
</OUTCOME>
<OUTCOME ID="changeAttitudeandGroup" SCORE="3">
<CONDITION >
NOT "0" AND "1" AND "4" AND NOT "2" AND NOT "3"</CONDITION>
<CONTENT TYPE="text/plain">
</CONTENT>
</OUTCOME>
<OUTCOME ID="justChangeAttitude" SCORE="2">
<CONDITION >
NOT "0" OR NOT "1" OR NOT "2" OR NOT "3" OR "4"</CONDITION>
<CONTENT TYPE="text/plain">
</CONTENT>
</OUTCOME>
<OUTCOME ID="ConsistentlySulk" SCORE="0">
<CONDITION >
NOT "0" AND NOT "1" AND "2" AND "3" AND NOT "4"</CONDITION>
<CONTENT TYPE="text/plain">
</CONTENT>
</OUTCOME>
<OUTCOME ID="contradictory" SCORE="-2">
<CONDITION >
"2" OR "3" AND "0" OR "1" OR "4"</CONDITION>
<CONTENT TYPE="text/plain">
</CONTENT>
</OUTCOME>
</QUESTION>

Now from reading this, the only thing it can really end up as in QTEXT terms would be
---MCQ---
Which of these are valid responses to belonging to unappealing social groups?
Leave the group
@Seek to change the group
Remain in the group but complain about it intensely
Do passive aggressive things like excessive slovenliness of clothes whenever the group meets
@Try to change your attitude to see the good in each member of the group

- which is to say, set the "true" answers to be that of the **choices implied by the highest-scoring outcome**.

In this example we see a massive loss of the original data when "compressed" into the new format. But this is the curse of readability

BQL - Basic Question Language

2007-04-23T13:01:00.000-07:00

BQL is an xml language we have created which represents in XML form the question in as basic a form as possible (it stands for Basic Question Language).

BQL example - Multiple Choice, Order and Cloze Questions

2007-04-23T12:49:00.000-07:00

<?xml version="1.0" encoding="UTF-8"?>



<ns:quiz xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ns="http://mcqfm/quiz/1_0"
xsi:schemaLocation="http://mcqfm/quiz/1_0 /C:/Documents%20and%20Settings/Administrator/TextFileReadApp/src/newXMLSchema.xsd">

<ns:item model="mcq" id="1">
<ns:choicequestion totalscorevalue="1" minvalue="1">
<ns:content>Is this computer turned on?</ns:content>
<ns:answer correct="false" scorevalue="0" id="1">Don't know</ns:answer>
<ns:answer correct="true" scorevalue="1" id="2">yes</ns:answer>
<ns:answer correct="false" scorevalue="0" id="3">No</ns:answer>
</ns:choicequestion>
</ns:item>

<ns:item model="order" id="2">
<ns:orderquestion totalscorevalue="1" minvalue="1">
<ns:content>What is the correct order of the following notes:</ns:content>
<ns:orderanswer correctorder="3" id="1" correct="true" scorevalue="1">me</ns:orderanswer>
<ns:orderanswer correctorder="1" id="2" correct="true" scorevalue="1">doh</ns:orderanswer>
<ns:orderanswer correctorder="2" id="3" correct="true" scorevalue="1">ray</ns:orderanswer>
<ns:feedback>The correct order is: doh, ray me</ns:feedback>
</ns:orderquestion>
</ns:item>

<ns:item model="cloze" id="3">
<ns:gapquestion totalscorevalue="1" minvalue="1">
<ns:content>Fill in the missing gap</ns:content>
<ns:sentencepart gap="false" id="1">A light wave is </ns:sentencepart>
<ns:sentencepart gap="true" id="2">electromagnetic</ns:sentencepart>
<ns:sentencepart gap="false" id="3">whereas a soundwave is </ns:sentencepart>
<ns:feedback>The correct answer is electromechanical</ns:feedback>
</ns:gapquestion>
</ns:item>
</ns:quiz>

BQL Schema

2007-04-23T12:40:00.000-07:00

<?xml version="1.0" encoding="UTF-8"?>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:ns="http://mcqfm/quiz/1_0"
targetNamespace="http://mcqfm/quiz/1_0"
xmlns="http://mcqfm/quiz/1_0" elementFormDefault="qualified">

<xs:element name="quiz" >
<xs:complexType>
<xs:sequence>
<xs:element name="item" type="item" maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
</xs:element>

<xs:complexType name="item">
<xs:choice>
<xs:element name="choicequestion" type="question"/>
<xs:element name="orderquestion" type="question"/>
<xs:element name="gapquestion" type="question"/>
</xs:choice>
<xs:attribute name="model" type="xs:string" use="required" />
<xs:attribute name="id" type="xs:integer" use="required" />
</xs:complexType>

<xs:complexType name="question">
<xs:sequence>
<xs:element name="content" type="content"/>
<xs:choice>
<xs:element name="answer" type="answer" maxOccurs="unbounded"/>
<xs:element name="orderanswer" type="orderanswer" maxOccurs="unbounded"/>
<xs:element name="sentencepart" type="sentencepart" maxOccurs="unbounded"/>
</xs:choice>
<xs:element name="feedback" type="feedback" minOccurs="0"/>
</xs:sequence>
<xs:attribute name="minvalue" type="xs:integer" use="optional" />
<xs:attribute name="totalscorevalue" type="xs:integer" use="required" />
</xs:complexType>

<xs:complexType name="sentencepart">
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="gap" type="xs:boolean" use="required" />
<xs:attribute name="id" type="xs:integer" use="required" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>

<xs:complexType name="answer">
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="correct" type="xs:boolean" use="required" />
<xs:attribute name="id" type="xs:integer" use="required" />
<xs:attribute name="scorevalue" type="xs:integer" use="optional" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>

<xs:complexType name="orderanswer">
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="correctorder" type="xs:integer" use="required" />
<xs:attribute name="scorevalue" type="xs:integer" use="optional" />
<xs:attribute name="correct" type="xs:boolean" use="required" />
<xs:attribute name="id" type="xs:integer" use="required" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>

<xs:simpleType name="content">
<xs:restriction base="xs:string">
<xs:whiteSpace value="collapse"/>
</xs:restriction>
</xs:simpleType>

<xs:simpleType name="feedback">
<xs:restriction base="xs:string">
<xs:whiteSpace value="collapse"/>
</xs:restriction>
</xs:simpleType>

</xs:schema>

Reflections so Far

2007-04-23T02:14:00.000-07:00

1.R2Q2 was tricky to set up, however they've updated their Installation Tool (UPDATED 13/3/07) so this may be simpler now. But once it was working and I figured out how to use it (they also seem to have added help since), it was fairly simple. However it is only possible to load one question at a time and questions need to be in a zip file, thus making its use limited and adding questions one at a time a bit cumbursome, this would also have implications later.

2.In QM perception 4 it is possible to load a number of questions together as a quiz. However QM Perception is more complex than R2Q2 which allows it to have more functionality but also makes it harder to figure out (not being a great one for reading unstructions).

3.The knock on effect to development of our mcqfm project was a requirement for single questions in QTI for R2Q2. We considered outputting QTI as single questions but this would mean restricting the user at the interface to one question at a time and making the development of QTI questions one at a time rather more tiresome. In the end we have opted to add a surplusWrapperNode as a root Node to encapsulate the QTI questions as a quiz. The surplusWrapperNode being a temporary expedient at the proof of concept stage, with the idea that in the future a further operation of removing this surplusWrapperNode and zipping up will be required before the questions can be usefully used by r2q2.
Probably what we might do by the end of the project is output the lot as an in qti 2.1 terms, then the new tools which are being developed could display that.

4.Development of the QML for QM Perception was relatively straightforward, as I've said already they allow for a quiz thus making it possible to generate a text stream which is immediately importable into perception.

5.In the BQL (the Basic Question Language - we use as an intermediate notation between the QTEXT humanly readable notation and the full XML -QML/QTI- representation) we wanted to generate XML that was expressive but simple, an extension of the humanly readable QText. It could be expressive in its use of elements, e.g , and attributes e.g scorevalue, correctorder etc. while still retaining readability.An example question for the order model is as follows:

<ns:item model="order" id="2">

<ns:orderquestion totalscorevalue="1" minvalue="1">

<ns:content>What is the correct order of the following notes:</ns:content>

<ns:orderanswer correctorder="3" id="1" correct="true" scorevalue="1">me</ns:orderanswer>

<ns:orderanswer correctorder="1" id="2" correct="true" scorevalue="1">doh</ns:orderanswer>

<ns:orderanswer correctorder="2" id="3" correct="true" scorevalue="1">

ray</ns:orderanswer>

<ns:feedback>The correct order is: doh, ray me</ns:feedback>

</ns:orderquestion>

</ns:item>

It might be nice to add scoring for questions, adding differential scoring for different responses in mcq or cloze questions is certainly a possibility - but would that complicate the QText to such an extent that its fundamental goal - to allow *anyone* to produce questions in QTI would become compromised (since it would require more recondite knowledge and therefore become only usable by people who know the spec).

6.Further questions on R2Q2 that might be represented by QText are inline choice, associate, match, gapmatch and hottext and mathematical template questions.

by Caroline

Sequence Diagram of MCQFM

2007-04-20T07:43:00.000-07:00

Click on the image to the left to see a sequence diagram of the workings of MCQFM so far

Use case:

2007-04-20T06:35:00.000-07:00

To make additions to the programme such as exporting to another form of XML would simply require the addition of a new xslt file. It would not be necessary to change either the QText or the BQL.Obviously a separate tool bar button would be required on the jsppage to pass the OutputType to the servlet. The servlet (mcqfmservlet.java) would require an additional else if statement (similar to the ones under //output for QML and QTI) to pass the outputType to the questionList GetOutput method.Similarly the GetOutput method in the questionList class would require another else if statement to determine the relevant xsl file name (sheetname).The relevant .xsl file does the transformation from BQL to the required form of XML, therefore no other changes would be required at this stage.

Programme Description

2007-04-20T06:29:00.000-07:00

The user interacts with a tool bar button to create questions in simple text called QText. The user then interacts with a tool bar button to select the type of quiz required (QML/QTI at the moment) and to signal the request to the system. In response the browser passes the popultaed FORM to the servlet. The servlet extracts the type of question output required and the QText string.

The QText gets split up into individual questions until we reach the end of the quiz. The type of each question (order, multiple choice etc) is determined and used to decide which type of object to create. The question string is passed to the constructor of the class and added at the end of a queue object derived from a linked list.Once all the questions have been added to the list the servlet calls the questionList GetQuizXML method.

This creates an instance of an XML document and adds the root node and namespace declarations. The GetXML method of each object in turn is called which converts the string passed in to XML elements. Each element gets appended to the document in turn to create the complete BQL quiz XML. BQL is an xml language we have created which represents in XML form the question in as basic a form as possible (it stands for Basic Question Language).

Once the BQL document is complete the servlet calls the questionList GetOutput method, passing in the OutputType (QML/QTI) as a parameter. The xsl file name is determined and the BQL document is transformed into QML or QTI using the relevant xsl file. The result is passed back and the document returned to the servlet which is then returned in the HTTP response.

Totally Alpha Version Available

2007-04-16T09:17:00.000-07:00

The first outputs of mcqfm are the earliest form of the demonstrator. This is at:

http://smirkboard.herts.ac.uk:8080/mcqfm/bqljsppage.jsp

Please try it out and see if the xml generated is to your liking.

To install it on your own tomcat, download the following file

http://smirkboard.herts.ac.uk/mcqfm/mcqfm.zip

Then unzip it to the webapps directory of your tomcat installation. Then just run the run_me_then_delete_me.bat file (a small hack to get round the fact we've hard coded the locations of some of the files into the java classes - this will create the necessary directories and copy the xsl files into them).

The source code (if you want to see it) is at

http://smirkboard.herts.ac.uk/mcqfm/mcqfm_with_source.zip

Origins

2007-02-23T03:59:00.000-08:00

The origins are in the TML and Euromet projects of the late 90s.

The javascript bases for the html generated is from work by Helene Missou on the TML (JTAP) project. You can't get to her page any more, but you can get it from Web Archive: see

•http://web.archive.org/web/19980611031649/http://w ww.ilrt.bris.ac.uk/~ethm/jscripts/js-intro.htm

And superb documentation for it can be found here:

http://web.archive.org/web/19980611031649/http://www.ilrt.bris.ac.uk/~ethm/download/jscript.zip

The macro idea comes from the Euromet project (of which I was a developer) - see

•http://page.mi.fu-berlin.de/~lernmat/oldbin/toolkit/fdt-full.html

First Entry

2007-02-23T03:56:00.000-08:00

The MCQFM project exists to make it easy to generate loads of QTI questions.

It is based on a currently existing page which generates HTML and QML questions quickly. This is at:
http://homepages.feis.herts.ac.uk/~comqsjb/quiz.htm
Try it out!

However, this is a perl script which uses macro preprocessing to generate the html and qml. How retro, how web1.0 I hear you say!

Therefore MCQFM will do a number of web2.0 things with this idea.

Change it from a web form into a web service
Generate the assessment items using XSLT instead of macro-preprocessing