Quick guide to R/exams
R/exams
is an R package that generates a reproducible workflow for designing, producing and marking exams.
Here, I provide a short walkthough for generating an online test for Blackboard - see the R/exams website for some tutorials.
Installation
Install R/exams via CRAN with install.packages("exams")
.
Running the package for the first time
exams
uses rtools
to create zip files. Make sure the proper rtools
is installed. An error message will appear if rtools
does not have permission to create zip files.
To give permission for creating .zip for windows:
Control Panel > System and Security > System > Advanced System Settings > Environmental Variables > add “C:/RTools/bin”
You may also need to install dependent packages like tth
for math notation.
Types of questions
- Single correct answer MCQ (schoice)
- Multiple correct answer MCQ (mchoice)
- Numeric answer (num)
- String (string)
Cloze is another option but is not supported by blackboard, this permits a combination of the above for the MCQ answer list
Building an exam question
There are several file types that are supported when writing your question but I have stuck with ‘.Rmd’. There are four parts of a question file:
- The question
- The answer list
- The solution list for giving feedback
- The meta-information
Each of these sections are defined by a header tag marked by =====. # does not work as a tag.
Question
The tag for defining the question is:
Question
========
The various random number generators in R will be your friend.
sample() # pick n random number(s) from a vector of discrete numbers
runif() # generate a vector of continuous numbers, can set min and max
rnorm() # generate a vector of continuous numbers from a normal distribution with a defined mean and standard deviation
Things to note
- Code chunk options are in effect. By default figures will have captions, turn it of with
fig.caption = ""
. Other useful figure options are figure sizes. Figure options can also be defined when compiling the exam - You can show code with
echo = TRUE
, turn off R formatting withresults = "asis"
& hide results withresults = "hide"
- LaTeX formatting is fine in markdown and outside code chunks
- To allow for randomised questions within a question file I had to escape the R code chunk to render the output as html, else the randomised question would be rendered as R output
- You can generate a file to go with the question using any R write to file function. Include the file in the question as normal for Rmarkdown - [filename](filelink). Leave this as default or
exams
will not be able to find the file
The answer list
The answer subheading is defined by the tag:
Answer
========
Bulleted markdown after this tag will be considered the options for an MCQ answer list.
Things to note
exams
contains several helper functions to make it easy to generate lists of answers.
answerlist
accepts a vector of answers. It also generates the answer subheading so there is no need to type it in.- Recommend using html in case LaTeX does not render properly, particularly when called as a string in an R code chunk.
- Do not randomise the answer list here, use the metadata, else an incorrect answer will be assigned to be the correct one.
comment = NA
in the code chunk options will remove the # from the R output.
- There are also helper functions for various things.
mchoice2string()
turns the solutions vector above into binary responses for the meta-information section.num_to_choice
generates a MCQ list of numbers for a numeric answer.
Solution
You can provide feedback via the solution header, including which answers are correct
Solution
========
Your solution here or correct answer: code for answer
(or answer[]
)
Answerlist
-———
* True
* False
* etc.
Important There should be no spaces after the header tag title, i.e. markdown formatting
Question metainformation
This is an important section of the question because it defines the correct answer. Metainformation is defined by the tag:
Meta-information
================
Useful variables are:
- exname = title of question, becomes name of the pool in blackboard
- extype = type of question (num/schoice/mchoice)
- exsolution: order of correct answers in binary (e.g. 01010) for MCQ or R code for numeric output - e.g.
- extol = the tolerance range for numeric questions
- exshuffle = Whether to shuffle the answers or not. This can be used to randomly select a subset of answers from an answer list. Provide a number of answers for non-numeric questions (e.g. 4 for 4 answers).
TRUE/FALSE
is also accepted.
An example question
This example displays the correct answer and 3 randomly chosen option out of 6 possible answers.
Question
========
What is your name?
\```{r question, results = "hide", echo=F}
# list of possible answer as a character vector
knights_of_camelot <- c("Arthur, King of the Britons",
"Sir Lancelot the Brave",
"Sir Robin the Not-Quite-So-Brave-as-Sir-Lancelot",
"Sir Galahad the Pure",
"Sir Bedevere the Wise",
"Patsy")
# solution to the vector above
solutions <- c(FALSE, TRUE, FALSE, FALSE, FALSE, FALSE)
# explanations (feedback for students)
explanations <- c("I didn't vote for him",
"His favourite colour is blue",
"He doesn't know the capital of Assyria",
"His favourite colour is blue. No, yel...",
"Knows nothing about swallows",
"Clip Clop")
\```
\```{r answerlist, echo=F, results = "asis"}
# helper function to format the list of possible answers
answerlist(knights_of_camelot, markup = "markdown")
\```
Solution
========
\```{r solutionlist, echo = FALSE, results = "asis"}
answerlist(ifelse(solultions, "True", "False"), explanations, markup = "markdown")
\```
Meta-information
================
exname: Bridgekeeper
extype: schoice
exsolution: `\r mchoice2string(solutions)`
exshuffle: 4
This renders like this:
Exam 1
Question
What is your name?
Sir Bedevere the Wise
Arthur, King of the Britons
Sir Galahad the Pure
Sir Lancelot the Brave
Solution
False. Knows nothing about swallows
False. I didn't vote for him
False. His favourite colour is blue. No, yel.
True. His favourite colour is blue
You could also skip the solutions vector and include it in the explanations vector like c("False. I didn't vote for him")
. Of course, the solution does not appear immediately in blackboard but make sure the option for solutions and feedback to appear is checked.
Compiling the exam
I have written a script to compile the exam.
The compiling function (exams2blackboard
) requires a list of file names to generate the exam. Each file represents a question. Versions of a question (n
) generate a pool of questions. The list should not contain subdirectories or files not to be included in the exam. exams2html
is a means of checking a file/list of exam questions renders properly in html (or exams2pdf
).
There are several means of customising the metadata of the exam. Here I have:
- turned off partial marks - is TRUE by default
- used custom directories to search for the questions and save the output zip.
- set 10 copies for each question using the variable
n
- set the name of the zip file using the variable
name
- set the number of points for each question to 1, default = 10
library("exams")
options(device.ask.default = FALSE)
## content and output directory
mydir <- "C:/Users/kongj/OneDrive - TCDUD.onmicrosoft.com/Teaching/Biostats/Midsem MCQ"
## define exam questions (each item in list is a pool)
myexam <- list.files(paste0(mydir,"/questions"), pattern = ".Rmd")
# render single question
#exams2html(list("question3.Rmd"), edir = paste0(mydir,"/questions"))
## generate .zip with Blackboard exam with n replicates
exams2blackboard(file = myexam, n = 10, name = 'Jacinta', dir = mydir,
edir = paste0(mydir,"/questions"),
eval = list(partial = FALSE, negative = FALSE),
points = 1
)
Things to note
- You can create a mix of questions in the exam by defining it in the list of questions but it is not recommended for generating pools of questions
- If the exam is complied with no partial marks, then the blackboard exam will have no partial marks even if the option is checked within blackboard
- You can generate a pool of questions by compiling a single question and uploading that zip file to Blackboard under “Import Pool”. This is useful for creating random block tests in Blackboard if the compiled zip file is not suitable as an test right away