Quick guide to R/exams

R/exams is an R package that generates a reproducible workflow for designing, producing and marking exams.
Here, I provide a short walkthough for generating an online test for Blackboard - see the R/exams website for some tutorials.

Installation

Install R/exams via CRAN with install.packages("exams").

Running the package for the first time

exams uses rtools to create zip files. Make sure the proper rtools is installed. An error message will appear if rtools does not have permission to create zip files.
To give permission for creating .zip for windows: Control Panel > System and Security > System > Advanced System Settings > Environmental Variables > add “C:/RTools/bin”
You may also need to install dependent packages like tth for math notation.


Types of questions

  • Single correct answer MCQ (schoice)
  • Multiple correct answer MCQ (mchoice)
  • Numeric answer (num)
  • String (string)

Cloze is another option but is not supported by blackboard, this permits a combination of the above for the MCQ answer list


Building an exam question

There are several file types that are supported when writing your question but I have stuck with ‘.Rmd’. There are four parts of a question file:

  1. The question
  2. The answer list
  3. The solution list for giving feedback
  4. The meta-information

Each of these sections are defined by a header tag marked by =====. # does not work as a tag.

Question

The tag for defining the question is:

Question
========

The various random number generators in R will be your friend.

sample() # pick n random number(s) from a vector of discrete numbers
runif() # generate a vector of continuous numbers, can set min and max
rnorm() # generate a vector of continuous numbers from a normal distribution with a defined mean and standard deviation

Things to note

  • Code chunk options are in effect. By default figures will have captions, turn it of with fig.caption = "". Other useful figure options are figure sizes. Figure options can also be defined when compiling the exam
  • You can show code with echo = TRUE, turn off R formatting with results = "asis" & hide results with results = "hide"
  • LaTeX formatting is fine in markdown and outside code chunks
  • To allow for randomised questions within a question file I had to escape the R code chunk to render the output as html, else the randomised question would be rendered as R output
  • You can generate a file to go with the question using any R write to file function. Include the file in the question as normal for Rmarkdown - [filename](filelink). Leave this as default or exams will not be able to find the file

The answer list

The answer subheading is defined by the tag:

Answer
========

Bulleted markdown after this tag will be considered the options for an MCQ answer list.

Things to note

  • exams contains several helper functions to make it easy to generate lists of answers.
  • answerlist accepts a vector of answers. It also generates the answer subheading so there is no need to type it in.
  • Recommend using html in case LaTeX does not render properly, particularly when called as a string in an R code chunk.
  • Do not randomise the answer list here, use the metadata, else an incorrect answer will be assigned to be the correct one.
    comment = NA in the code chunk options will remove the # from the R output.
  • There are also helper functions for various things. mchoice2string() turns the solutions vector above into binary responses for the meta-information section. num_to_choice generates a MCQ list of numbers for a numeric answer.

Solution

You can provide feedback via the solution header, including which answers are correct

Solution
========
Your solution here or correct answer: code for answer (or answer[])
Answerlist
-———
* True
* False
* etc.

Important There should be no spaces after the header tag title, i.e. markdown formatting


Question metainformation

This is an important section of the question because it defines the correct answer. Metainformation is defined by the tag:

Meta-information
================

Useful variables are:

  • exname = title of question, becomes name of the pool in blackboard
  • extype = type of question (num/schoice/mchoice)
  • exsolution: order of correct answers in binary (e.g. 01010) for MCQ or R code for numeric output - e.g. 
  • extol = the tolerance range for numeric questions
  • exshuffle = Whether to shuffle the answers or not. This can be used to randomly select a subset of answers from an answer list. Provide a number of answers for non-numeric questions (e.g. 4 for 4 answers). TRUE/FALSE is also accepted.

An example question

This example displays the correct answer and 3 randomly chosen option out of 6 possible answers.

Question
========
What is your name?

\```{r question, results = "hide", echo=F}
# list of possible answer as a character vector
knights_of_camelot <- c("Arthur, King of the Britons", 
                        "Sir Lancelot the Brave",
                        "Sir Robin the Not-Quite-So-Brave-as-Sir-Lancelot",
                        "Sir Galahad the Pure",
                        "Sir Bedevere the Wise",
                        "Patsy")

# solution to the vector above                        
solutions <- c(FALSE, TRUE, FALSE, FALSE, FALSE, FALSE)

# explanations (feedback for students)
explanations <- c("I didn't vote for him",
        "His favourite colour is blue",
        "He doesn't know the capital of Assyria",
        "His favourite colour is blue. No, yel...",
        "Knows nothing about swallows",
        "Clip Clop")
\```
        
\```{r answerlist, echo=F, results = "asis"}
# helper function to format the list of possible answers
answerlist(knights_of_camelot, markup = "markdown")
\```


Solution
========
\```{r solutionlist, echo = FALSE, results = "asis"}
answerlist(ifelse(solultions, "True", "False"), explanations, markup = "markdown")
\```

Meta-information
================
exname: Bridgekeeper
extype: schoice
exsolution: `\r mchoice2string(solutions)`
exshuffle: 4

This renders like this:

Exam 1

    Question

    What is your name?

        Sir Bedevere the Wise
        Arthur, King of the Britons
        Sir Galahad the Pure
        Sir Lancelot the Brave

    Solution
        False. Knows nothing about swallows
        False. I didn't vote for him
        False. His favourite colour is blue. No, yel.
        True. His favourite colour is blue

You could also skip the solutions vector and include it in the explanations vector like c("False. I didn't vote for him"). Of course, the solution does not appear immediately in blackboard but make sure the option for solutions and feedback to appear is checked.


Compiling the exam

I have written a script to compile the exam. The compiling function (exams2blackboard) requires a list of file names to generate the exam. Each file represents a question. Versions of a question (n) generate a pool of questions. The list should not contain subdirectories or files not to be included in the exam. exams2html is a means of checking a file/list of exam questions renders properly in html (or exams2pdf).

There are several means of customising the metadata of the exam. Here I have:

  • turned off partial marks - is TRUE by default
  • used custom directories to search for the questions and save the output zip.
  • set 10 copies for each question using the variable n
  • set the name of the zip file using the variable name
  • set the number of points for each question to 1, default = 10
library("exams")
options(device.ask.default = FALSE)

## content and output directory
mydir <- "C:/Users/kongj/OneDrive - TCDUD.onmicrosoft.com/Teaching/Biostats/Midsem MCQ"

## define exam questions (each item in list is a pool)
myexam <- list.files(paste0(mydir,"/questions"), pattern = ".Rmd")

# render single question
#exams2html(list("question3.Rmd"), edir = paste0(mydir,"/questions"))

## generate .zip with Blackboard exam with n replicates
exams2blackboard(file = myexam, n = 10, name = 'Jacinta', dir = mydir,
                 edir = paste0(mydir,"/questions"),
                 eval = list(partial = FALSE, negative = FALSE),
                 points = 1
                 )

Things to note

  • You can create a mix of questions in the exam by defining it in the list of questions but it is not recommended for generating pools of questions
  • If the exam is complied with no partial marks, then the blackboard exam will have no partial marks even if the option is checked within blackboard
  • You can generate a pool of questions by compiling a single question and uploading that zip file to Blackboard under “Import Pool”. This is useful for creating random block tests in Blackboard if the compiled zip file is not suitable as an test right away
Avatar
Jacinta Kong
Postdoctoral Fellow

My research interests include species distributions, phenology & climate adaptation of ectotherms.

Next

Related