On the Blackboard class management system, tests may be assigned where the order of the questions is randomized for different students (useful to somewhat improve security, make sure no question is biased due to ordering, etc.). However, a problem arises: when individual results are downloaded, the questions still appear in this randomized order for each separate student. That is: questions don't match down columns, they don't match in the listed "ID" numbers, etc.; and therefore there's no obvious way to assess or correlate individual questions between students or with any outside data source (such as a final exam, pretest/post-test structure, etc.). A brief discussion about this problem can be found on the "Ask the MVP" forum on the Blackboard site (link).
Now here's a solution: I wrote a computer application to take downloaded Blackboard results in this situation and sort them back into consistent question ordering, and thereby make them usable for correlation analysis (in a spreadsheet, SPSS, etc.). Java executable JAR file is below, first download that. Test results should be downloaded from Blackboard as a comma-delimited CSV file in the "long download" format ("by question and user" in the Blackboard Download Results interface), in the same directory (save as default "downloadlong.csv"). Then, on the command line, run the JAR file on by typing "java -jar BlackboardResultSort.jar".
The program reads the downloaded data file and outputs two separate files. The first, "questions.csv", is a key to the questions, listing each Question ID, Possible Points, and full Question text. The second, "users.csv", is a matrix of the different users (test-takers) and their scores on each question (each row is one user, and each column is their score for one particular question, consistent as per the questions.csv key). This makes it far more convenient to add outside data correlate success on any particular question with overall results. Ping me here if you have other questions.
Subscribe to:
Post Comments (Atom)
Usage note: Consider inserting a dummy student at the start of your data with the questions appearing in the original order (or whatever order you want to see them in for the assessment).
ReplyDelete