In this section we will provide information about forming a line of inquiry, supporting what question(s) to ask and what data may be available to support the inquiry.
Using data to understand student learning is a long-standing practice in educational research. Recent improvements to Quercus analytics and in our ability to assemble and analyze large sets of data with the dashboard offers UofT instructors greater access to data and tools.
Instructors typically use LA data and tools to
- gain a sense of the likelihood of success of students prior to the start of a course,
- identify students who may be in danger of failing or performing poorly in a course while the course is being offered,
- identify learning behaviors that are correlated with student success (or the lack thereof) in a course,
- identify course materials and assignments that are correlated with student success, and
- encourage students to engage in effective learning behaviors and engage with relevant instructional materials (e.g., through messaging).
Adopted from: https://learninganalytics.colostate.edu/reasons/
To begin, we can clarify and consider what is in and out of scope with LA at UofT.
Learning Analytics Activity at U of T
In Scope
- Course improvement
- Improvement of learning activities
- Course materials development and improvement
- Instructional improvement and self-evaluation
- Program review, evaluation, and accreditation
Out of Scope
- Research requiring oversight by the IRB (Institutional Review Board)
- Instructor evaluation by the institution, including tenure review
Data Used for LA Purposes
In Scope
Data from face-to-face, online, and hybrid courses, and other learning experiences, such as:
- student work
- learning and student engagement
- assessment (grades, rubrics, direct assessment of outcomes)
- attendance and participation
- formative and summative assessment
- course evaluations (for instructors personal use only)
Out of Scope
- Student-disclosed mental health information
- University Health Services records and other HIPAA-protected data
- Data on student appeals, misconduct, or complaints
- Students’ financial aid data
- Disability status
- Religious, political, or union participation
What to Investigate? How to decide what to ask/look for?
Text about inquiry process to be developed here.
Borrow from https://wp.nyu.edu/tlt/learning-analytics/analytics-insights-portal/
Planning
Analyze assessment or content data at a lesson or whole-course level.
- Saves time in your curriculum review.
- Focuses data in meaningful ways.
Find engagement drop-off around lessons or specific materials.
- Helpful for learning what material you might want to swap.
- See how long students are spending on your course materials and which days they tend to spend the most time so you can adjust your lesson due dates to their study habits.
Teaching
Combine assessment data with engagement data during the course.
- Change a student trajectory during the course.
- Add supplementary material where you see gaps.
Identify low engagement early.
- Be confident in nudging a student you want to see improve.
See the student pathways in your course.
- Know which order students are viewing your material and make suggestions on their study habits as needed.
Adopted from: https://wp.nyu.edu/tlt/learning-analytics/analytics-insights-portal/