Skill Guide

Data-driven decision-making using institutional analytics and learning management system data

The systematic process of extracting, analyzing, and interpreting quantitative and qualitative data from institutional analytics platforms and Learning Management Systems (LMS) to inform and validate strategic, operational, and pedagogical decisions.

This skill enables organizations to move beyond intuition-based management, directly linking learning interventions and operational changes to measurable outcomes like retention, completion, and skill acquisition. It maximizes ROI on educational technology investments and institutional resources by identifying precise points of intervention and evidence of impact.

1 Careers

1 Categories

9.0 Avg Demand

15% Avg AI Risk

How to Learn Data-driven decision-making using institutional analytics and learning management system data

1. **Master LMS Data Fundamentals:** Learn the data dictionary of a major LMS (e.g., Canvas, Moodle, Brightspace). Understand core metrics: login frequency, page views, time-on-task, submission rates, and grade distributions. 2. **Learn Basic Quantitative Analysis:** Build proficiency in spreadsheet software (Excel/Google Sheets) for pivot tables, VLOOKUPs, and basic charting to summarize LMS data exports. 3. **Understand Key Educational Metrics:** Familiarize yourself with definitions of leading indicators (engagement) and lagging indicators (grades, completion).

1. **Apply Analytical Frameworks:** Move from describing *what* happened to diagnosing *why*. Use frameworks like Cohort Analysis (comparing student groups) or Funnel Analysis (tracking progression through a course module). 2. **Integrate Multiple Data Sources:** Combine LMS data with SIS (Student Information System) data (e.g., demographics, prior GPA) to uncover equity gaps or correlation patterns. Avoid the common mistake of drawing causal conclusions from correlation alone. 3. **Conduct A/B Testing:** Design and analyze simple interventions (e.g., two different types of discussion prompts) using split groups within the LMS to measure impact.

1. **Build Predictive Models:** Develop or interpret early-alert models that use LMS activity patterns (e.g., sudden drop in logins) to flag at-risk students for proactive intervention. 2. **Design Data Dashboards for Leadership:** Architect actionable dashboards (in Power BI, Tableau) that align LMS analytics with institutional KPIs (e.g., gateway course success, term-to-term persistence). 3. **Establish Data Governance & Ethics:** Create policies for data privacy (FERPA/GDPR), algorithmic bias review, and ethical use of student data for decision-making. Mentor teams on interpreting data within its proper context.

Practice Projects

Beginner

Case Study/Exercise

Identifying At-Risk Students in a Single Course

Scenario

You are a course coordinator. You have exported weekly LMS activity data for an introductory math course (N=200). Midterm grades just posted, and 40% of the class scored below 70%.

How to Execute

1. **Isolate the Data:** In a spreadsheet, filter the data for the week prior to the midterm. 2. **Calculate Engagement Metrics:** Compute each student's total page views and time spent in the course for that week. 3. **Correlate with Grades:** Create a scatter plot of 'Time Spent' vs. 'Midterm Grade'. Use conditional formatting to highlight students with low time spent AND low grades. 4. **Report Findings:** Draft a one-page brief stating: 'X students with below-median engagement had a Y% probability of scoring below 70%.'

Intermediate

Case Study/Exercise

Evaluating the Impact of a New Pedagogical Intervention

Scenario

The university piloted a 'weekly video feedback' tool in 20 sections of a critical writing course. The dean wants to know if it should be scaled. You have LMS data (engagement) and final grade data for the 20 sections (treatment) and 20 comparable sections (control) from the prior term.

How to Execute

1. **Define Treatment & Control:** Clearly label sections using the new tool as the treatment group and prior sections as the control. 2. **Match Cohorts:** Use propensity score matching or carefully select control sections with similar prior student performance distributions (from SIS data) to reduce bias. 3. **Analyze Key Metrics:** Conduct a difference-in-differences analysis. Compare the change in average final grade and student satisfaction survey scores between groups. 4. **Account for Confounding:** Check if other variables (e.g., instructor experience, class size) differed significantly and note them as limitations in your report.

Advanced

Project

Institutional Dashboard for Gateway Course Success

Scenario

You are the Director of Institutional Research. The provost needs a live dashboard to monitor the 'gateway course' initiative (courses with historically high DFW rates) across all departments, linking student effort to success and disaggregated by demographic subgroup.

How to Execute

1. **Define KPIs with Stakeholders:** Work with deans to select 3-5 core metrics (e.g., % scoring C or better, LMS engagement index, withdrawal rate). 2. **Architect the Data Pipeline:** Design an automated ETL (Extract, Transform, Load) process pulling daily from LMS API and nightly from SIS into a data warehouse. 3. **Build the Dashboard:** Use Power BI/Tableau to create interactive visuals: trend lines over time, drill-down by department/course/section, and a demographic filter (race, Pell status, first-gen). 4. **Implement a Review Protocol:** Establish a monthly meeting cadence where department chairs use the dashboard to conduct 'data digs' and present action plans. Document how dashboard insights directly led to a specific resource allocation or curricular change.

Tools & Frameworks

Software & Platforms

Learning Management System (Canvas, Moodle, Brightspace)Data Visualization Tools (Tableau, Power BI)Spreadsheet Software (Excel/Google Sheets)Statistical Software (R, Python with Pandas/Seaborn)BI/Data Warehouse (Snowflake, BigQuery)

LMS is the primary data source. Visualization tools translate complex data into actionable dashboards for leadership. Spreadsheets are for ad-hoc analysis and data cleaning. Statistical software is used for advanced modeling and hypothesis testing. A BI platform is essential for scalable, automated reporting at the institutional level.

Analytical Frameworks & Methodologies

Cohort AnalysisA/B Testing (Randomized Controlled Trials)Predictive Analytics (Logistic Regression, Random Forest)Root Cause Analysis (5 Whys, Fishbone Diagram)Logic Models & Theory of Change

Cohort Analysis tracks outcomes for specific student groups over time. A/B Testing isolates the causal impact of a specific change. Predictive models identify risk factors proactively. Root Cause Analysis drills down from a symptom (low grades) to a systemic cause. Logic Models connect activities (e.g., tutoring) to outputs (participation) to outcomes (retention), providing the theoretical backbone for data analysis.

Interview Questions

Answer Strategy

The interviewer is testing systematic problem-solving and the ability to connect data to actionable insights. **Strategy:** Outline a structured investigative process, not just a single metric. **Sample Answer:** 'I'd start by confirming the retention drop is real and not a data artifact. Then I'd segment the data: Did it drop for all students or specific cohorts (e.g., those entering with lower pre-requisite grades)? I'd analyze LMS engagement patterns for the retained vs. withdrawn cohorts, looking at specific early milestones-like completion of the first module quiz or participation in Week 3 discussion. A key step would be correlating this with instructor feedback data and student survey comments to identify if the issue is content difficulty, platform usability, or lack of community. The output would be a diagnosis pinpointing 1-2 primary levers for intervention.'

Answer Strategy

Tests courage, analytical rigor, and influence. **Core Competency:** Demonstrating data advocacy and change management. **Sample Response:** 'Many assumed mandatory synchronous sessions were key to our online course success. I analyzed engagement data from 50 course sections, comparing synchronous-attendance-optional models versus mandatory models. I controlled for course difficulty and instructor. The data showed no statistically significant difference in final grades, but student satisfaction scores were significantly higher in the optional model, especially for working adults. I presented this analysis to the curriculum committee, framing it as a matter of equity and flexibility. The result was a policy change to make synchronous sessions optional but highly encouraged, with recordings and asynchronous alternatives, improving both satisfaction and accessibility.'