Auditing the Intellum LMS for Improved User Experience

⏰ Duration: 3 months

🏢 Client: Google

👩🏻 Role: Lead Researcher

🧰 Tools: Google Sheets, Google Apps Scripts, ChatGPT

🔍 Methods: Benchmarking, Descriptive Statistics

*Data is owned by client and visuals of the audit cannot be shown in the portfolio.

📝 Executive Summary

This project's task was to audit the client's course content in their Intellum Learning Management System (LMS). The goal was to assess the quality of instructional content, categorize courses based on relevance, enhance content to align with current trends, and improve learner engagement. Quantitative research methods, including data collection, cleaning, analysis, and data visualization, were employed throughout the project. The systematic process involved data collection, analysis, identification of outdated materials, enhancements, and the establishment of a feedback loop with learners.

Key findings highlighted the importance of engagement metrics, user feedback, and collaborative tools in effective UX research. The project recommended monitoring engagement metrics, implementing regular user feedback mechanisms, and exploring additional collaborative and analytical tools for future projects. Post-implementation, the project led to the identification of outdated materials, improved relevance of instructional materials, and an enhanced user experience. This case study underscores the significance of data-driven approaches in optimizing learning management systems, emphasizing the value of user feedback, engagement metrics, and efficient collaboration tools in UX research.

📖 Introduction

In this project, I delved into the project that was undertaken to enhance the Intellum Learning Management System (LMS). The Intellum LMS served as a dynamic educational platform utilized by various organizations. The primary challenge was to audit and improve the system to ensure that the instructional materials provided were not only relevant but also effective for learners. As a UX researcher, my role was to lead the research efforts aimed at achieving this objective.

🏆 Project Goals

The main objectives of this project as as follows:

Quality Evaluation

Approach: Implement a systematic evaluation process to review instructional content on the Intellum LMS, assessing factors such as accuracy, educational value, and alignment with industry standards.
Outcome: Ensure that all instructional materials meet high-quality standards and remain relevant to the current needs of learners.

🔍 Research Methods & Process

Data Collection and Cleaning

Process: The process involved extracting 15,000 rows of data from the Intellum LMS database. This data was comprehensive, covering aspects such as course titles, descriptions, upload dates, user engagement metrics (like completion rates, time spent per course), and direct user feedback.

Cleaning Techniques: To ensure data integrity, we applied rigorous cleaning techniques. This included removing duplicates, correcting inconsistencies, and standardizing data formats. We also anonymized user data to maintain privacy and compliance with data protection regulations.

Data Analysis

Tools and Techniques: Google Sheets was selected for its accessibility and collaboration features. Advanced functions like XLOOKUP, INDEX, and MATCH were used to organize and cross-reference data efficiently. This allowed us to analyze relationships between different data types, such as course usage and user feedback. Generative AI helped to create the advanced functions and to check if there were errors in the work.

Google Apps Scripts Usage: We utilized Google Apps Scripts for automation tasks like data cleaning, formatting, and creating custom functions for more nuanced data analysis. It also facilitated version control and streamlined collaborative editing among the research team. Generative AI was also useful for checking code errors as well as suggestions for efficiency in analyzing data.

Data Visualization

Approach: Data visualization played a key role in identifying trends and patterns. We created various charts and graphs within Google Sheets to visually represent data, such as user engagement over time, feedback scores, and course completion rates.

Insights Gained: Visualizations helped us quickly identify courses with low engagement or high negative feedback, guiding our focus towards areas needing immediate attention.

👍 Recommendations

These findings collectively contributed to the project's success in enhancing the Intellum LMS, ensuring improved user experience, and maintaining the relevance of instructional materials. The insights gained from this research underscored the significance of data-driven approaches in optimizing learning management systems, reinforcing the importance of user engagement metrics, user feedback, and efficient collaboration tools in UX research and decision-making.

Key Finding: Our analysis revealed that course completion rates and the amount of time users spent on each course were strong indicators of course effectiveness.
Explanation: Courses with higher completion rates and longer engagement times were generally well-received by learners. This finding underscored the importance of not only delivering informative content but also creating courses that are engaging and conducive to completion.
Implications: To optimize the Intellum LMS, it became evident that instructional designs should captivate learners and encourage them to complete the courses. Strategies to enhance engagement, such as incorporating interactive content or more engaging course materials, were recommended.
Further Actions: It was recommended to regularly monitor these engagement metrics to assess course performance and identify areas where engagement strategies could be improved.

Key Finding: The use of collaborative tools such as Google Sheets and Google Apps Scripts significantly improved the efficiency of data analysis and collaboration among the research team.
Explanation: These tools allowed for real-time data manipulation and analysis, making it easier to identify trends and collaborate on findings.
Implications: The finding highlighted the importance of selecting the right tools for data analysis in UX research. The ability to quickly analyze large datasets and collaborate effectively can significantly impact the speed and quality of insights gained.
Further Actions: It was suggested that future projects could benefit from exploring additional collaborative and analytical tools to further streamline the data analysis process. Continuous learning and adaptation of new technologies in data analysis were deemed essential to maintaining efficiency in UX research.

Key Finding: User feedback emerged as a critical component in assessing the relevance and effectiveness of courses.
Explanation: Feedback from users provided direct insights into their perceptions and highlighted specific aspects of courses that were either well-received or needed improvement.
Implications: This finding emphasized the value of establishing a continuous feedback loop with learners to keep course content relevant and effective. It also highlighted the need for a responsive system where user feedback directly informs content updates or decisions regarding course deprecation.
Further Actions: The recommendation was to implement regular user feedback mechanisms, such as surveys and open forums, and to integrate this feedback into the course development process to maintain a high-quality LMS.

💭 Reflection

The project underscored the significance of data-driven approaches in optimizing learning management systems. It emphasized the value of user feedback, the importance of engagement metrics, and the efficiency of collaborative tools in conducting effective UX research.

Overall, the project successfully leveraged advanced data analytics to enhance the Intellum LMS, resulting in improved user experience and content relevance. This case study demonstrates the depth and rigor of the research process and highlights the importance of continuous adaptation to new technologies in data analysis.