Interactive learning through AI-powered Project Simulation
Less than 15% of people finish their online courses. This project is about how I, Ajay Pawriya, at Bluelearn fixed this issue.
NOTE: Because of company pivot, this project' production was paused mid-way. But I loved this concept, enough to not let it die. Hence, I am currently building a MVP version of it myself using Bubble.io. You can check it out here.
ROLE
Product Designer
TEAM
Ajay Pawriya (Me), Rohan (PM)
WHEN
2024
Problem Statement
Context
Bluelearn is India's largest online students community of 200k+ students. It's like an online university for students to Learn, Earn and Network.
For Learn, Bluelearn has a feature called "Projects" which is very much similar to how normal projects work on Udemy, Coursera and other online learning websites.
But we noticed that the completion rates for users completing these projects was around 15% which is very similar to traditional online learning platforms like Coursera, Udemy etc.
The primary reason being, Students often lose motivation and excitement of doing these projects/courses.
In the landscape of major Ed-tech platforms like Coursera and Udemy, there has been little to no innovation in the last decade. We are still using that same old step-by-step static format of doing courses.
Redefined Problem Statement
How can we create a more engaging learning experience that maintains user interest throughout the learning process?
Target Audience
Age: 17-25 yrs old
Employment: Tech savvy, unemployed actively looking for job/internships or looking to upskill
People living in Tier 1 and 2 areas, since they know the real benefit of getting a job and learning on site
Research
Competitor Analysis
Existing platforms rely heavily on passive learning methods. Innovations like job simulations provided by platforms like The FORAGE are interesting but lack interactivity and personalization, which are crucial for engagement.
What I liked about Forage
Job Simulations that make you feel like you are actually working in a company while doing a project.
Solutioning
Let's look within
Remember our parents telling us?
Learning by doing is the best way of learning
Ask anyone the amount of learnings they had while doing an actual project? Or Ever took a project and then learnt how to do it along the way?
I think you get the gist of it. We all learn the best, when we are actually practicing it and then apply it in real-time.
Solution Considerations
The primary considerations were:
Scalability - Because traditional courses platform rely on huge number of educators creating content for the course. This is a very high friction activity and very operations heavy as well.
Personalization - No point building a new format if the users are not interested or engaged enough to try out the product. Hence it has to be personalized.
Finalized Solution
The Idea of "Project Simulation" powered by AI, using an LLM API for dynamic and personalized project generation and chat support.
Usability Testing
Tech Feasibility Check
No point building anything if later down the line you realize that the solution is not feasible. Hence, I tend to keep communicating with my Dev team to do feasibility checks alongside.
💡 TAKE MY WORD FOR IT: Often times you'll get easier and crazier solutions from your developers than sitting alone and brainstorming.
Me and Rohan(PM) sharing the idea with our tech team to get their ideas
It was a big step that we were taking towards reimagining online learning. It would take a lot of Bandwidth from multiple teams to work on this.
To convince the stakeholders for this project and the user feedback as well, we ran a MVP testing batch for a week.
MVP Testing Process
MVP Testing structure
Experiment Setup
Multiple users were requested to volunteer for the Beta testing programme for this. To ensure no bias or skewed results, we took small chunks of users from various cohorts.
People who had never heard of bluelearn
Bluelearn users who never used Projects feature in the app
Bluelearn users who had completed atleast 1 project from bluelearn library
A WhatsApp group was created for each participant, including the user, the product manager, and an AI client powered by ChatGPT. Participants were unaware that ChatGPT facilitated the interactions.
MVP Beta testing WhatsApp groups
User-Client Interaction
I, as client, used ChatGPT to interpret and respond to users' messages, simulating dynamic, real-time conversations. This setup aimed to mimic genuine client-project communications without disclosing the AI's involvement.
But the user wasn't made aware that the responses are coming from AI model.
Experiment Flow and Environment
Scope of conversation that happened in Beta Testing WhatsApp groups
AI Training and Prompting
We trained our own custom AI ChatGPT to take on this task. It required us to iterate over our prompt over 20 times.
Prompt was rigorously tested along with all the edge cases and possible responses.
A sneak-peak into earlier version of our prompt 👀. (Don't try to steal it. Its very early variant and is very inefficient)
Initial version of the Tech prompt for MVP testing
Observation & Feedbacks
Engagement: The responsive AI-driven interaction significantly enhanced user engagement with average <1hr response time
Completion Rates: About 80% of participants completed and submitted their projects, demonstrating improved engagement compared to traditional methods.
User Experience Feedback: All participants noted an improvement with the AI simulation; 67% found it better than traditional projects, and 33% considered it much superior.
Overall Positive feedback from users during beta testing
Final Designs
This was an additional feature that we had launched on bluelearn, hence I used existing brand guidelines, components and style guide.
💡By the way, I took up the lead to build a robust and scalable Design system for bluelearn from scratch. Check out the case study here.
Revenue opportunity: After 3 project generations, the users can buy credits to generate more projects.
Since the project creation takes upto 30 seconds, we planned to reuse our physics engine component from one of our landing pages to keep the user busy by fidgeting with it. They can throw these shapes around and fidget with the,.
Final project submission and closure
Possible ideas for next versions
Feedback and Evaluation through AI
Since the entire chat between the client and the user happened on-the-platform, we can use it to give users feedback on multiple aspects like:
Communication style
Promptness in replies
Ability to ask the right questions
Introducing BL Coins was also on the table
Automatic Case Study Builder
The entire conversation contains the steps, the challenges, the scope and much more data that can be used to automatically prepare draft 1 of case study for the projects. (If you're a designer you'd know how tiresome it is for some to write case studies to update their portfolio)
Community Feedback on Project Submissions
No matter how advance an AI model et, the real and qualitative feedback is best given by real humans. Hence, in future we also plan to revamp our existing submissions feed to be more suitable for feedback collection.
Checkout the Figma Prototype for some ideas of new submissions feed that I had.
Original Figjam Brainstorming file
Have a behind-the-scenes look at what the brainstorming for this feature looked like in early stages.
Conclusion
Growth happens by doing things you are unqualified to do
-Some person on Twitter
This was the first time I ever worked on AI powered ideas. This project not only enhanced the learning experience but also pushed the boundaries of educational technology by integrating AI into the core of the learning process. The results from the MVP testing underscored the potential of this innovative approach, setting the stage for further development and refinement.