Tuesday, November 18, 2008

Powerful Web Analytics in Education

I did a quick analysis of an educational course and I have some ideas and a question. Not much has been done in education with web analytics, so it is ripe for the taking.
First I believe that unique pageviews may be very beneficial (as long as students are not deleting their cookies too often). This may also be interesting to compare to pageviews to see how often people return to certain pages. One of the main things that I found with unique pageviews was that people were not completing the course. A lot of them would start, but many of them never finished.
Tracking the direction of students through the course may also be very beneficial. I did not use the most dependable way to track students, but I did look at the site overlay and found that about a third of students are skipping over the middle part of the lesson straight to the end. I would be interested to back this information up with usability testing or interviewing to find if the middle of the lesson is thought of as a waste of time, or just is not helpful. I would guess a little bit of both.
I did not do any analysis of time on page, but I would trust that it would be a rich source of data.
One of the interesting statistics I looked at was bounce rate. I tried to compare high bounce rate on a normal educational page to the type of page it was. It appeared that a high bounce rate was connected to content that was harder to understand. The bounce rates were lower with about 50 instances, but may be pointing in the right direction. I also thought that bounce rates were hitting a page and then directly leaving it, but I must be mistaken. I saw some pages with a 100% exit rate, but a lower bounce rate. If bounce meant hitting a page and leaving directly from it, then the exit rate should always be higher. So my question is what is a bounce rate? If it is hitting a page and leaving it, or just a short time on a page, then we can distinguish between some of the user actions. The same with exit rates. If they are leaving a page more than other similar pages, maybe they are going to a textbook or somewhere else to learn a concept. We may need to bolster the content to meet the students' needs. Could bounce rates and exit rates be a key to making educated guesses about the content on a page? At least it could lead us to some A/B testing and see if the statistics change.
It seems that the forced path that students go through in a course changes the statistics that are needed to analyze a website. Getting someone to buy items on a website is very different than helping a student learn and complete a course. The search for beneficial KPIs may be quite a challenge. It will be an interesting collaborative effort by all that are studing this emerging field.