Evaluating Asynchronous Client training in Edtech: A Program Evaluation of Finalsite’s Composer Learning Path

Abstract

Client-facing training in EdTech remains a blind spot in evaluation research. Most corporate training studies focus on internal employee development, leaving unanswered questions about how technology companies effectively prepare their external clients to use complex products. This mixed-methods program evaluation examines Finalsite's Composer Learning Path, an asynchronous training program redesigned in 2024 from lengthy video courses into 73 micro-courses serving over 50,000 learners.Drawing on Self-Determination Theory, Kirkpatrick's Four-Level Evaluation Model, and Merrill's First Principles of Instruction, I evaluated the training through survey data from 212 participants and semi-structured interviews with 7 stakeholders. The evaluation addressed seven research questions examining learner expectations, content alignment, perceived quality, confidence development, and transfer to practice. What emerged was a paradox. The training looked great—professionally produced, interactive, well-received by learners—but it was strategically misaligned with who was taking it. The program assumed all users needed comprehensive platform mastery when four distinct personas showed up with wildly different needs. Full-time webmasters thrived. Teachers managing a single classroom page felt overwhelmed by content they'd never use. The pathway was built for 30-40% of its audience. The data surfaced other surprises. Participants reported higher confidence in advanced skills than foundational ones—an inverted mastery pyramid that shouldn't happen if learning is progressing normally. This pattern pointed to false confidence built in simplified simulations that didn't prepare users for their actual sites. The takeaway is that organizational readiness and implementation timing can undermine even well-designed instruction. Recommendations target both the training itself and the broader client success ecosystem that shapes whether learning ever gets applied.