|
|
|||||
|
||||||
|
|
|
|
|
|
||
How do you know that technology is making a difference? As states and districts across the country continue to make significant investments in technology infrastructure and teacher time to develop the skills to master the use of this infrastructure, answering the "does it make a difference" question becomes increasingly important.
In numerous resources on this site, we detail processes for evaluating impact and research that indicates that technology does indeed positively impact teaching and learning. In this article, we will focus on an even more basic issue that should confront anyone interested in documenting and measuring technology's impact, that is, the relationship between planning goals and the results of the work spent achieving those goals.
Good Goals
What is a goal? Conventionally, a goal is a statement of intent or action. Within the context of educational technology planning, a goal is statement for what you wish to achieve through the application of technology to a teaching and learning environment. In a related manner, you might have a goal for performing some set of actions (e.g., professional development) which have as their end result the improvement of teacher and/or student abilities to master curriculum, perform technical tasks, or achieve some set of standards or proficiencies. Once again, all goals are designed to describe something you want to achieve.
With this in mind, it seems strange that districts with technology plans often have seemingly no idea of what they hope to achieve through the use of educational technology. How could this be? All technology plans have goals, so why aren't these goals called into play when it comes time to evaluate or assess technology's impact? In our experience, there are several reasons for this disconnect.
The technology plan which reads mostly as an infrastructure plan -- e.g., "We will purchase and install three computers per classroom." -- lends itself mostly to assessing the physical infrastructure and not the impact of that infrastructure. If your "goal" is simply to buy computers, then you measure the success of that goal simply by counting the purchased computers; but this tells you nothing about how those computers actually impacted teaching and learning. Therefore…
Create goals for educational outcomes, not technology-specific outcomes
Even when goals relate to student, teacher, or curricular outcomes, the relationship is often very vague. Goal statements along the lines of "We will use technology to improve student learning" or "Teachers will master the use of technology tools" provide no guidance as to the intended or supposed relationship between technology and student learning (for example). While it is not necessary to minutely detail cause and effect in a goal statement, it is important to at least categorically create relationships between technology and the intended impacts it will have on those who use it. Therefore…
Be specific as to what your teachers and/or students will achieve through the use of technology tools
Students, teachers, and the broader community work together in a complex system. In other words, student achievement is not a random phenomenon disassociated from teacher training, assessment, community support, and other key pieces of the "system" we think of as education. Even when examining a single piece of this system -- e.g., technology -- you need to be cognizant of the relationship of all of the pieces and this needs to be reflect in our planning goals. Thus, in technology plans, you need to write goals which focus on students, teachers, community, and then the relationship between these various components. Goals for student achievement which do not account for the role of teacher training to support this achievement are significantly incomplete. Therefore…
Make sure that your student and curricular goals are supported by equivalent goals for teacher training and learning.
A more detailed explanation of goal creation -- including what constitutes "good goals" -- can be found in our Planning into Practice publication as well as other technology planning resources linked to our website. By following these tips you can create meaningful technology planning goals which lend themselves to evaluation.
Measuring Progress
With good goals in your technology plan, you now have a sound basis for creating meaningful evaluation questions. Essentially, your questions should be rephrased goal statements. For example:
Goal = All district teachers will be provided with the professional development necessary to embed technology as a tool for student learning across the curriculum.
Evaluation Question = Have our teachers been provided with the professional development they need to effectively embed technology as tool for learning across the curriculum?
Clearly, a meaningful answer to this question is not "yes" or "no." Rather, you need to determine what constitutes professional development necessary for teachers to achieve this goal/objective (i.e., embedding technology as a tool for learning…). Ideally, a description of this professional development is provided in the action plans which support this particular planning goal in your technology plan. Maybe this isn't the case; and if so, you now have a concrete example of how the action plan portion of your technology plan could be updated. If your plan does provide this level of detail, then your next task is to take these "steps" and document whether or not they have been taken.
Districts we work with to create educational technology evaluations take these intended outcomes and further break them down into a set of indicators. Realizing that most achievement is realized in degrees versus absolutes (i.e., "yes we did" or "no we didn't"), we assist districts in creating indicator rubrics which portray varying degrees of success in meeting particular planning goals. Once data is collected on current levels of achievement, districts can place their achievement on a scale from less- to more-successful. In most cases, this represents a clear measure of progress and provides a graphic picture of what has been done and what remains to be done toward meeting a given technology goal.
It's All Relevant
To recall how we initially approached this topic, it's all about your goals for your district's intended outcomes. Meaningful evaluation comes from assessing progress toward meeting the goals you had for the process or activity which you are evaluating. Therefore, meaningful technology evaluation is not about trying to attribute various educational outcomes (e.g., test scores) to various educational inputs (e.g., technology). You can only evaluate cause and effect in those systems where there has been a prior connection made between causes and effects. Your planning goals should establish this connection and this is why your evaluation must be rooted in these goals.
Information on this site that has been produced by Sun Associates is Copyright 1997 - 2013 Sun Associates and is available for individual, one-time, use by educators. Duplication is prohibited without permission. All other material is the property of its authors and Sun Associates makes no warranty for its use or accuracy.
Last updated, 4/2/01