Ashley’s Blog

It’s hard to believe that this semester is nearly over and that the holidays are right around the corner. I have learned an immense amount of information relating to evaluations. I found the Appreciative Inquiry approach quite interesting and feel like it should be incorporated into all evaluations. I don’t think it’s necessary to conduct the entire evaluation using AI, but it certainly can be beneficial to ask at least one or two AI questions that will bring a positive light to the evaluation.

I also learned a great deal of information by conducting my own “hypothetical” evaluation. I never realized how involved evalautions could be until I started my own. To a certain extent, I am really glad that we didn’t have to actually conduct the evaluation because I just don’t know where I would have gotten the time to get everything completed. On the other hand, it would have been interesting to see how my evaluation would have turned out by disecting all the collected data and identifying alternative aspects of the program. Overall though, I feel fortunate to have learned about the process of completing an evaluation because I have a greater respect for those that do complete evaluations.

I am very pleased with the amount of work I was able to accomplish on my paper this past week, and I am thankful for Kona and Pam who looked it over and offered suggestions. I have implemented all of the formatting changes that Kona suggested, and I plan to make other changes that Pam suggested this week along with reading it through one more time…as if I haven’t read it enough. Therefore, my goal is to have the final draft of my paper finished before Thanksgiving.

I didn’t have much of a chance to work on my PowerPoint this week, so that is also something I need to complete this week or next week. I really don’t think it should be too challenging now that I have my paper finished. The end is definitely in sight…

I’ve really enjoyed having a couple weeks with no assignments due. However, it’s been somewhat difficult for me to stay motivated to work on my paper. I haven’t had many questions on what goes into the paper or how to write it, but it’s been hard to continue working on it. With all that said, I am almost finished with the paper. I’ve finished the results section and am nearly finished with the discussion section. My goal is to finish the discussion this afternoon, and then post my paper this evening or tomorrow for review.

These past couple of weeks, I also began working on my PowerPoint presentation. I think it’s going to be very challenging to keep the presentation to only 5 minutes when there is so much material in the paper. I’m trying to keep the slides as simple as possible so that I can focus more on the discussion section of the paper which in my opinion is one of the most important parts of an evaluation. It is in this section where different possible outcomes are outlined based on the results of each interview and survey.

This class has really opened my eyes about all that is involved in an evaluation. I didn’t realize that it took so much work and attention to detail. I also didn’t know what to expect when I first signed up for this course, and I definitely had something else in mind when I read the course title. However, now that we are finishing up the loose ends, I have come to “appreciate” all that I have learned. I’m not sure if or when I’ll ever be able to use the knowledge that I’ve learned, but I’m sure it will come in handy sometime in my teaching career.

The one thing that I wished we could have done in this course was look at other types of evaluation. It seems that we spent much of our focus on Appreciative Inquiry, which is a great way of conducting an evaluation, but I’d also like to know what else is out there. I’m sure that other types of evaluation methods are just as successful, so it would have been interesting to look at those as well.

As many have already said and as I’ve mentioned, I’m really grateful that we have been able to develop our evaluation projects along the way in order to receive feedback. As I looked at the last stages of our evalaution this week, I wished we could have develop our results section in much the same way that we did the first three sections so that we could have received feedback on it. It also would have “forced” me to write it and not put it off like I did this week. I did, however, put the first three sections together to make sure they flowed nicely and formated the paper in a similar way to the example papers that were posted. I feel like I still have a lot of work to do, but this week I am going to work on the results and discussion section. Then the following week, I hope to put my PowerPoint together for the presentation. I can’t believe that November is already here. Time continues to fly by the older I get…and I’m not even that old!

This week we first looked at evaluation systems. Preskill states that an evaluation system “is a means for identifying, developing, implementing, and sustaining those things that support evaluative inquiry.” There are five essential components of an evaluation system. They are leadership commitment, evaluation vision and philosophy, evaluation strategic plan, evaluation design and implementation requirements, and personnel and financial resources. If one or more of these components are missing, then the evaluation system may not work efficiently or effectively. Then we looked at building evaluation capacity through AI by means of “improving an organization’s ability to achieve its mission or a person’s ability to define and realize his/her goals or to do his/her job more effectively.” Both of these topics essentially made me understand how important it is to remember to think of everything and to be aware of who I’m interviewing or surveying and how it’s being done.

As for my final course project, I have made some revisions to the first three parts, but I still need to put all the sections together to make sure they flow logically and finish up the last two parts. After that, I’ll need to put my PowerPoint together. I’m really glad that we had the chance to complete this project in pieces and receive feedback along the way. It has been a very helpful process.

The process of developing surveys was not as easy as I originally thought it would be. I really had to sit down and think about the most important things I wanted to ask, and then figure out how to ask them. One thing I struggled with was wording the closed-ended questions with an AI approach. It’s very easy to write AI open-ended questions, but the textbook didn’t really focus on writing AI closed-ended questions. Therefore, I simply tried to keep the questions positive while incorporating the succeses, wishes, and values quetsions at the same time. I hope I have accomplished this.

I really enjoyed working with Michael, my dyad partner, again this week. We were both able to bounce ideas off one another in order to make our questions better. One thing that he had to keep telling me was to ensure that I was specific enough in order to get the repsonses that I needed. I am thankful that he was able to see that I wasn’t being specific so that I could change my questions. It’s always nice to have someone test out your questions to make them as good as they possibly can be.

This week we were challenged with writing interview questions for our evaluation. I knew the questions I wanted to ask, but the hard part for me was putting an appreciative inquiry spin on them. As I read chapter 4 in the text, Preskill said to ask questions about successes, values, and wishes. Therefore, I really tried to ask questions that related to these three things, while still asking questions that I wanted answers to. I also found that wording the questions was very important when writing them with an AI approach. This is where Michael, my dyad partner, helped out. We didn’t conduct a “true” interview because neither one of us would have been able to answer the questions, but we did help each other with wording and content.

One of the things Michael told me was to be more specific in the questions that I asked. If I was looking for a certain response, then he said I should give examples or criteria for the interviewees to think about. However, I didn’t want to be so limiting that my examples or criteria would sway their responses in a way that they wouldn’t think about other things. I hope that made sense. So, I did make the questions a little bit more specific, but I tried not to make them limiting.

Overall I enjoyed having the opportunity to bounce my ideas and questions off Michael before posting them in the forum…and I hope that I helped Michael out in the process too! It was nice to approach the assignment in a different way this week as we usually post our thoughts, then get opinions on it.

Here are the interview questions that I’ll ask the math teachers (in case you haven’t read them in the forum):

1. Think back to all your former and current students enrolled in the math tutoring program. Tell me about one or two students who had success due to their involvement in the tutoring program. Why were they successful?
2. Again, think about those students most successful in the math tutoring program, and describe them based on the program’s three qualifying criteria: poor attendance habits, free/reduced lunch, and overall ability in math. Do these students generally meet all three criteria?
3. Based on your response to question 2, do you feel that the tutoring program should be opened up to all students, not just those meeting one or more criterion, in order to make the program more successful?
4. When the math tutor is in your classroom during the morning hours, describe a situation in which she has been most helpful and provided the most assistance to the students. Does this situation occur often or are there times when the tutor’s assistance is not needed?
5. If you could improve two or three aspects of the current math tutoring program to make it more beneficial for the students, what would they be, and why? In thinking of your response, you may want to refer to student selection, schedule of the tutor, or other aspects of the program.

Sometimes, it’s difficult for me to write broad questions when as a math teacher I look at every specific detail. However, I found it very easy to write the key evaluation questions because I was able to base them on the three aspects of the math tutoring program. They were:

• Does the current two-day per week schedule of Mrs. Smith provide enough service to the students?
• Is the method of choosing students for the program effective?
• Are the grades of students enrolled in the tutoring program increasing or remaining the same?

The students are obviously the most important part of this program and if they aren’t benefiting, then something has to change. Therefore, I wanted my questions to all focus on the students The first question is definitely one that I have wanted to answer since I began teaching at the school. Math is one of those quick-paced subjects. Sure everything builds, but new concepts are being taught nearly every day (unlike say an English class when students are writing a research paper). Wouldn’t it benefit the students if the tutor were there every day?

In the first section of the evaluation, I mentioned that students with absence problems were chosen to be involvled in the program. I’ve wondered whether this is beneficial since the math tutor is only at the school two days a week, which is why I asked the second question. What if those students are always absent those two days? What benefit are they getting from the program?

You can’t very well evaluate a tutoring program without measuring the success of the students involved. Therefore, I wanted to see whether the students were improving or staying at the same grade (hopefully their grade is not decreasing) while being enrolled in the tutoring program.

As I read Chapter 2 in Reframing Evaluation Through Appreciative Inquiry this week and all the various posts by my peers, the similarities and difference between AI and evaluation became much more clear. However, I did notice that there really wasn’t a clear distinction between the two because they both have a lot of similar characteristics. As I mentioned in my discussion posting, it was hard for me to actually decide what type of evaluation I was looking at because evaluation can be completed in so many different ways. I know that one of the major focuses in this course is looking at AI, but it would be really interesting to compare and contrast all the types of evaluation. I realize that we probably wouldn’t have enough weeks in the semester, but isn’t AI just another form of evaluation? What makes it special compared to another type of evaluation?

Overall this week, I learned that the type of evaluation method chosen definitely depends on the needs of the client or stakeholders being evaluated. Not all evalaution methods will work in every situation. Therefore, it’s very important to look at all the characteristics of each evaluation method so that the right one is chosen. In addition, it could be that two different methods are needed for an evaluation. The great thing about AI is that it lends itself well to a variety of situations.

I really liked this week’s assignment. It was really different from any that I’ve had in the CTER program so far. I liked the fact that we were able to go out and pick our own articles to read rather than be told to read a given article and write about it. I feel like sometimes we all end up saying similar things about the articles given to us, but this week allowed us all to write completely different things because we weren’t all reading the same thing. On a somewhat different note, I felt like I spent a lot of extra time looking for and reading various articles that I haven’t had to do in the past. However, all of the reading really gave me a better idea of what AI is and how it can be used in varying degrees.

I really enjoyed reading Scott’s annotated bibliography of “Using appreciative inquiry to evaluate project activities of a nongovernmental organization supporting victims of trauma in Sri Lanka.” It showed me that AI can be used in evaluating all areas of life. We mostly talk about evaluating school-based programs, but other organizations can also use AI effectively.

I found Ming’s article, “Building on success: Transforming organizations through appreciative inquiry,” also interesting. I liked how they used the basic AI approaches, but changed certain things in order to make it work for their evaluation. For instance, they used a group interview process instead of an individual interview because of the large quantities of people involved. I think it’s great that the positive approaches of AI can be modified and still work for all types of situations.

Finally, Marsha did a great job with her annotated bibliography of “Top ten secrets of effective e-learning.” This article especially intrigued me because we are all “e-learning” through CTER right now. I think that the ten secrets were very accurate, but I have to wonder if the author would have gotten the same results if she would have used a different approach than AI.