As we shared while we were making our preparations for National Chemistry Week in October, the ChemAttitudes project has moved into our next phase, in which we will select and revise a subset of our pilot activities into a dissemination kit that will be distributed to 250 sites and made available for free digital download. The pilot activities were developed in the first phase of the project, using a process called "design-based research" (DBR). In that phase, our activity developers worked closely with partnering project evaluators to develop, test, reflect upon, and iteratively revise the activities so that they achieve our desired learning outcomes. First, a team of activity developers worked with one another and with our partnering researchers to develop a suite of activities targeting increases in visitors' sense of interest, relevance, and/or self-efficacy. Next, the developer/researcher pairs identified research questions for testing and exploration and testing their activities on the floor, gathering data to answer their questions through observations, capturing video, and administering surveys to visitors. Then the researchers and developers reflected together on what transpires before revising the materials further and testing the activities again on the floor.
To convey a sense of what it's been like to with regard to chemistry, we thought it would be great to get some reflections from some of our ChemAttitudes activity developers about what they've taken from the process thus far.
Angela Damery and Becky Smick from the Exhibit Hall Interpretation department at the Museum of Science observed that "the highlight of Design-Based Research for us has been that it drives rapid iteration. We often have ideas of ways to improve programs, but time pressures and other priorities get in the way of following through in a timeframe that allows for a coherent picture of the impact of the changes. While keeping up with the evaluation schedule has been a real challenge for us in working on this project, it has forced us to think creatively about our activities, identify improvements, make changes, and take time to reflect on how visitors respond to them. The opportunity to get feedback directly from visitors through our evaluators, rather than just working from our own observations of their responses, has also been an invaluable aspect of the process."
Emily Hostetler, an Education Associate in the Forum department at the MOS shared her perspectives on developing and revising three activities through the design-based research process: "Participating in a design-based research project can be quite satisfying. In the past, I have created hands-on activities for visitors and only had their immediate reactions or post-activity conversations with their peers to judge what they learned or liked about the activity. With a DBR approach, I had a chance to take a risk with the content and format of the activities because I knew I would have insight from visitors on what went really well, or what missed the target. For example, I created one activity based on “survival chemistry” where the visitors had to pretend they only had a small supply of chemicals and had to survive in an extreme environment using simple chemical reactions. It seemed like a fun and interesting way to get visitors to find relevance in everyday objects that have chemistry, however most visitors did not see the relevance of being stranded somewhere with chemicals. Fair point. So, based on the data collected, I changed the activity to something more relevant to many of our visitors – helping a friend accomplish a task using chemistry. With this change, almost every visitor found the activity relevant to their lives.
I was beyond thankful and surprised at the number of family groups who were willing to support our research by answering survey questions after going through a (sometimes lengthy) activity interaction. We even had one family group with children who did not speak English and their parents translated every survey question and response for our researchers! It is because of these visitors that we are able to create interesting, relevant, and engaging activities for visitors at museums across the U.S."
Thor Carlson, an exhibit developer at the Science Museum of Minnesota shared his reflections both on developing materials focused on the topic of chemistry and the iterative development process of design-based research:
"It happens almost every time. A pack of curious school kids or an eager family approaches the table. When I say “We’re going to do some chemistry today,” their eyes light up, more brightly the younger they are. These kids have heard of chemistry but don’t have much first-hand experience. They almost always go “all in” to participate. The questions flow naturally and quickly.
As presenters, we get to see it with our eyes. But we also get to quantify participants’ feelings with the help of our research team members who survey them after the experiments. Within days, we’re using their feedback to make adjustments to the activities based on those insights.
Getting to do this in an informal science environment, we’re lucky. We’re not surrounded by intimidating lab equipment or blackboards full of equations. Often we’re using tools these kids are familiar with: plastic cups, markers, popsicle sticks, chewing gum, a board game. But the content is there and real-life connections are made. Said one 8-year-old as she was walking away: 'I thought chemistry was science, but it’s everywhere.' I couldn’t have said it any better.""
While the ChemAttitudes project is specifically focused on informal learning about chemistry, we're excited that the design-based research process will help us more broadly as we think about the way that we develop and revise hands-on activities on a wide variety of topics.
Susan Heilman, an educator in our Live Presentations department says "It’s been really interesting to design an activity using DBR. Knowing that there was an iterative process in place it took some of the pressure off of the first-time implementation of a new activity. We knew that it was going to be not perfect the first time, and we liked that it was not perfect the first time so that we could more easily learn what would or would not work with that particular activity. I also think that informing the visitors that they were part of not only prototyping for a new Chemistry activity, but also prototyping of a relatively new way of doing prototyping (DBR in not so many words), that the visitors felt more of a part of the development process.
During the first iteration of the battery activity, some of the best conversations I had were when I asked the visitors for their input on what topics or information I should plan to have in the second iteration. Did they want to hear more about lithium ion batteries or more about storing solar energy. Those led to some of the most authentic conversations in that activity."
We're excited about the improvements that the DBR process is enabling for these activities, and can't wait to share the final activities with NISE Network partners in the next phase of the project!