More About Evaluation and Evaluators

Evaluation in the NISE Network

The NISE Network Evaluation group is a multi-organizational team dedicated to informing and improving the work of the Network by conducting evaluation studies that closely examine the public, professional, and field-wide impacts of NISE Net activities. For the first five years of the Network, group members worked closely with nearly all NISE Network core groups and teams. In each area, evaluators worked with the Network teams, conducting studies that encouraged professional inquiry and provided constructive feedback. For the second five years of the Network, the Evaluation group focused on conducting summative evaluations to understand the impact of the NISE Network. This has included studies on the products that have the greatest impact on the public, such as the Nano Mini-exhibit and NanoDays, as well as a three-year study looking at the impact of the Network as a whole on the informal science education professionals who are involved. In addition, the Evaluation group has continued helping other Network teams to gather data to inform their own work through a process called Team-Based Inquiry. Over the course of the NISE Net, over 250 front-end, formative, and summative evaluation studies have been conducted about NISE Net products and practices. 

To document some of the challenges we faced and lessons we learned about evaluating one of the largest informal education networks ever formed, we created a reflection document that can be found here:

We also documented how our team used an Evaluation Committee of Visitors (COV), or external review panel, to support and oversee the internal Evaluation workgroup. Our reflections on evaluation advisory committees can be found in this one-pager: 

Group of evaluators and large EVALUATE sign

Who are the Evaluators?

Different projects have included staff members from the following organizations.
If you have any questions or want further information about the evaluation and research, please contact Liz Kollmann (ekollmann@mos.org) or Marta Beyer (mbeyer@mos.org).

Space and Earth Informal Science Education (SEISE) project (2016-2020)

  • Museum of Science, Boston
  • Science Museum of Minnesota
  • External committee of Visitors: Gina Svarovsky, University of Notre Dame

Building with Biology - Multi-Site Public Engagement with Science - Synthetic Biology project  (2018) Evaluators in action

  • Museum of Science, Boston
  • Science Museum of Minnesota
  • Rockman et al

Nanoscale Informal Science Education Network project (2005-2017)

Over the course of the NSF-funded Nanoscale Informal Science Education Network project, the evaluation team has included staff members from the following organizations:

  • 2005-2010 (Years 1-5) evaluators: Museum of Science, Boston; Science Museum of Minnesota; Oregon Museum of Science and Industry; Exploratorium; Inverness Research Associates; and Multimedia Research
  • 2011-2017 (Years 6-12) evaluators: Museum of Science, Boston; Science Museum of Minnesota; Oregon Museum of Science and Industry
  • Committee of Visitors (COV): In  addition, the Nanoscale Informal Science Education Network summative evaluation work during 2011-2017 was overseen by an external Committee of Visitors that included: Frances Lawrenz, University of Minnesota; Bruce Lewenstein, Cornell University; Saul Rockman, Saul Rockman et al, and Carol Weiss, Harvard University.

Surveys and Reports

 

What are different types of Evaluation and Research Studies?

Evaluation is a process used to improve the design and function of educational experiences in informal learning environments and to measure progress toward goals. Research studies are designed to generate findings that will inform the broader field of informal science education.

  • Front-End Evaluation
    During the early stages of a project, front-end evaluation is used to learn about visitors’ familiarity with a topic, their interests and feelings, and what understandings they will bring to the experience.
  • Formative Evaluation
    During the design and development stage of a project, formative evaluation is used to help achieve desired goals; formative evaluation involves iterative testing and modification of prototype exhibits, materials, and programs.
  • Summative Evaluation
    At the end of a project, summative evaluation is used to assess the impact of a completed project; summative evaluation usually includes observing visitors while they experience a program or exhibit and interviewing or surveying them before and/or after this experience.
  • Research Reports
    Research reports summarize the results of research studies conducted on the impacts of informal science education efforts on both public audiences and museum professionals.
     
  • Evaluation Tools
    Capacity building tools for informal science educators to empower themto get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The Team Based Inquiry (TBI) process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.

Summative Evaluation Studies

 

Summative Evaluation icon

Summative evaluation typically occurs at the end of a project to assess the impact of the completed project. This process usually includes observing visitors while they experience a program or exhibit and interviewing or surveying them before and/or after this experience.The NISE Network Evaluation Team has conducted several summative studies investigating issues related to public and professional impacts across the Network.

Browse Summative Evaluation Reports

Below is a list highlighting a few of the NISE Net summative evaluation reports.

Front-End Evaluation

 

Front-End Evaluation icon

During the early stages of a project, front-end evaluation is used to learn about visitors’ familiarity with a topic, their interests and feelings, and what understandings they bring to an experience.

Browse Front-End Evaluation Evaluation Reports

 

Examples of front-end studies for NISE Net have included reviews of existing projects about nanoscience and a study investigating public awareness, interest, knowledge, and attitudes related to nanotechnology. Links to these studies are below:

Formative Evaluation

 Formative Evaluation icon​

During the design and development stage of a project, formative evaluation is used to help achieve desired goals; formative evaluation involves iterative testing and modification of prototype exhibits, materials, and programs.

Browse Formative Evaluation Evaluation Reports

 

In the first five years of the Network, all formative evaluations were conducted by members of the Evaluation Team and included several rounds of testing of exhibit prototypes, program work such as forums and stage presentations, along with interactive media products. In the second five years of the Network, formative evaluations were conducted by NISE Net practitioners through a Team-Based Inquiry process.

Below are examples of formative studies that were conducted by the NISE Net Evaluation group in the first five years of the network.

  • Who Decides Forum is a formative evaluation that looked at ways to improve visitor enjoyment, engagement and perceived learning at a discussion forum that was held at five institutions.
  • Intro to Nanotechnology is a formative evaluation that explored the effectiveness of an initial set of NISE Net prototype exhibits in 2006.
  • Exploring Measurement-Stretchability is a formative evaluation that looked at children and adults’ engagement and comprehension of a nano activity focused on the nanoscale.
  • Treating Tumors with Gold is a formative evaluation that gathered visitors’ feedback to a stage presentation on research using nanoshells and infrared light in cancer therapy.

 

Evaluation Tools 

Capacity building tools for informal science educators to empower themto get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The Team Based Inquiry (TBI) process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.

Browse Evaluation Tools

 

Team-Based Inquiry - evaluation capacity-building

Team-Based Inquiry cycle

Team-Based Inquiry (TBI) is a practical approach to empowering education professionals to get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The TBI process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.

  • The Team-Based Inquiry Guide explains each step of the TBI process and features ways TBI is used in the NISE Network to improve educational experiences and professional practice. 
  • The Team-Based Inquiry Training Videos were designed to support museum practitioners in learning about and using TBI in their own institutions by describing this process and providing examples of how a team might work through the different steps. 
  • NISE Network Program Evaluation Tools includes guidelines and templates to facilitate program evaluation. 

More evaluation tools are available from Center for Advancement of Informal Science Education (CAISE) available at https://www.informalscience.org

Research

Research reports summarize the results of research studies conducted on the impacts of informal science education efforts on both public audiences and museum professionals.

Research Report iconThe NISE Network has conducted several multi-year research projects looking at different aspects of the Network that have the potential to inform the field of informal science education. Whereas the evaluation studies conducted by the NISE Network measure progress toward NISE Net goals, the research studies are designed to generate findings that will inform and advance the broader field of informal science education. Below is a description of these projects. For more details, contact members of the research team.

 

 

​

Learn more about NISE Network Research Studies including

  • 1) Museum-Scientist Partnerships
  • 2) Organizational Change
  • 3) Public Learning and Decision Making
  • 4) Tracking NISE Net’s Real World Impact

 

 

Links

Evaluation and Research main page

 

Newsletter

Learn about upcoming opportunities, events, ways to grow professionally, and inspiring stories from network partners.

Subscribe