Evaluation in the NISE Network
The NISE Network Evaluation group is a multi-organizational team dedicated to informing and improving the work of the Network by conducting evaluation studies that closely examine the public, professional, and field-wide impacts of NISE Net activities. For the first five years of the Network, group members worked closely with nearly all NISE Network core groups and teams. In each area, evaluators worked with the Network teams, conducting studies that encouraged professional inquiry and provided constructive feedback. For the second five years of the Network, the Evaluation group focused on conducting summative evaluations to understand the impact of the NISE Network. This has included studies on the products that have the greatest impact on the public, such as the Nano Mini-exhibit and NanoDays, as well as a three-year study looking at the impact of the Network as a whole on the informal science education professionals who are involved. In addition, the Evaluation group has continued helping other Network teams to gather data to inform their own work through a process called Team-Based Inquiry. Over the course of the NISE Net, over 250 front-end, formative, and summative evaluation studies have been conducted about NISE Net products and practices.
To document some of the challenges we faced and lessons we learned about evaluating one of the largest informal education networks ever formed, we created a reflection document that can be found here:
We also documented how our team used an Evaluation Committee of Visitors (COV), or external review panel, to support and oversee the internal Evaluation workgroup. Our reflections on evaluation advisory committees can be found in this one-pager:

Who are the Evaluators?
Different projects have included staff members from the following organizations.
If you have any questions or want further information about the evaluation and research, please contact Liz Kollmann (ekollmann@mos.org) or Marta Beyer (mbeyer@mos.org).
Space and Earth Informal Science Education (SEISE) project (2016-2020)
- Museum of Science, Boston
- Science Museum of Minnesota
- External committee of Visitors: Gina Svarovsky, University of Notre Dame
Building with Biology - Multi-Site Public Engagement with Science - Synthetic Biology project (2018)
- Museum of Science, Boston
- Science Museum of Minnesota
- Rockman et al
Nanoscale Informal Science Education Network project (2005-2017)
Over the course of the NSF-funded Nanoscale Informal Science Education Network project, the evaluation team has included staff members from the following organizations:
- 2005-2010 (Years 1-5) evaluators: Museum of Science, Boston; Science Museum of Minnesota; Oregon Museum of Science and Industry; Exploratorium; Inverness Research Associates; and Multimedia Research
- 2011-2017 (Years 6-12) evaluators: Museum of Science, Boston; Science Museum of Minnesota; Oregon Museum of Science and Industry
- Committee of Visitors (COV): In addition, the Nanoscale Informal Science Education Network summative evaluation work during 2011-2017 was overseen by an external Committee of Visitors that included: Frances Lawrenz, University of Minnesota; Bruce Lewenstein, Cornell University; Saul Rockman, Saul Rockman et al, and Carol Weiss, Harvard University.
Surveys and Reports
- NISE Network's online surveys are powered by SurveyGizmo.
What are different types of Evaluation and Research Studies?
Evaluation is a process used to improve the design and function of educational experiences in informal learning environments and to measure progress toward goals. Research studies are designed to generate findings that will inform the broader field of informal science education.
- Front-End Evaluation
During the early stages of a project, front-end evaluation is used to learn about visitors’ familiarity with a topic, their interests and feelings, and what understandings they will bring to the experience.
- Formative Evaluation
During the design and development stage of a project, formative evaluation is used to help achieve desired goals; formative evaluation involves iterative testing and modification of prototype exhibits, materials, and programs.
- Summative Evaluation
At the end of a project, summative evaluation is used to assess the impact of a completed project; summative evaluation usually includes observing visitors while they experience a program or exhibit and interviewing or surveying them before and/or after this experience.
- Research Reports
Research reports summarize the results of research studies conducted on the impacts of informal science education efforts on both public audiences and museum professionals.
- Evaluation Tools
Capacity building tools for informal science educators to empower themto get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The Team Based Inquiry (TBI) process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.
Summative Evaluation Studies
Summative evaluation typically occurs at the end of a project to assess the impact of the completed project. This process usually includes observing visitors while they experience a program or exhibit and interviewing or surveying them before and/or after this experience.The NISE Network Evaluation Team has conducted several summative studies investigating issues related to public and professional impacts across the Network.
Browse Summative Evaluation Reports
Below is a list highlighting a few of the NISE Net summative evaluation reports.
- Summative Evaluation of the Sun, Earth, Universe Exhibition (2020): Summative evaluation report for the Sun, Earth, Universe exhibition examining the exhibition's reach and impact on the public.
-
Explore Science: Earth & Space Activity Toolkits Summative Study (2019): This summative evaluation study of the toolkits was designed to address and answer questions regarding the number and range of audiences reached by the project, visitor engagement, relevance, science identity, and their understanding around content areas.
-
Building with Biology Participant Impact Evaluation Report (2018): This report focuses on outcomes for public audiences who engaged in public events with hands-on activities and forum programs about synthetic biology as part of the Building with Biology project.
- Multi-Site Public Engagement with Science - Synthetic Biology Final Evaluation Report (2018): This report focuses on outcomes for the scientists and informal science educators involved as event hosts and/or facilitators in the Building with Biology project.
- NISE Net Museum & Community Partnerships Project Evaluation Report (2017): This study focused on NISE Net’s efforts to support collaborations between museums and local community organizations, investigating impacts on professional participants as well as the extent to which the project reached underserved audiences with relevant, educational, and engaging materials.
-
NISE Net Years 6‐10 Evaluation Summary Report (2017): The purpose of this report is to summarize the findings from the Years 6-10 (2010-2015) Nanoscale Informal Science Education Network (NISE Net) summative evaluation studies and to discuss factors that contributed to the achievement of NISE Net’s goals.
- NISE Network Professional Impacts Summative Evaluation (2016): This investigation is based on the NISE Network goals for professionals and explores how involvement with NISE Net impacts an individual professional’s sense of community, learning about nano, and use of nano educational products and practices.
- NISE Network Public Impacts of Nano-Rich Organizations Summative Evaluation (2016): This study was designed to explore how the NISE Network’s range of resources were used by the diverse range of Network partners, and how individual institutions felt that they were doing at providing their publics with positive experiences and learning outcomes.
- NISE Network Public Impacts Summative Evaluation (2012-2015): Several studies provide an overview of the Network's overall impacts on the public.
- Public Reach Estimations for the NISE Network (2015): The purpose of this document is to consolidate and archive all of the major public reach estimates. Over the life of the project, the NISE Network is estimated to have reached over 30 million people through NanoDays and the Nano exhibition.
- Summative Evaluation Study of NanoDays 2014 Events: In the spring of 2014, the Nanoscale Informal Science Education Network (NISE Net) Public Impacts evaluation team conducted a summative study of NanoDays, a nationwide festival of educational programs about nanoscale science, engineering, and technology.
- The Public Impacts Mini-Exhibition Study Year 8 Summative evaluation (2013) focused on the reach of the Nano mini-exhibition and whether or not the exhibit was successful in providing visitors with an engaging and educational experience. This study also analyzed the mini-exhibition’s success in different contexts and for different types of audiences, including Hispanic visitors and visitors with disabilities, and whether or not it was a catalyst for new public programming around nano.
- A Year 6 Network Communication Study was conducted to learn about how the Network's primary communication components (NanoDays, face-to-face meetings, the regional hub structure, and the nisenet.org website) are being used by actively involved partners.
- The Exhibits and Programs Year 5 Summative Evaluation studied the impact of a set of NISE Net exhibits and programs on museum visitors.
- The Year 5 Delivery and Public Reach Study examined nano educational activities that NISE Net partners are doing and estimated the public reach of the Network in 2010.
- Review of NISE Network Evaluation Findings: Years 1-5 is a review of the findings from over 240 evaluation reports from the first five years of the Network. The review is divided into six chapters:
- Connecting ISE Professionals with Nano Informal Science Education
- Connecting the University-Affiliated Individuals with Nano Informal Science Education
- Engaging the Public in Learning about Nano through NISE Network Educational Products
- Engaging the Public with Societal and Ethical Implications Content through NISE Network Products
- Making the Unfamiliar Interesting and Relevant for Museum Visitors
- Reaching Public Audiences
Front-End Evaluation
During the early stages of a project, front-end evaluation is used to learn about visitors’ familiarity with a topic, their interests and feelings, and what understandings they bring to an experience.
Browse Front-End Evaluation Evaluation Reports
Examples of front-end studies for NISE Net have included reviews of existing projects about nanoscience and a study investigating public awareness, interest, knowledge, and attitudes related to nanotechnology. Links to these studies are below:
- Nanotechnology and the Public Year 1 Front-End Evaluation is a 2005 literature review that summarized findings from secondary sources that describe public awareness, interest, knowledge and attitudes related to nanotechnology.
- Compilation of Nanoscale Exhibit Projects Year 1 Front-End Evaluation is a 2005 literature review of existing Nanoscale Communication Projects.
Formative Evaluation
During the design and development stage of a project, formative evaluation is used to help achieve desired goals; formative evaluation involves iterative testing and modification of prototype exhibits, materials, and programs.
Browse Formative Evaluation Evaluation Reports
In the first five years of the Network, all formative evaluations were conducted by members of the Evaluation Team and included several rounds of testing of exhibit prototypes, program work such as forums and stage presentations, along with interactive media products. In the second five years of the Network, formative evaluations were conducted by NISE Net practitioners through a Team-Based Inquiry process.
Below are examples of formative studies that were conducted by the NISE Net Evaluation group in the first five years of the network.
- Who Decides Forum is a formative evaluation that looked at ways to improve visitor enjoyment, engagement and perceived learning at a discussion forum that was held at five institutions.
- Intro to Nanotechnology is a formative evaluation that explored the effectiveness of an initial set of NISE Net prototype exhibits in 2006.
- Exploring Measurement-Stretchability is a formative evaluation that looked at children and adults’ engagement and comprehension of a nano activity focused on the nanoscale.
- Treating Tumors with Gold is a formative evaluation that gathered visitors’ feedback to a stage presentation on research using nanoshells and infrared light in cancer therapy.
Evaluation Tools
Capacity building tools for informal science educators to empower themto get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The Team Based Inquiry (TBI) process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.
Browse Evaluation Tools
Team-Based Inquiry - evaluation capacity-building
Team-Based Inquiry (TBI) is a practical approach to empowering education professionals to get the data they need, when they need it, to improve their products and practices and, ultimately, more effectively engage public and professional audiences. The TBI process involves an ongoing cycle of inquiry: question, investigate, reflect, and improve.
- The Team-Based Inquiry Guide explains each step of the TBI process and features ways TBI is used in the NISE Network to improve educational experiences and professional practice.
- The Team-Based Inquiry Training Videos were designed to support museum practitioners in learning about and using TBI in their own institutions by describing this process and providing examples of how a team might work through the different steps.
- NISE Network Program Evaluation Tools includes guidelines and templates to facilitate program evaluation.
More evaluation tools are available from Center for Advancement of Informal Science Education (CAISE) available at https://www.informalscience.org
Research
Research reports summarize the results of research studies conducted on the impacts of informal science education efforts on both public audiences and museum professionals.
The NISE Network has conducted several multi-year research projects looking at different aspects of the Network that have the potential to inform the field of informal science education. Whereas the evaluation studies conducted by the NISE Network measure progress toward NISE Net goals, the research studies are designed to generate findings that will inform and advance the broader field of informal science education. Below is a description of these projects. For more details, contact members of the research team.
Learn more about NISE Network Research Studies including
- 1) Museum-Scientist Partnerships
- 2) Organizational Change
- 3) Public Learning and Decision Making
- 4) Tracking NISE Net’s Real World Impact
Links
Evaluation and Research main page
- Evaluation Tools
- Summative Evaluation Reports
- Research Studies
- Front End Evaluation Reports List
- Formative Evaluation Reports List
- More about Evaluation and Evaluators
- Browse Evaluation and Research Reports List