Category: Research

 

Abstract

ASAMI—Afterschool Science and Math Integration—integrates skills of mathematics with interesting concepts and hands-on activities in astronomy-based science in the middle school. Common Core Mathematics Standards and Next Generation Science Standards (NGSS) are used as ASAMI effectively teaches algebra standards/concepts with Hands-On Universe (HOU) curricula to engage 12–14-year-old English Language Learners (ELLs). In our 2014–15 school year pilot and field tests of ASAMI, students classified as ELL met twice a week for a total of four hours a week, at a middle school in California, USA. The evaluation of ASAMI shows that these learners improved their test scores on Common Core Mathematics Standards items [Gain = (post-test−pre-test)/pre-test] by 46 percent in our first six-week trial and by about 93 percent in our second semester in the school year. Four other pilots resulted in similar gains. The main algebraic focus and assessment items focused on ratios, proportion, and linear equations, which are used throughout the curriculum of the HOU. Our assessments show that ASAMI is a very effective tool to help focus instruction, and they demonstrate success in learning through the integration of math and science.   While the desire for integrated math and science curricula has been expressed for decades, few quantitative studies of achievement gains have surfaced (Czeniak et al. 1999).

Background and Introduction

Hands-On Universe

Afterschool Science and Math Integration (ASAMI) is based on Hands-On Universe (HOU) astronomy activities that are often computer/technology based.   HOU was based for many years at the Lawrence Hall of Science (LHS) at the University of California, Berkeley, and developed significantly within the Hall. Alan Friedman’s leadership at LHS in astronomy education help build the discipline of “Hands-On” astronomy. HOU has many linkages directly traceable to Alan, and the appendix describes the heritage of HOU through Alan.

Over its almost twenty-five years of activities, HOU has brought the wonder and the data of the Universe into classrooms all around the world. Approximately one thousand American teachers have been in HOU teacher workshops. Through the Galileo Teacher Training Program (GTTP), approximately 20,000 teachers in 100 nations around the world have ben in HOU workshops.   Formal external evaluations submitted to the U.S. National Science Foundation have usually demonstrated that HOU changed students’ attitudes positively towards STEM careers and helped students appreciate math, science, and technology. In HOU students measure objects on and off the computer and make models of celestial systems. We currently plan to start a new round of United States HOU Teacher workshops and are actively seeking funding. ASAMI is the most recent version of HOU. It uses HOU’s images, software, activities, and methods, adopted for ELL middle school students.

Program Goals

One goal of ASAMI is that students master enough math so that they can explore careers in STEM fields. Our pre-tests of the ELL students demonstrated that these students were lacking important skills and would have grave difficulties pursuing STEM careers. All citizens of the world are now facing major technological and scientific challenges. Every student needs to become an active, well-informed and educated citizen. The ELL students in our study required some remedial interventions in their education to succeed in the disciplines of math and science.   We wanted these students to to engage in and appreciate math and science, using hands-on, HOU-inspired activities, both on and off the computer.

NGSS Middle School Topics

The Next Generation Science Standards (NGSS) recommend that science education in grades K–12 be built around three major dimensions: scientific and engineering practices; crosscutting concepts that unify the study of science and engineering through their common application across fields; and core ideas in the major disciplines of natural science.   The Framework for K-12 Science Education (Quinn et al. 2012) also identifies seven crosscutting concepts that bridge disciplinary boundaries, uniting core ideas throughout the fields of science and engineering.   Among the seven crosscutting concepts presented in Chapter 4 of the Framework is the following: “Scale, proportion, and quantity. In considering phenomena, it is critical to recognize what is relevant at different measures of size, time, and energy and to recognize how changes in scale, proportion, or quantity affect a system’s structure or performance.”

The first three standards of Middle School – Earth Science Standards of NGSS (NGSS, 2013) support well our objectives in ASAMI:

(1)     MS-ESS1-1.     Develop and use a model of the Earth-sun-moon system to describe the cyclic patterns of lunar phases, eclipses of the sun and moon, and seasons.  [Clarification Statement: Examples of models can be physical, graphical, or conceptual.]

(2)     MS-ESS1-2.      Develop and use a model to describe the role of gravity in the motions within galaxies and the solar system.  [Clarification Statement:  Emphasis for the model is on gravity as the force that holds together the solar system and Milky Way galaxy and controls orbital motions within them. Examples of models can be physical (such as the analogy of distance along a football field or computer visualizations of elliptical orbits) or conceptual (such as mathematical proportions relative to the size of familiar objects such as their school or state).] [Assessment Boundary: Assessment does not include Kepler’s Laws of orbital motion or the apparent retrograde motion of the planets as viewed from Earth.]

(3)     MS-ESS1-3.      Analyze and interpret data to determine scale properties of objects in the solar system.  [Clarification Statement: Emphasis is on the analysis of data from Earth-based instruments, space-based telescopes, and spacecraft to determine similarities and differences among solar system objects. Examples of scale properties include the sizes of an object’s layers (such as crust and atmosphere), surface features (such as volcanoes), and orbital radius. Examples of data include statistical information, drawings and photographs, and models.] [Assessment Boundary: Assessment does not include recalling facts about properties of the planets and other solar system bodies.]

Such topics in the NGSS were included in ASAMI and were found in all of the activities. (See Appendix 2.)

Common Core Seventh- and Eighth-Grade Math

The NGSS clearly require the inclusion of the mathematical concepts of scale and proportion. Meanwhile the State of California has also adopted the Common Core Mathematics Standards,which include, for grade seven: “Analyze proportional relationships and use them to solve real-world and mathematical problems,” and for grade eight: “Understand the connection between proportional 
relationships, lines, and linear equations.” Many middle school students have had difficulty in understanding these concepts. The Trends in International Mathematics and Science Study (TIMSS) reports: “Students also found the proportionality items difficult. For example, one of the least difficult problems in this area asked about adding 5 girls and 5 boys to a class that was three-fifths girls. On average, fewer than two-thirds of the students across countries correctly answered that there would still be more girls than boys in the class” (Beaton 1996). Such students are subsequently unable to achieve mastery of algebra, the gatekeeper to more advanced mathematical and scientific courses. Research referenced in this article shows that an integrated curriculum provides opportunities for more relevant, less fragmented, and more stimulating experiences for learners.

Target Audience

ASAMI had its first pilot study done at a diverse middle school in El Cerrito, CA, during 2012–2013.  Then the leaders of ASAMI identified three middle schools in Hayward, CA, as appropriate schools for collecting research data about its effectiveness.   The principals of these schools wanted ASAMI to serve their many students who are English Language Learners.  Table 1 below indicates that ELLs are a significant segment of learners in California overall and in Hayward in particular. Our pre-tests indicate this population is very challenged to master the standards of Common Core Mathematics.

To meet the needs of the English Learners, the ASAMI program included several tutors who are bilingual in English and Spanish.  Although the lessons were taught in English, the tutors were always available to help the English Language Learners to understand the assignments and to feel accepted.  Here are data from Ed-Data of California from the year 2013–2014:

Table 1. Demographics of Schools in Target Area

School or Educational System	Hispanic or Latino Students	English Language Learners	Free or Reduced Price Meals
California	53.3%	22.7%	59.4%
Hayward Unified School District	61.1%	30.9%	70.5%
Winton Middle School	76.9%	22.2%	78.9%
Bret Harte Middle School	51.1%	10.1%	67.8%
Cesar Chavez Middle School	67.3%	27.8%	81.1%

 

The ASAMI program provides all of the hands-on materials and often sends the students home with items they constructed.  Leaders at the schools help greatly by recruiting the students, monitoring their attendance, and phoning the parents of absentees.  From interviews (to be published), it was very clear that parents want their children to succeed in STEM and are eager to cooperate with this after-school program.  Our interviews indicate that many English Language Learners struggle to learn a new language and simultaneously keep up with the pace of study in the classroom.

M. Calderon (2007) has stated: “The Hispanic dropout rate is the highest in history.” We have observed that ELL students often become discouraged, fail to compete, and are ready to drop out of participation in school activities.  The ASAMI program is achieving a caring, enjoyable environment where the students are making progress.

Fry observed: “An analysis of recent data from standardized testing around the country shows that the fast growing number of students designated as English language learners (ELL) are among those farthest behind” (2007, i)  The ASAMI project has been used successfully to serve this needy population.   The faculty of ASAMI have endeavored to use the best practices (Rolstad et al. 2005; Short and Echevarria 2004) to serve these students. Many of the previous studies tend to focus on language acquisition. The ASAMI program adds the acquisition of science and math literature. Integrating inquiry-based science and language learning brings success to ELLs, according to Stoddart, who wrote: “The authors of this article take the alternate view that the integration of inquiry science and language acquisition enhances learning in both domains” (2002, 664).

ASAMI Activities

Table of Some ASAMI Activities

An exemplary list of ASAMI activities is shown in Appendix 2.  Each activity usually required one to two hours in an after-school session.

Modeling Pedagogy and Support of the NGSS Practice Matrix

ASAMI endeavors to implement at the middle-school level the Modeling Pedagogy, which is widely used in many high-school physics classes.   The lead ASAMI teacher, Jennifer Perazzo, uses these instructional strategies. Moreover, creating and evaluating models is a major goal of NGSS. The table in Appendix 3 shows examples of the use of models in the NGSS.

The website of the American Modeling Teachers Association explains: “Modeling Instruction . . . applies structured inquiry techniques to the teaching of basic skills and practices in mathematical modeling [and] proportional reasoning” (http://modelinginstruction.org). Modeling Instruction has proven to be one of the most reliable pedagogies to improve student learning In the Modeling Instruction pedagogical approach, students work in groups of three. They voice their preconceptions, collect experimental data, build a model in their small groups, and document their ideas on whiteboards. Then the students assemble with their classmates for a “board meeting” to present their work and develop a class consensus model.

An example of how we implemented the model in ASAMI is shown in the diagram below.

ASAMI Assessments of Common Core Math

The first goal of the evaluation was to assess the effects of students’ participation in ASAMI on their understanding of proportional reasoning. To measure these outcomes, evaluators developed pre- and post-program content tests and surveys. Math assessments only were developed and implemented. The content tests contained five proportional reasoning items taken from four sources: (1) the California STAR test database, (2) the National Assessment of Educational Progress (NAEP) item database; (3) the New England Common Assessment Program; and (4) the Silicon Valley Mathematics Initiative’s Mathematics Assessment Collaborative project.

An exemplary assessment item is shown below.

The lead teacher and main content developer had not studied the assessments and was unaware of the detailed questions. Her focus was to develop and teach activities that were hands-on activities emphasizing Common Core math principles and tools.

Results of Assessments

Test Scores

We deployed our five assessment items in pre- and post-test sessions at the beginning and end of ASAMI. At Portola Middle School, only interviews were undertaken. All of the Common Core Math assessments were administered in the school years 2013–2014 and 2014–2015. While these assessments are viewed as a preliminary study, it is clear there was a gain in students’ capabilities. Before starting ASAMI, students’ skills were very low. Every group of ASAMI students had test scores that improved significantly beyond the control group’s gains. In summary, students had about double the learning gain, compared to a preliminary control class. Hence, we view the ~2X more learning as a lower limit, compared to traditional learning.

The number of students assessed was typically about twenty per class, and the standard deviations were usually around one point, When we combine the data, the results become much more significant, with the summed results approaching significance at greater (1/sqrt(4)), at a 4 sigma significance.  These results are very encouraging.

It is interesting to note that the eighth-grade ASAMI students, who had undergone normal math education for most of a year, had pre-test scores similar to those of entering ASAMI seventh graders.  These incoming eighth graders had learned little in the year and a half of math education since their entrance into middle school.

Student Interviews and Informal Observations

Interviews and observations were done at Portola Middle School, with parental consent and student assent forms per the UC Berkeley Committee for the Protection of Human Subjects Protocol # 2012-03-4125. These data suggest that students found the ASAMI activities to be highly engaging and quite different from typical classroom practices. Students worked diligently in groups on complex math and science problems, persisting on new and challenging tasks with the help of their ASAMI leaders. During one session, for instance, evaluators observed students using Salsa J software to calculate astronomical distances. A group of four students sat or stood in front of a computer, with one student running the program and the others providing guidance. The students were so engaged in the activity that they wanted only a brief snack break before returning to their work.

The root of ASAMI’s appeal may be in its “useful application” approach to mathematics. Rather than teaching proportional reasoning as an abstract skill, ASAMI embeds it into science problems that pique students’ interest. In fact, one student described the program as “an astronomy program which sneaks in math,” noting that she often didn’t “realize how much [math] you’re doing” until later. It was only in the hours after ASAMI that she felt the full mental impact of what she had done: “My brain’s tired. I’ve done too much math.”

Another student also praised the ASAMI’s activities, calling them “Math in a fun way. You don’t know you’re doing math but you are,” she said. “I liked how they put the math. They didn’t just give you like a paper with math problems and say do this. It was in a way where it was math but it wasn’t just math, it was something else like astronomy.”  This same student commented that ASAMI was a very different from her regular math classes: “Most of the time now in school the teacher’s on the whiteboard, we do problems, we do our homework and our work, but it’s nothing like this, with measuring, with astronomy, with ratios, you know, it’s not like how they put it.” Before ASAMI, she didn’t think that mathematics had much to do with science. “I didn’t really think I needed science to do math. I just thought science was science and math was math and they were two different things.” Now that she has been through the program, she wishes that all students could have the same experience. “By them [math and science] being joined together it makes it more interesting and more fun because you’re not just doing math and you’re not just doing science, but you’re doing both of them at once.”

General Observations and Success Factors

We believe there are several reasons why ASAMI has worked well.

• Individual Tutoring

We employed two or three Spanish-speaking high school and community college students in the ASAMI sessions. Hence, ASAMI participants received a lot of individual tutoring, and with the help of their own peer groups, were somehow convinced to undertake rigorous work and struggle with Common Core topics.

• Fun and Exciting Activities

Math was always fun and often had instant consequences/feedback if you got things wrong. For example, in the playdoh recipe scaling activity, at least half of the students got the ratios wrong (many subtracted instead of using ratios!) and they made playdoh with much too liquid a consistency.   There was always fun and excitement in the hands-on activities, and we could keep them both involved and working rigorously, competing against other after-school activities. Students, when asked if this work was more fun and interesting than their normal math classes would give staff a condescending look and say “Duh…”

• Parent and Community Support

We had great support from the parents. The leader of ASAMI community relations, Mr. Jesus Heredia, continually cultivated a strong relationship with the parents. The parents wanted ASAMI for their children, and if children did not attend the ASAMI sessions, the parents were informed, and usually the students came back. For these reasons, there was very low attrition in the student population (<12%). ASAMI was observed by staff to be desired by the parents as it promoted Common Core learning with an emphasis on technology, college, and jobs.

• Strong Support from our Hosting Schools

Winton Middle School and Bret Harte Middle School provided superb hosting of our system. We had support from the administrators and from the after-school programs (Youth Enrichment Program), and custodial staff.

• Strong Support from the School English Learner Advisory Committee (ELAC)

We undertook very careful communications and briefing with the ELAC, especially at Winton Middle School; they were convinced of ASAMI’s value, and they felt that ASAMI was their program.

• Strong Support from the Hayward Unified School District (HUSD) Office and School Board

ASAMI benefitted from great support from the HUSD central office. The whole development of our program, the funding systems, the invoicing and multiple layers of approval (including School Board approval) were all undertaken with vigor and enthusiasm by District staff.

• Undying Dedication to Rigor and Common Core Math in Every Instance!

We did not have to dig deeply to find how proportions and ratios are used in our science problems, so we could both emphasize Common Core and complete these activities. For example, students learned in HOU that proportion and scale are used widely in the Universe and that, in fact, the Universe makes no sense without proportion and scale.

• Comments from an ASAMI Teacher

One new instructor, Mr. Ben-Shalom, writes of ASAMI: “At first I was skeptical that struggling students would want to participate in yet more academics during their after-school time, and yet this program has amazed me. ASAMI will not work for everyone, but those students who it has reached have shown a kind of dedication and enthusiasm about math and science that I thought not possible. And this is due to ASAMI’s solid repertoire of lessons and activities that are engaging and will help these students succeed.”

Future Work

We are confident of our test score gains and students’ indications of excitement about STEM topics. Future work (proposals are in the planning stages) will include a deeper study of these results and a more thorough explication of the success factors. As one local collaborator noted: “The ASAMI initiative has snowballed through the science department and inspired more student-centered and hands-on activities, generally.” We will endeavor to spread ASAMI throughout the Hayward Unified School District and then beyond into other California schools, many of which are blessed with students and families eager to master the Common Core STEM topics and need some extra help from ASAMI as their language acquisition and skills develop.

About the Authors

Kristin M. Bass, Ph.D., is a Senior Researcher at Rockman et al, a San Francisco-based external evaluation company. Kristin’s areas of expertise include assessment development and validation, program fidelity, research design, and quantitative analysis. At Rockman, she primarily directs projects related to formal and informal STEM education. Kristin has a B.A. in psychology from Yale University and a Ph.D. in education and psychology from the University of Michigan.

Gabriel Ben-Shalom is a recently graduated teacher, who finished his student teaching with Ms. Lobo at Winton Middle School and became available to teach ASAMI for eighth-grade students.  Gabriel benefitted in his own education from hands-on and conceptually deep activities, and he was eager to be involved in ASAMI, particularly as he witnessed U.S. Science and Math education move into an era of Common Core and NGSS.  He was very delighted when he found the students tackling hard problems and making progress in their own learning.  In fact, as we note in the paper, the eighth-grade students in Gabriel’s class had very large gains on the math Common Core assessment items, which is a tribute to his teaching skills.

Jesus Herdia is an English Language Learner (ELL) Specialist at Winton Middle School in Hayward, CA. He was formerly a teacher, but moved into ELL work when he saw the tremendous potential of these students, coupled with their strong need for activities that engaged and supported their core learning.  Hence, ASAMI spoke naturally to his sense of what the students needed.  Jesus was diligent in working with the families of the students, and through his efforts, we saw very low attrition in the ASAMI classes.  Jesus helped convince the English Language Advisory Committee that ASAMI was in their children’s best interest.   Jesus also played an essential role in the total running and management of ASAMI and was in the ASAMI classroom almost continuously.

Rainbow Lobo is a teacher in the Science Department at Winton Middle School in the Hayward Unified School District.  She teaches science and technology and has been an advocate of hands-on, student-centered learning for most of her career.  Students in her technology elective class demonstrated large gains in their grades after a year of Lobo’s class. She provided ASAMI’s home (her classroom), ideas on classroom management, and continuous input and ideas in this study.

Carl Pennypacker is a physicist and educator who has been fortunate to play pivotal roles in some decent projects. He received his B.A. from UC Berkeley in 1972, with the group of Luis Alvarez. Together with Richard Muller, Pennypacker has helped form and develop many of the central ideas that have led to the discovery of Dark Energy. He and his team were winners of the Gruber Prize and the Breakthrough Prize for this work, and the student he co-advised, Saul Perlmutter, went on to accrete the Nobel Prize for this work. Pennypacker helped co-found, with a group of great teachers and educators, the Hands-On Universe project. This project has led to the training of 1000 teachers in the United States, and about 20,000 around the world, and is part of the French National Curriculum and the Bavarian State curriculum.

Jenifer Perazzo is a Hands-On Universe Teacher Lead. She is also a certified Modeling Instruction teacher. During the school year she is a Science Specialist for an elementary school in Pleasanton, CA. She introduces students and teachers to the EU-HOU astronomical image analysis tool, Salsa J, a software program dedicated to image handling and analysis in the classroom. Jenifer created and taught most of the ASAMI activities for the seventh-grade class.

David R. Stronck is a Professor in the Department of Teacher Education, California State University, East Bay. Oregon State University awarded him an M.S. in Biological Sciences and a Ph.D. in Science Education. He is the sole author of 22 articles reporting statistical research in major journals of learned societies.   He has a total of more than 200 publications, including eight books. For ten years, he was the editor of journals for science teachers. Stronck has been the director of projects that have been funded at more than $3 million.   He has directed or co-directed 15 grants for the National Science Foundation. The Genentech Foundation for Biomedical Sciences funded his projects serving high-school students, for more than one million dollars. He has also directed four grants from the U.S. Dept. of Education. He presents at an average of five different conferences annually, e.g., the National Science Teachers Association.

References

Beaton, A.E. 1996. Mathematics Achievement in the Middle School Years: International Association for the Evaluation of Educational Achievement’s Third International Mathematics and Science Study (TIMSS). Chestnut Hill, MA: TIMSS International Study Center, Boston College.     http://eric.ed.gov/?id=ED406419 (accessed June 24, 2015).

Calderon, M. 2007. Teaching Reading to English Language Learners, Grades 6-12: A Framework for Improving Achievement in the Content Areas. Thousand Oaks, CA: Corwin.

Committee on Guidance on Implementing the NGSS. 2015. Guide to Implementing the Next Generation Science Standards. Washington, DC: The National Academies Press.

Czemiak, C.M., W.B. Weber, A. Sandmann, and J. Ahern. 1999. “A Literature Review of Science and Mathematics Integration.” School Science and Mathematics 99 (8): 421–430. http://onlinelibrary.wiley.com/doi/10.1111/j.1949-8594.1999.tb17504.x/epdf.

Fry, R. 2007. How Far behind in Math and Reading Are English Language Learners? Report. Washington, DC: Pew Hispanic Center. http://eric.ed.gov/?id=ED509863 (accessed June 24, 2015).

National Governors Association Center for Best Practices and the Council of Chief State School Offices. 2010. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Offices.

NGSS Lead States. 2013. Next Generation Science Standards.   Washington, DC: The National Academies Press.

Quinn, H., H. Schweingruber, and T. Keller, eds. 2012. A Framework for K-12 Science Education. Washington, DC: The National Academies Press.

Rolstad, K., K. Mahoney, and G.V. Glass. 2005. “The Big Picture: A Meta-Analysis of Program Effectiveness Research on English Language Learners.”   Educational Policy 19 (4): 572–584. http://epx.sagepub.com/content/19/4/572.short (accessed June 24, 2015).

Short, D., and J. Echevarria. 2004. Teacher Skills to Support English Language Learners. Educational Practice Report 3. Santa Cruz, CA: Center for Research on Education, Diversity and Excellence.

Stoddart, T. 2002. “Integrating Inquiry, Science and Language Development for English Language Learners.” Journal of Research in Science Teaching 39 (8): 664–687.

Appendix 1: Alan Friedman and HOU

Alan Friedman established and directed the Lawrence Hall of Science Planetarium (University of California, Berkeley) in 1973. For over a decade his leadership set the legacy of audience participation planetarium shows and hands-on science at Lawrence Hall. He was a pioneer in the field and involved hundreds of planetariums through Participatory Oriented Planetarium (POP) workshops and the publishing of the Planetarium Educator’s Workshop Guide, which evolved into Planetarium Activities for Successful Shows (PASS; now at http://www.planetarium-activities.org/). To this day LHS helps bring that style of show into the digital age and encourages other digital planetariums to include live audience participation in their repertoire of shows, rather than just recorded programs. Among the planetarium shows Alan developed were Stonehenge and Finding Your Star (now Constellations Tonight), in which the presenter hands out star maps to all the audience members and teaches them how to use them. Using star maps was to become a favorite tool of HOU observers in the guise of Uncle Al’s Hands-On Universe Starwheels. Cary Sneider became Planetarium Director after Alan Friedman, and it was under Cary that the first connection with HOU was made in 1991. Cary had been invited to the seminal HOU organizing workshop but was unable to attend and asked Assistant Director Alan Gould to go in his stead. At the workshop, Alan presented an activity from one of the planetarium shows, Moons of the Solar System, in which the audience members kept track of the moons of Jupiter and discovered the relationship between the moons’ orbital periods and their orbital radii. That ultimately evolved into one of the favorite activities in the HOU high school curriculum. Years later, Alan Gould became Co-Director of HOU for a number of projects.

Appendix 2: Typical ASAMI Activities

ASAMI Activity	What Students Do	Math Common Core Concept
Derive a correct recipe and then make playdoh	Students scale from a recipe that requires too much of one ingredient	Ratios and proportion
Make a scale map of their school, from Google Maps	Use Google Maps and HOU image processing to measure true diameters of objects and measure their school, culminating in a scale map of some buildings, etc.	Ratios, proportion, scale, measurement
Make a scale solar system	Students take an existing playdoh recipe and scale it for the smaller amount of materials they are given	Ratios and proportion
Lunar Craters – find a lunar crater as big as your county from computer images	Students find a crater as big as their county, plot a map of the State of California on a moon map, use different map scales and compare maps.	Proportion and ratios
Asteroid Impact – drop various size stainless steel balls into birdseed on a tray	Students drop various mass spherical objects into bird seed (works better than flour) from various heights, and plot crater size versus height, mass, etc.	Energy, proportion, mass, etc.
Water Rockets	Build and launch, then measure and graph results from experiments with water rockets	Proportion

 

Appendix 3: Model Building in the NGSS

Practices Matrix from the NGSS (http://www.nextgenscience.org/next-generation-science-standards): Model-building in NGSS. The word “model” has been highlighted by the authors of this article.

 

 

 

Abstract

Restoration of forest ecosystems following the loss of biodiversity associated with non-native species invasions is an issue of civic consequence that has the potential to engage audiences of all ages, backgrounds, and abilities. In this project, the strong sense of community connection felt toward a local forest preserve was leveraged to inspire native plant seed collection, propagation, and planting for a community-driven forest restoration project. As part of a larger project, informal science education was integrated into a general education environmental science course to engage college students in this civic project and in intergenerational community building. The introduction of students to informal science education (ISE) through collaboration with an outdoor education center was successful at increasing awareness of ISE as a potential career path, developing environmental science content knowledge, inspiring interest in restoration projects among elder participants, and building community. Intergenerational workshops resulted in bidirectional knowledge exchange among participants related to a strong sense of place shared by both generations.

Background

In 2013, a partnership between a small liberal arts college and an environmental outdoor education center was funded through a SENCER-ISE II grant to infuse civic engagement into informal science learning and integrate informal science education into higher education science teaching. During the first year of the grant work, college students, middle-school students, senior adults, and partnership institutions became an intergenerational community of practice centered around the critical issue of biodiversity loss through species invasions. The overall project included multiple components: young students collecting seeds of native plants, college students cleaning and propagating plants and initiating restoration research, and older community members participating in civic engagement activities related to restoration. The focus of this article is on the incorporation of informal science education methods into a general education, first-year college environmental science course using intergenerational learning and civic engagement. The intention of this portion of the larger project was to enhance student learning and promote community building by involving senior adults and college students in an intergenerational learning experience. The project combines aspects of informal science education with intergenerational learning and civic engagement. The project was designed to strengthen the link between environmental science learning and action (Ballantyne et al. 1998) by engaging participants in a topic relevant to their lives and involving them in interactive learning (Falk 2001).

 

Introduction

Informal Science Education and Civic Engagement

“Experiences in informal environments for science learning are typically characterized as learner-motivated, guided by learner interests, voluntary, personal, ongoing, contextually relevant, collaborative, nonlinear, and open-ended” (National Research Council [NRC] 2009, 11). In formal venues, learning is compulsory, structured, and teacher-centered, with content more central than social aspects of learning (Wellington 1990). Non-formal learning, a process that fits between formal and informal learning, is more structured but is more easily adaptable than formal education (Eshach 2007). The numerous definitions of informal, non-formal, and formal learning were recently reviewed by Stocklmayer et al. (2010). In this study, informal learning is understood as taking place outside of the classroom; it is learner-centered, includes both academic and social aspects of importance and, although it is not entirely unstructured, it relies to some degree on the learner’s intrinsic motivations (Wellington 1990; Malcolm et al. 2003; Martin 2004). Research in teaching and learning in informal settings shows that, among other benefits, informal science education (ISE) is effective in increasing interest and engagement in science and increasing general scientific literacy, (Bouillion and Gomez 2001; NRC 2009; Stocklmayer et al. 2010), and that ISE is pertinent throughout a learner’s lifetime (NRC 2009).

Because informal learning is personal and relevant as well as voluntary (NRC 2009), it is necessarily related to learning through civic engagement. In the spirit of SENCER, civic engagement is both personal and relevant, because society is replete with “wicked problems” that resist simple resolution and require interdisciplinary approaches grounded in civic responsibility (Lawrence 2010). In this sense, learning through civic engagement is similar to community-based service learning in that it is a meaningful connection between students and community, where students use new skills in real-world situations to serve their community. Experiential learning through civic engagement and tackling capacious problems takes this one step further; it exposes the interconnections that make problems “wicked” and promotes deeper learning on the part of both the students and the community. Service learning and civic engagement may be especially important in environmental education where there is a risk of leaving students feeling despondent and powerless as they learn more about environmental issues (Hillcoat et al. 1995). Service and civic engagement have the potential to awaken agency and empower students to make change (Bloom and Holden 2011).

Community-based service learning at its best encourages reflection that promotes civic responsibility, academic success, and personal growth (Arenas et al. 2006). Service learning increases awareness of environmental issues, conservation knowledge, enjoyment of nature, student motivation and engagement in school, and strengthens bonds between community members (Schellner 2008). Importantly, positive environmental attitudes and behaviors ignited through service lasted beyond the service-learning experience (Schellner 2008).

Intergenerational Learning and Community Building

The new generation of older people lead active lifestyles and have interest in future-oriented activities that promote personal fulfillment and social integration characteristic of the “active aging paradigm” (Chadha and Malik, 2010). This project leverages the desire for continued lifelong learning and significant community involvement among elders to facilitate civic engagement through intergenerational learning. Intergenerational learning opportunities are most often defined as occurring with youth under age 21 and adults over age 60 (Kaplan 1997; 2002) and are common in fields of social and health sciences (Roodin et al. 2013). Intergenerational learning programs create intentional exchange of resources and learning among generations (Kaplan 2002). Importantly, intergenerational learning is based on reciprocity of benefit and thus is expected to be mutually beneficial for all generations involved (Ellis and Granville 1999; Tam 2014). Lifelong learning may be intergenerational but typically takes place in informal settings (reviewed in Brostrӧm 2003); thus, the articulation of intergenerational learning in informal settings is a natural combination with potential to enhance education and community connectedness.

Intergenerational learning programs have been successful with a range of age groups in a variety of venues, though most of the documentation of their success comes from students working in gerontology (Roodin et al. 2013). There were both curriculum and relationship-based benefits from a service-learning course in which college students worked with elderly participants (Tam 2014). Community elders working with primary school students (Peterat and Mayer-Smith 2006) showed cross-generational social learning and reciprocity of benefit. On a much larger scale, the Granddad Program in Sweden was successful at bringing senior adult male role models into schools as volunteers (Brostrӧm 2003). Many community-based intergenerational experiences focus on environmental activism, and seniors make especially good environmental steward role models, because they possess the self-motivation for protecting the Earth for future generations (Ballantyne et al. 1998). When seniors were incorporated into a residential outdoor education program, children who worked with senior adults (as compared to the control group) gained more information on a wider variety of topics, and there was a trend toward improved environmental attitudes (Shih-Tsen and Kaplan 2006). In an ISE program, seniors were paired with students on an urban farm, and program participants showed increased environmental awareness associated with the experience (Mayer-Smith et al. 2007).

The benefits of intergenerational service learning programs are well documented (see reviews in MacCallum et al. 2006 and Roodin et al. 2013), and are generally classified as relationship-based and curriculum-based (Tam 2014). Through bidirectional information flow including sharing life experiences and constructive knowledge exchange, participants increase their understanding of each other (Springate et al. 2008). Intergenerational learning programs or courses have the effect of reducing age-related stereotypes (Kaplan, 1997), with students reporting a more positive and appreciative attitude towards the older generation (Zucchero 2009 and 2011; Penick et al. 2014). Benefits to the elderly include benefits attributed to lifelong learning (Brostrӧm 2003): improved self-esteem and life satisfaction (Newman et al. 1997), physical, social and psychological as well as economic benefits (Tam 2011; 2014), maintenance of cognitive functioning (e.g., Ardelt 2000; Boulton-Lewis et al. 2006; reviewed in Tam 2014), and promotion of pro-social values (Brostrӧm 2003).

The benefits to youth from intergenerational learning are better documented than benefits to college students. Intergenerational learning experiences are reported to increase confidence and self-worth and improve practical skills among youth (MacCallum et al. 2006). Youth involved in intergenerational activities showed increased enjoyment in school, were less likely to become involved with drugs, displayed enhanced literacy development (MacCallum et al. 2006) and became more civic-minded and viewed their citizenship in more action-oriented terms (Kaplan 1997). Although many intergenerational service-learning experiences involve young children, working with college students has been shown to enhance the general well-being of older adults also (Hernandez and Gonzalez 2008). Our project adds to this literature by documenting bidirectional information flow and a sense of community belonging among college students and elders.

Project Description

Antioch College and the Glen Helen Outdoor Education Center (OEC) are situated in a Midwestern USA town of approximately 3500 residents, where the median age is 48 and the population is aging; approximately 47.5 percent of the population is aged 50 and older (US Census Bureau 2010).   The College has approximately 200 students and very small class sizes. The OEC is within close walking distance to the college campus.   Over 2700 grade school students and in-service teachers participate in educational programs that meet state teaching standards and are designed and led by a team of paid and trained naturalists at the OEC. The OEC is located within the city limits in a 1000-acre nature preserve (Glen Helen or The Glen) that receives over 10,000 visitors annually and is an important part of the local community.

We used the critical community issue of biodiversity loss to involve students and community members in forest restoration in the local nature preserve. The Glen encompasses a forest ecosystem negatively impacted by invasive species, most notably by bush honeysuckle. Bush honeysuckle has been documented to prevent growth of native understory plants through resource competition, allelopathy, and depleted soil seed banks (Cipollini et al. 2008; Cipollini et al. 2009; McKinney and Goodell 2010; Arthur et al. 2012; Bauer et al. 2012). Forests with invasions of bush honeysuckle also have lower amphibian species diversity and richness, altered patterns of pollinator visitation, song bird assemblages, and soil fungal communities, higher soil compaction, lower soil quality, and lack of certain other qualities that are indicators of a healthy forest understory (Watling et al. 2011). Restoration of forest ecosystems following invasive species removal is dependent on replanting native forest understory species and involves the consideration of numerous intertwined ecological principles that must be in place to sustain and promote the return and establishment of a biodiverse community (Vidra et al. 2007; Swab et al. 2008; Aronson and Handel 2011). Through this project, youth at the OEC, college students, and senior adult community members participated in the propagation of native plants for a forest restoration project in Glen Helen.

As part of our project, college students in the course entitled Introduction to Environmental Science visited the OEC, observed a naturalist-led hike, studied native and invasive species in class and in the Glen, and offered plant propagation workshops to senior adults at a local senior center. Workshops in which students participated were held in the “great room” at the Center, a large, open area. Eight tables with planting supplies were situated in a circle around the room and each table was attended by a student with a different native plant species to propagate. Chairs were arranged so participants could sit or stand at stations and there was ample room for moving from station to station. The workshop began with an introduction to the project, invasive species impacts, and restoration efforts in the Glen. Then participants were encouraged to help clean or plant seeds at any of the stations and to move among stations. The effect was to optimize personal, intergenerational interactions in an experience with direct relevance to people with some connection to the Glen.

The objectives of this curriculum innovation were to

1. Introduce students to informal science education (ISE) as a potential career path
2. Teach content knowledge related to invasive species and biodiversity loss
3. Design and implement an intergenerational learning opportunity that results in bidirectional knowledge sharing

The workshops were designed to engage older adults and college students in meaningful work and ultimately create a sense of community purpose while encouraging environmental responsibility and civic engagement. This type of community connection through active civic engagement promotes the personal fulfillment and social integration sought by elder community members (Chadha and Malik 2010). College students benefit from working with adults of a different generation and forming ties that spill over and enhance community life (Roodin et al. 2013).

Methods

There were two primary activities in the curriculum design; one introduced students to ISE and the second put the students into the position of informal science educators in an intergenerational workshop. We scaffolded the student-led workshops by introducing students to the OEC and having them observe and reflect upon an informal science lesson. The class walked to the OEC at the beginning of the quarter to meet the Director, tour the facility, and discuss OEC programs. During the quarter, students were required to attend one naturalist-led hike, observe the lesson, and submit a reflective assignment within two weeks of completing the hike. The reflection activity included a description of the lesson, suggestions on how to improve or extend the experience, and thoughts on the importance of ISE in education. Two weeks before the workshops, students participated in class work that introduced them to the project, biodiversity, and issues related to invasive species. They chose a native plant (from a list of those available) and completed individual research on the natural history of the plant. Students designed and printed an information sheet on the species and were told to be prepared to describe their species and the project and to answer questions during the workshops. They submitted the species information sheet for feedback and grading before the workshops. Students were divided into two groups to offer two workshops at the local senior center during February 2014. In the workshops, students managed their own “propagation stations,” provided information on their native plants, and cleaned and planted seeds with workshop participants. Students had learned seed cleaning and planting before the workshops in a separate classroom activity.

Students taking the class in fall 2013 participated in the naturalist-led hikes, but workshops were offered only during winter 2014 quarter. Thus, included here are two sets of student reflections on OEC involvement and one set (winter quarter) of workshop assessments. Student responses to an open-ended question on the hike reflection assignment were coded using presence/absence codes based on the assessment prompts (Table 1). Codes included experience (positive or negative), expressed interest in ISE (yes or no), and recognition of ISE as important to the student’s education (yes or no). Two additional codes were added to the analysis of the winter quarter reflections: awareness of ISE before the class (yes or no), and whether or not students noted learning something that they previously did not know about ISE (new learning). To further quantify interest in ISE, students were asked in 2014 if they were interested in a cooperative working experience (co-op) as a naturalist assistant. They could answer yes, no, or maybe and were asked to provide an explanation of their choice. Given the presence/absence format of codes, there was very little room for interpretation. A second coder, unfamiliar with the project, coded the same student responses; the inter-coder reliability, calculated as the proportion of individual excerpts and codes that the individual coders applied similarly, was 95 percent.

To assess knowledge sharing and community building during the workshops, students completed workshop reflection sheets, and older adult participants were asked to complete a post-workshop survey before leaving the Senior Center. Before the start of the workshop, students were asked to keep a tally of the number of participants with whom they interacted and to remember conversation topics. Students completed the reflection sheet immediately at the end of the workshop. The survey for older adult participants included ten statements with 10-point anchored responses that ranged from 1 (not at all) to 10 (very much or a great deal) with the prompts “How much did this workshop…” and “To what degree…” and a space for additional comments.

Four exam questions were used to evaluate student content knowledge about biodiversity and invasive species: (1) What are the five major threats to biodiversity that we discussed in class? (2) What is the number one cause of the loss of biodiversity on the planet?   (3) Outside of bush honeysuckle, what are two additional examples of invasive species that are negatively impacting ecosystems in the USA?  (4) Bush honeysuckle and other invasive plants impact native plants by shading, competition for space and soil nutrients. Describe two additional negative impacts that this invasive has on natural ecosystems (outside of impacts on plants under the honeysuckle). In addition to these questions, students were asked to rate the extent of their knowledge about bush honeysuckle as an invasive species compared to their knowledge before they started the class. Answers were on a five-point Likert scale ranging from none to very high. 

Results

Naturalist-led Hikes

Students who attended their required naturalist hike and submitted a reflection assignment all provided adequate descriptions of the lesson and responded to additional questions appropriately. This indicated that the students attended and engaged in the lesson. Students had an enjoyable experience at the OEC, expressed interest in ISE, recognized the importance of informal learning opportunities and in most cases were interested in additional ISE experiences.   Some students noted that the cold weather was the only aspect of the experience that they did not enjoy, but 100 percent of students in both classes described positive experiences overall.

Some students began with an interest and strengthened or acknowledged that interest, whereas others gained interest in ISE through their participation in the hike at the OEC. Interest ranged from very interested to no interest (Table 1) and, 86 percent (fall) and 87 percent (winter) of students expressed interest in ISE. Students who expressed interest in ISE, recognized ISE as a potential career path and a way to garner teaching experience. One student wrote, “…I am very interested, in fact, that is what I hope to do as a career.” Another wrote, “I am definitely interested in informal science education…. Even if I do not choose being an educator in my profession, I will probably run into a situation where I will be teaching in some way, and informal education can be a great option to handle this opportunity.” One student was interested in education but not specifically informal science education: “…I am somewhat interested in education as a possible career. I’m not entirely sure if informal science education would be the specific career path….” For some students, their experience at the OEC led them to reconsider ISE: “Before this hike I would not have believed I had any interest in informal science education [;] however now I believe I might,” whereas another student, even after this experience, was “still not very interested in informal science education … I have other things that I want to do.” It is not possible to determine whether the lack of interest was because it was specifically science education; none of the students were science majors.

In 2014, when asked about interest in a co-op work position as a naturalist assistant, of the twelve students who replied, only two gave a negative response; the others chose either yes or maybe. The two students who were not interested explained their response by their lack of knowledge in science, lack of interest in working with children, and the need for experience related to their non-science major. Although these two students did not recognize how this experience might benefit them regardless of their major, another student commented, “I would say it’d be a better fit for an environmental science major, or someone who has a bigger interest in being a teacher someday! However, I think it’d be a good experience to have and I would consider it!” Two students who chose “yes” and one who chose “maybe” specifically tied their response to their positive experience on the naturalist-led hike.

Almost all students in both classes (87 percent in fall and 100 percent in winter) provided anecdotes describing the importance of informal learning to their education or, more commonly, in educating youth in environmental science. Many students provided examples of their own positive experiences with informal science education at their grade and secondary schools and through interactions at nature centers. No one described a negative experience with informal science education, and most were very interested in the “outdoors,” and especially in learning more about the specific nature reserve used in this project.

Among the students who described themselves as previously aware of informal science education (86 percent, n = 7, in winter quarter), five of them described how their view changed after the hike. Two admitted that before their experience in the class, they had different concepts of what it meant to work in informal science education (e.g., park ranger). Two students gained appreciation for ISE: “…I never knew how amazing it was” and “Before this hike I knew what informal science education was but I never really considered it as one of the career paths….”   One became aware of the OEC for the first time and another gained awareness of the importance of naturalist jobs: “Looking back however I can understand the importance of her [the naturalist’s] job and of other careers such as hers.”

Increased awareness was often tied to “new learning” about ISE. Although the assessment prompt did not specifically ask about new understanding, half of the students in the 2014 class indicated that they learned something new about ISE through their experience. For example, one student commented, “Visiting the OEC gave me a different perspective on the types of education I might be suited for or interested in” and another, “I had not thought very much about a career in informal science education but now I definitely see how important it is to teach young ones about nature.”

Table 1. Prompts for college student reflection about their experience on a naturalist hike at the outdoor education center
Class	Prompt
Fall 2013 (n = 15)	“Write a short paragraph about your experience with the OEC. Include whether or not you might be interested in informal science education and how informal science education has been or may be important to your education.”
Winter 2014 (n = 12)	Same text as above with the following addition: “Were you aware of environmental education/informal science education as a career before this exercise?”

 

Senior Adult Workshops

The workshops received very positive reviews from students and adult participants. The reflections that the participants provided on the surveys indicated that the workshops facilitated bidirectional sharing of knowledge across generations and a sense of community building. One shortcoming of the workshops was that they occurred during a particularly cold and snowy winter, which limited attendance by senior adults.   There were eight students at each workshop and twelve adult participants at the first and only six at the second workshop. Not all participating adults chose to complete a post-workshop survey, and so, our sample sizes for adult reflections are low. The structure of the workshops encouraged adult participants to move from station to station and interact with several students. Thus, although the number of participants was low, all students had the opportunity to engage with multiple participants during the course of the workshop.

Bidirectional Knowledge Sharing

Post-workshop surveys completed by students showed that on average, each student shared their knowledge of native plants with four adult participants and, on average, three older adult participants shared knowledge with the student. Students listed the types of information that they shared with adult participants, which included information on the plant’s habitat, pollination, use of natural insecticides, forest understory, mesic wetlands, similarities to other plants, planting methods, germination requirements, types of plants (herbaceous and woody), and invasive species impacts. The responses indicated that students were synthesizing and sharing what they had previously learned in class as part of this project or other class activities.

The examples that students provided indicated that participants shared their knowledge of plants as well as general knowledge about a wide range of topics.  Students commented that they learned about tree diseases, organic gardening methods, the history of the Glen, how to recognize some native flowers, and how seeds are dispersed. Adult participants were sharing their expertise with students while the students shared information with them. For example, when asked to provide examples of knowledge shared by participants, students wrote

“One woman talked about the dogwoods she had….”

“…the paw paw festival and different kinds of paw paw cultivation…”

“…the trees [she] saw in the Glen…”

“past/current gardening experiences, talking about their lives in general…”

“…The seeds are long because they can be carried easier by the wind….”

“…got a great book recommendation” and

“I feel like I learned a lot from those who visited my station.”

Sense of Community

Student reflections revealed a positive sense of community connectedness. For example, some student responses to the prompt “How did the experience influence your connection to the community (outside of the campus community) and connection to the Glen?” included

“It felt good to chat with community members and to see how they feel about…”

“I loved to meet members of the community … and get to hear their stories.”

“I was able to make connections based on common interests”

“…It made me feel more connected and more open to the community.…”

“I felt more strongly connected to both the Glen and the community, particularly because we took action to improve the Glen with the help of the community.”

 

And several students indicated a desire to become more involved in the community:

“…encourage me to reach out more to the community at large; they are awesome!”

“I would like to … be more involved with the Yellow Springs community.”

Among the eighteen adult participants in the two workshops, only 14 elected to complete a survey. The highest rated survey questions were “To what degree did you enjoy interactions with students?” and “How much did this workshop increase your interest in getting involved further in Glen Helen restoration efforts?” (Figure 1). On average, all responses were over six out of ten possible levels and indicated an overall satisfaction with the workshops. Interestingly, older adults did not feel that they shared their knowledge with students to the same degree that they increased their own knowledge and that students shared with them. This is contrary to the student’s description of knowledge exchange and appreciation for information shared by older adults. Older adult participants liked the degree of interaction possible in the workshop and expressed a stronger personal connection to the community as a result of their participation.

Content Knowledge

Exam questions for students in the environmental science class were graded as “all or none” to assess content knowledge. Fourteen students completed the four assessment questions included on their exam in winter 2014. Among those 14 students only two described their prior knowledge of honeysuckle as an invasive plant as high and both of these students had some experience working with invasive plant removal in the Glen through other opportunities. All students identified the most common cause of biodiversity loss and correctly listed invasive species in addition to bush honeysuckle; 93 percent were able to provide additional negative impacts of honeysuckle on an ecosystem, and 86 percent correctly listed five threats to biodiversity. Despite their perceived initial lack of knowledge about honeysuckle as an invasive species, students gained knowledge about invasive species during the course of the class activities.

Discussion

Students increased their understanding of informal science education, biodiversity, and invasive species impacts and strengthened connections to the local community through participation as informal science educators in intergenerational plant propagation workshops. The naturalist-led hikes provided students with concrete examples of informal science education in action and appropriate scaffolding for stepping into the role of informal science educator. Students and senior adults alike were extremely positive about the workshops, and within the workshops there was successful bidirectional, cross-generational information sharing.

Student participation in naturalist-led hikes as an introduction to ISE was successful at stimulating interest in and increasing awareness of ISE as a potential career path among college students. This project focused on increasing awareness of the OEC as a local environmental education resource and the potential for students to participate in ISE as part of their science career. Other studies have shown that students’ career planning was enhanced and that they changed their beliefs about careers following short summer programs (Barnett et al. 2011). Anecdotally, there is an indication that the interest in ISE persisted among students: one student applied to the OEC for a paid naturalist position.

The combination of ISE, intergenerational learning and civic engagement with college student participants is relatively unique. Informal science education programs at museums or zoos (NRC 2009), for example, are generally designed for unidirectional knowledge flow from an educator to a diverse public audience. Many intergenerational learning programs at the college level are situated in gerontology programs and often these programs neither promote nor are designed for bidirectional knowledge exchange (Roodin et al. 2013; Tam 2014). Such programs are more correctly deemed multigenerational rather than intergenerational (Tam 2014). In the case of this project, workshops were truly intergenerational, and bidirectional knowledge sharing was easily documented. Sharing of knowledge between students and older adult participants suggests that academic knowledge was in no way privileged over community knowledge (Trickett 1997), and this epistemic equality promoted knowledge flow and, most likely, community connectedness.

Community building as an objective of informal science education and intergenerational learning is based in the theoretical framework described as tapping in to “funds of knowledge” (Basu and Barton 2007). These “funds” are the cultural and historical knowledge residing in the community. Communication of this community knowledge may enhance science education by making science more relevant to the lives of students (Basu and Barton 2007). In this project, intergenerational workshops were described by students as strengthening community connectedness, and the appreciation that students expressed for the knowledge shared with them by senior adults appeared to enhance this community connection and support the overall positive evaluation of the experience.

The success of intergenerational experiences in the context of civic engagement is dependent in large measure upon choosing a critical issue whose approach serves both the public and academic communities.   For this project, it was the connection to place, Glen Helen, that was a driving force for a successful program. Place-based experiential learning has been shown to enhance undergraduate student content knowledge in the plant sciences (Bauerle and Park 2012) and influence individual agency related to environmental issues (Rodriguez et al. 2008; reviewed in McIreneny et al. 2011) and public participation in science (Haywood 2014).   Glen Helen is a valued resource in the community, and satisfaction with the workshops was related to the perception that older adult participants were helping the Glen. Workshops also stimulated interest in being involved with Glen Helen restoration projects, and student reflections on the naturalist-led hikes indicated an interest in learning more about Glen Helen.

Students demonstrated an understanding of content related to invasive species, biodiversity, and native plants on an exam, but more impressively, students communicated content knowledge to adult participants in workshops. Communication of their knowledge to community members indicates that students have some confidence in their abilities and understanding of science. When graduate education students assumed the role of informal science educators, they honed communication skills and increased their confidence in using skills and knowledge gained in the classroom (Crone et al. 2011).

The success of the workshops and the project overall can to some degree be attributed to the consideration of recommendations from previous research on intergenerational service learning. In general, students benefit from authentic learning and participatory experience coupled with structured reflection (NRC 2009). This was incorporated into the project in the form of an educator-community partnership rooted in a civic issue relevant to the lives of participants. Intergenerational ISE programs are best when they incorporate opportunities for significant personal interaction (Fenichel and Schweingruber 2010), something that the senior adults prized about their workshop experiences. It is also important that there is a potential for one-on-one interactions and that programs proceed at a leisurely pace (Shih-Tsen and Kaplan 2006) and take into consideration the mobility or limitations of participants. This project offered student-led workshops that had all of these characteristics.

Shortcomings of the project are primarily related to the low participation by older adults and the lack of a control group. Attendance at the workshops was complicated by poor weather, and this is especially pertinent for older adults who may experience decreased mobility. Winter was chosen as the best time for propagation workshops based on the college schedule and conditions needed for germination and establishment of plant stock for the restoration project. Thus, there was a trade-off between appropriate conditions for participants and logistics imposed by the academic and research schedules. Why some senior adults chose not to complete a survey is not clear. Also, it is not possible to know whether student content knowledge was enhanced as a result of the intergenerational interactions, because there was no control group for comparison. Additionally, because some assignments were graded, it is possible that some student responses lack sincerity, but we have no way of knowing whether this is true. Despite low numbers, results indicate a very positive response by both students and adult participants that is sufficient to warrant scaling up the project.

Whether the benefits of the experience are long-lasting or coupled with increased environmental activism is unknown but an interesting question for further research. Civic engagement tends to increase among students who participate in service learning with older adults (Hegeman et al. 2010; Karasik et al. 2004), and these interactions with a larger community may influence personal ecological identities (Morris 2002). Thus, it is possible that programs that combine ISE, civic engagement, and intergenerational learning yield benefits far beyond those documented for this project.

About the Author

Linda Fuselier is faculty in the Biology Department at the University of Louisville where she is responsible for the redesign of a large enrollment, non-majors biology curriculum. Before moving to Louisville, Linda was at Minnesota State University, Moorhead where she was Biology faculty and Director of Women’s and Gender Studies. At MSUM, she worked with biology faculty to infuse research into the undergraduate curriculum and formed an interdisciplinary team of faculty tasked with infusing the SENCER approach into biology, physics and Women’s Studies classes. Her current research involves creating SENCER-based modules using contemporary women’s issues and designing curriculum materials to for science information literacy.

Acknowledgements

Janene Giuseppe, Director of the OEC, coordinated naturalist-led hikes and collaborated on numerous aspects of the project. Corinne Pelzl, Activities Director at the Yellow Springs Senior Center, worked with the College and hosted and helped advertise and organize workshops. Yellow Springs community members and Environmental Science students participated voluntarily in the project and the project received exempt status through the Antioch College IRB. Glen Helen Ecological Institute permitted native plant restoration and George Bieri, Land Manager, provided expertise on local flora and assisted the restoration project. This project was funded by SENCER-ISE II grant from NCSCE.

Literature Cited

Alliance for Service-Learning Education Reform and Close UP Foundation. 1995. Standards of Quality for School-Based And Community-Based Service Learning. Alexandria, VA. http://tncampuscompact.org/files/qsbc.pdf (accessed May 18, 2015).

Ardelt, M. 2000. Intellectual Versus Wisdom-Related Knowledge: The Case for a Different Kind of Learning in Later Years of Life. Educational Gerontology 26: 771–789.

Arenas, A., K. Bosworth, and H.P. Kwandayi. 2006. Civic Service through Schools: An International Perspective. Compare 36 (1): 23–40.

Aronson, M., and S. Handel. 2011. Deer and Invasive Plant Species Suppress Forest Herbaceous Communities and Canopy Tree Regeneration. Natural Areas Journal 31: 400–407.

Arthur, M., S. Bray, C. Kuchle, and R. McEwan. 2012. The Influence of the Invasive Shrub Lonicera maackii, on Leaf Decomposition and Microbial Community Dynamics. Plant Ecology 213: 1571–1582.

Ballantyne, R., S. Connell, and J. Fien. 1998. Students as Catalysts of Environmental Change: A Framework for Researching Intergenerational Influence through Environmental Education.   Environmental Education Research 4: 285–299.

Barnett, M., M.H. Vaughn, E. Strauss, and L. Cotter. 2011. Urban Environmental Education: Leveraging Technology and Ecology to Engage Students in Studying the Environment. International Research in Geographical and Environmental Education 20: 199–214.

Basu, S.J., and A.C. Barton. 2007. Developing a Sustained Interest in Science among Urban Minority Youth. Journal of Research in Science Teaching 44: 466–489.

Bauer, J., S. Shannon, R. Stoops, and H. Reynolds. 2012. Context Dependency of the Allelopathic Effects Of Lonicera maackii on Seed Germination. Plant Ecology 213: 1907–1916.

Bauerle, T. L., and T.D. Park. 2012. Experiential Learning Enhances Student Knowledge Retention in the Plant Sciences. HortTechnology 22: 715–718.

Bouillon, L. M., and L.M. Gomez. 2001. Connecting School and Community with Science Learning: Real World Problems and School-Community Partnerships as Contextual Scaffolds. Journal of Research in Science Teaching 38: 878–898.

Boulton-Lewis, G.M., L. Buys, and J. Lovie-Kitchin. 2006. Learning and Active Ageing. Educational Gerontology 32: 271–282.

Brostrӧm, A. 2003. Lifelong Learning, Intergenerational Learning and Social Capital: From Theory to Practice. Institute of International Education: Stockholm University, Stockholm, Sweden. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.198.7003&rep=rep1&type=pdf (accessed May 17, 2015).

Carrus, G., M. Scopelliti, R. Lafortezza, G. Colangelo, F. Ferrini, F. Salbitano, M. Agrimi, L. Portoghesi, P. Semenzato, and G. Sanesi. 2015. Go Greener, Feel Better? The Positive Effects of Biodiversity on the Well-being of Individuals Visiting Urban and Peri-urban Green Areas. Landscape and Urban Planning 134: 221–228.

Chadha, N., and N. Malik. 2010. Intergenerational Learning Enhances Community Well Being. Indian Journal of Gerontology 24: 403–410.

Chen W., and C. Jim. 2008. Assessment and Valuation of the Ecosystem Services Provided by Urban Forests. In Ecology, Planning, and Management of Urban Forests: International Perspectives, M.M. Carreiro, Y-C. Song, and J. Wu, eds., 53–83. New York: Springer.

Cheng, J., and M.C. Monroe. 2012. Connection to Nature: Children’s Affective Attitude toward Nature. Environment and Behavior 44: 31–49.

Cipollini, K., G. McClain, and D. Cipollini. 2008. Separating Above and Belowground Effects of Alliaria petiolata and Lonicera maackii on the Performance of Impatiens Capensis. The American Midland Naturalist 160: 117–128.

Cipollini, K., E. Ames, and D. Cipollini. 2009. Amur Honeysuckle (Lonicera maackii) Management Method Impacts Restoration of Understory Plants in the Presence of White-tailed Deer (Odocoileus virginiana). Invasive Plant Science and Management 2: 45–54.

Crone, W.C., S.L. Dunwoody, R.K. Rediske, S.A. Ackerman, G.M. Zenner Petersen, and R.A. Yaros. 2011. Informal Science Education: A Practicum for Graduate Students. Innovative Higher Education 36: 291–304.

Ellis, S. W., and G. Granville. 1999. Intergenerational Solidarity: Bridging the Gap through Mentoring Programmes. Mentoring and Tutoring: Partnership in Learning 7 (3): 181.

Eshach, H. 2007. Bridging In-school and Out-of-school Learning: Formal, Non-Formal and Informal Education. Journal of Science Education and Technology 16: 172–190.

Falk, J.H. 2001. Free-choice Science Learning: Framing the Discussion. In Free-choice Science Education: How We Learn Science outside of School, J.H. Falk, ed., 3–20. New York: Teachers College Press.

Fenichel, M., and H.A. Schweingruber. 2010. Surrounded by Science: Learning Science in Informal Environments. Board on Science Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Haywood, B.K. 2014. A “Sense of Place” in Public Participation in Scientific Research. Science Education 98: 64–823.

Hegeman, C., P. Roodin, K. Gilliland, and K.B. O’Flathabháin. 2010. Intergenerational Service Learning: Linking Three Generations: Concepts, History and Outcome Assessment. Gerontology and Geriatrics Education 31: 37–54.

Hernandez, C., and M.Z. Gonzalez. 2008. Effects of Intergenerational Interaction on Aging. Educational Gerontology 34: 292–305.

Hillcoat, J., K. Forge, J. Fien, and E. Baker. 1995. “I Think It’s Really Great That Someone Is Listening to Us…”: Young People and the Environment. Environmental Education Research 1: 159–171.

Kaplan, M. 1997. The Benefits of Intergenerational Community Service Projects. Journal of Gerontological Social Work 28: 211–222.

Kaplan, M.S. 2002. Intergenerational Programs in Schools: Considerations of Form and Function. International Review of Education 48: 305–334.

Karasik, R., M. Maddox, and M. Wallingford. 2004. Intergenerational Service Learning across Levels and Disciplines: One Size (Does Not) Fit All. Gerontology & Geriatrics Education 25: 1–17.

Lawrence, R.J. 2010. Beyond Disciplinary Confinement to Imaginative Transdisciplinarity. In Tackling Wicked Problems through the Transdisciplinary Imagination, V.A. Brown, J.A. Harris, and J.Y. Russell, eds., 16–29. Washington, DC: Earthscan.

MacCallum, J., D. Palmer, P. Wright, W. Cumming-Potvin, J. Northcote, M. Brooker, and C. Tero. 2006. Community Building through Intergenerational Exchange Programs. Report to the National Youth Affairs Research Scheme (NYARS). Australian Government Department of Families, Community Services and Indigenous Affairs (FaCSIA) on behalf of NYARS. http://www.academia.edu/604333/Community_building_through_intergenerational_exchange_programs (accessed May 20, 2015).

Malcolm, J., P. Hodkinson, H. Colley. 2003. The Interrelationships between Informal and Formal Learning. Journal of Workplace Learning 15: 313–318.

Martin, L.M.W. 2004. An Emerging Research Framework for Studying Informal Learning and Schools. Science Education 88: S71–S82.

Mayer-Smith, J., O. Bartosh, and L. Peterat, 2007. Teaming Children and Elders to Grow Food and Environmental Consciousness. Applied Environmental Education and Communication 6: 77-85.

McIreneny, P., , J. Smyth, and B. Down. 2011. “Coming to a Place near You?” The Politics and Possibilities of a Critical Pedagogy of Place-based Education.   Asia-Pacific Journal of Teacher Education 39: 3–16.

McKinney, A., and K. Goodell. 2010. Shading by Invasive Shrub Reduces Seed Production and Pollinator Services In A Native Herb. Biological Invasions 12: 2751–2763.

Morgan, R.E., R.L. Bertera, and L.A. Reid. 2007. An Intergenerational Approach to Informal Science Learning and Relationship Building among Older Adults and Youth. Journal of Intergenerational Relationships 5: 27–43.

Morris, M. 2002. Ecological Consciousness and Curriculum. Journal of Curriculum Studies 34: 571–587.

National Research Council; Committee on Learning Science in Informal Environments, P. Bell, B. Lewenstein, A.W. Shouse, and M.A. Feder, eds. 2009. Learning Science in Informal Environments: People, Places, and Pursuits. Board on Science Education, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Newman, S., C.R. Ward, T.B. Smith, J.O. Wilson, and J.M. McCre. 1997. Intergenerational Programs: Past, Present and Future. Washington, DC: Taylor and Francis.

Penick, J.M., M. Fallshore, and A.M. Spencer. 2014. Using Intergenerational Service Learning to Promote Positive Perceptions about Older Adults and Community Service In College Students. Journal of Intergenerational Relationships 12: 25–29.

Peterat, L., and J. Mayer-Smith. 2006. Farm Friends: Exploring Intergenerational Environmental Learning. Journal of Intergenerational Relationship 4: 107–116.

Rodriguez, A., C. Zozakiewicz, and R. Yerrick. 2008. Students Acting as Change Agents in Culturally Diverse Schools. In The Multiple Faces of Agency: Innovative Strategies for Effecting Change in Urban School Contexts, A. Rodriguez, ed. Rotterdam, The Netherlands: Sense Publishers.

Roodin, P., L.H. Brown, and D. Shedlock. 2013. Intergenerational Service-learning: A Review of Recent Literature and Directions for the Future. Gerontology & Geriatrics Education 34: 3–25.

Schellner, J. 2008. Environmental Service Learning: Outcomes of Innovative Pedagogy in Baja California Sur, Mexico. Environmental Education Research 14: 291–307.

Shih-Tsen, L., and M. Kaplan. 2006. An Intergenerational Approach for Enriching Children’s Environmental Attitudes and Knowledge. Applied Environmental Education and Communication 5: 9–20.

Springate, I., M. Atkinson, and K. Martin. 2008. Intergenerational Practice: A Review of the Literature (LGA Research Report F/SR262). Slough, U.K.: NFER. http://www.nfer.ac.uk/publications/LIG01/LIG01.pdf (accessed May 20, 2015).

Stocklmayer, S.M., L.J. Rennie, and J.K. Gilbert. 2010. The Roles of the Formal and Informal Sectors in the Provision of Effective Science Education. Studies in Science Education 46: 1–44.

Swab, R., L. Zhang, and W. Mitsch. 2008. Effect of Hydrologic Restoration and Lonicera Maackii Removal on Herbaceous Understory Vegetation in a Bottomland Hardwood Forest. Restoration Ecology 16: 453–463.

Tam, M. 2011. Active Ageing, Active Learning: Policy and Provision in Hong Kong. Studies in Continuing Education 33: 289–299.

———. 2014. Intergenerational Service Learning between the Old and Young: What, Why and How. Educational Gerontology 40 (6): 401–413.

Trickett, E.J. 1997. Developing an Ecological Mind-Set on School-Community Collaboration. In Applied Ecological Psychology for Schools within Communities, J.L. Schwartz and W.E. Martin, Jr., eds., 139–163. Mahwah, NJ: Lawrence Erlbaum.

US Census Bureau. 2010. http://www.census.gov/ (accessed May 20, 2015).

Vidra, R., T. Shear, and J. Stucky. 2007. Effects of Vegetation Removal on Native Understory Recovery in an Exotic-rich Urban Forest. Journal of the Torrey Botanical Society 134: 410–419.

Watling, J., C. Hickman, and J. Orrock. 2011. Invasive Shrub Alters Native Forest Amphibian Communities. Biological Conservation 144: 2597–2601.

Wellington, J. 1990. Formal and Informal Learning in Science: The Role of the Interactive Science Centers. Physics Education 25: 247–252.

Zucchero, R. A. 2009. Outcomes of a Comentoring Project: Inspiration and Admiration. Educational Gerontology 35: 63–76.

———. 2011. A Co-mentoring Project: An Intergenerational Service-Learning Experience. Educational Gerontology 37: 687–702.

 

Introduction to Interdisciplinary Education

“As the pace of scientific discovery and innovation accelerates, there is an urgent cultural need to reflect thoughtfully about these epic changes and challenges. The challenges of the twenty-first century require new interdisciplinary collaboration, which place questions of meanings and values on the agenda. We need to put questions about the universe and the universal back at the heart of university.” –William Grassie (2013)

As the world becomes more complex, given the rapid expansion of technology, the changing nature of warfare, rising energy, and environmental crises, the value of an interdisciplinary education is increasingly obvious. Social, political, economic, and scientific issues are so thoroughly interconnected that they cannot be explored productively, either by experts or students, within clear-cut disciplinary boundaries.

Despite this fact, several problems arise when institutions try to incorporate interdisciplinary education into their programs. Boix Mansilla (2005) noted that the assessment of interdisciplinary work by students is of great concern. She explains that because faculty are often discipline-specific experts, they are unfamiliar with disciplines outside their realm of expertise and have difficulty defining interdisciplinary work. She goes on to explain that, as a consequence, “the issue [of standards] is marred by controversies over the purposes, methods, and most importantly, the content of proposed assessments” (2005, 16).

This paper offers one solution to this dilemma. The following analysis explores the current state of interdisciplinary education, both in academia broadly, and specifically, at West Point through its interdisciplinary Core Program. The sections that follow will highlight the current issues inherent in interdisciplinary education, define interdisciplinary education objectives, and finally, explain the adaptable, multi-functional, interdisciplinary rubric being implemented at the United States Military Academy (USMA), a rubric designed to resolve many of the issues interdisciplinary educators encounter.

The Current State of Interdisciplinary Education in Academia

“The demand is clear. Whether we try to take a stance on the stem cell research controversy, to interpret a work of art in a new medium, or to assess the reconstruction of Iraq, a deep understanding of contemporary life requires knowledge and thinking skills that transcend the traditional disciplines. Such understanding demands that we draw on multiple sources of expertise to capture multi-dimensional phenomena, to produce complex explanations, or to solve intricate problems. The educational corollary of this condition is that preparing young adults to be full participants in contemporary society demands that we foster their capacity to draw on multiple sources of knowledge to build deep understanding.” –Veronica Boix Mansilla (2005, 14)

There are currently several studies, including evaluation measures, defining the essence of interdisciplinary education. The above quote from Boix Mansilla’s “Assessing Student Work at Disciplinary Crossroads” highlights the challenge educators are experiencing in preparing students to meet today’s most pressing problems. This paper will not attempt to address the structure of interdisciplinary education as an institutional convention, but only to define the essential skills and capacities that a student with interdisciplinary understanding would demonstrate. These definitions are essential to understanding and creating a framework for interdisciplinary learning, which is arguably the first step in adequately integrating it into educational programs. Interdisciplinarity is a difficult construct to quantify, and many educators have been unable to frame a definition of it or to assess it in student work. As a consequence of these and other challenges, only a limited number of colleges or universities have implemented formal interdisciplinary programs at the institutional level.

Several analyses (Boix Mansilla 2005; Boix Mansilla and Dawes Duraising 2007; Rhoten et al. 2008; Stowe and Eder 2002) address the key issues surrounding interdisciplinary learning in higher education and offer proposals on how to address them, starting with the definition of the term “interdisciplinary.” One definition of interdisciplinary understanding is “the capacity to integrate knowledge and modes of thinking drawn from two or more disciplines to produce a cognitive advancement—for example, explaining a phenomenon, solving a problem, creating a product, or raising a new question—in ways that would have been unlikely through single disciplinary means” (Boix Mansilla 2005, 16; Boix Mansilla and Dawes Duraising 2007, 216). A definition is particularly important because “a clear articulation of what counts as quality interdisciplinary work, and how such quality might be measured, is needed if academic institutions are to foster in students deep understanding of complex problems and evaluate the impact of interdisciplinary education initiatives” (Boix Mansilla 2005, 16). An agreed-upon definition is currently lacking in academia, and this has resulted in inconsistent grading, teaching, and learning in interdisciplinary education.

One study of well regarded and established interdisciplinary programs in the U.S., which included Bioethics at the University of Pennsylvania, Interpretation Theory at Swarthmore College, Human Biology at Stanford University, and the NEXA Program at San Francisco State University, involved “69 interviews, 10 classroom observations, 40 samples of student work, and assorted program documentation” (Boix Mansilla and Dawes Duraising 2007, 4). The data were gathered in one-hour to 90-minute semi-structured interviews with faculty and students inquiring about the manner of assessment used in their respective programs. Next, examples of student work were used to give examples of what the institution viewed as meeting the definition of interdisciplinarity. From the interviews and student examples, the authors concluded that there are three core dimensions to student interdisciplinary work: disciplinary grounding, advancement through integration, and critical awareness (Boix Mansilla, 2005, Boix Mansilla and Dawes Duraising 2007). These core elements are represented graphically in Figure 1.

The first core element in Figure 1, disciplinary grounding, calls for strong base knowledge in individual disciplines. During the interviews, 75 percent of the interviewed faculty felt that strong subject-area knowledge was necessary for interdisciplinary education that did not sacrifice depth in exchange for breadth. However, the authors noted that the key to successful disciplinary grounding also included the thoughtful selection of which disciplines to use and how to use them. Advancement through integration, the second principle, is universal in all student work in the sense that students are supposed to learn from the work they do; however, what sets it apart in interdisciplinary education is that “students advance their understanding by moving to a new conceptual model, explanation, insight, or solution” (Boix Mansilla and Dawes Duraising 2007, 225). In the study, sixty-eight percent of faculty identified advancement through integration as a necessary element in interdisciplinary understanding and as the quintessential element for the advancement of student understanding. However, various programs and their students interpret this core element differently. For example, some students in in the NEXA Program at San Francisco State University strive for complex explanations, which evaluate the extent to which disciplines are interwoven to create a broad picture of how interconnected different disciplines are on a given topic. Other students in the same program prefer to use aesthetic reinterpretations to connect the literary, historical, and social elements of a given topic. Other students, such as those in the Bioethics program at the University of Pennsylvania, choose to focus on the development of practical solutions based on of the use of multi-disciplined ideas. The final principle from Figure 1, critical awareness, refers to student work being able to withstand examination and criticism and explicitly calls for evidence of student reflection in their work. Student work needs to “exhibit clarity of purpose and offer evidence of reflective self-critique” (Boix-Mansilla and Dawes Duraising 2007, 228).

Rhoten et al. (2008) also conducted a study focused on the similarities and differences between the learning outcomes of liberal arts and interdisciplinary programs. For this particular study, the researchers used student and faculty surveys, interviews, and tests to gather data for their analysis. The authors explain that most liberal arts programs “must develop student capacities to integrate or synthesize disciplinary knowledge and modes of thinking,” which is very similar to the type of synthesis that is expected from an interdisciplinary curriculum (Rhoten et al. 2008, 3–4). The main purpose behind this study was to identify the parallels between interdisciplinary and liberal arts programs, in order to show how a program can be made more interdisciplinary without changing its structure or content. Table 1 shows a summary of several parallels between a liberal arts education and an interdisciplinary education.

Rhoten et al. (2008) also analyzed empirical data to draw out trends on the “222 institutions considered ‘Baccalaureate College-Liberal Arts institutions’ under the 2000 Carnegie Classification system,” whether the interdisciplinary programs offered were majors, minors, optional courses, or required courses (Rhoten et al. 2008, 5). In general, “interdisciplinary programs are still ‘personally driven,’ whereas departments are ‘self-perpetuating'” (Rhoten et al. 2008, 6). “Personally driven” simply means that if students want to broaden their subject-area exposure they must do so on their own. “Self-perpetuating” refers to fact that departments within an institution need to act in their own self-interest in order to survive and thrive; therefore they tend to avoid interdisciplinary efforts. Interdisciplinary education does not support the mission of individual departments, and if students seek it, they must do so on their own initiative. One would therefore conclude that the only way to truly incorporate interdisciplinary education into schools is by making it institutionally mandated, at least for the core curriculum that all students are required to take.

Schools should strive to integrate interdisciplinary efforts into their institutions because “interdisciplinarity breeds innovation” (Rhoten et al. 2008, 12). Although such innovation carries tremendous benefits, the difficulty of measuring student and educator success was again identified as a barrier. Most schools that are already making efforts towards interdisciplinarity believe that they are somewhat successful according to Rhoten et al. (2008). However, in order to mark and measure success, and to continually improve interdisciplinary programs in schools, the authors propose a value-added assessment, which is intended to provide an “assessment regime that measures growth that has occurred as a result of participation in the institution or academic program” (Rhoten et al. 2008, 14). Moreover, some cross-cutting goals that are embedded especially in interdisciplinary studies, such as life-long learning, curiosity, creative thinking, synthesis, and integration, have acquired the reputation of being ineffable and, correspondingly, unassessable” (Rhoten et al. 2008, 83). This common problem was addressed by Stowe and Eder (2002) who identified several assessment measures that are placed on a continuum, as seen in Figure 2. These measures can also be used to better define interdisciplinary standards by providing a multi-tiered adjustable scale that can help to quantify the assessment of student work based on an instructor’s desired outcomes.

Stowe and Eder (2002) state that using a rubric to define and measure interdisciplinary work would improve the “apparently subjective nature” of interdisciplinary assessment. They further recommend the rubric as a “visible standard—a scoring guide—that allows the assessor and the public, for that matter, to recognize expectations and make increasingly fine distinctions about the quantity and quality of student learning” (96). They expand on their recommendation by noting that assessment must be focused on both improving interdisciplinary learning and “improving student learning,” and should be “embedded within larger systems… and create linkages and enhance coherence within and across the curriculum” (80). Without cooperation across different programs, it is impossible to foster an interdisciplinary learning environment.

An example of such cooperation can be seen at USMA, where several academic departments have moved towards a cooperative environment focused on interdisciplinary learning (Elliott et al. 2013). This paper will focus on the education of the USMA Class of 2016 from their freshman year, when the plan to use energy conservation and the NetZero project (an energy initiative by the Dept. of the Army on several Army posts, including West Point, to produce as much energy as it consumes by the year 2020) was adopted to infuse interdisciplinary themes into their core courses. The five Student Learning Outcomes from this effort include four individual discipline-focused outcomes as well as a fifth, which aims to “develop an interdisciplinary perspective that supports knowledge transfer across disciplinary boundaries and supports innovative solutions to complex energy problems/projects” (Elliott et al. 2013, 33). In a larger sense, this objective illustrates that interdisciplinary education addresses the mission of USMA and the Army’s focus on the “development of adaptive leaders who are comfortable operating in ambiguity and complexity will increasingly be our competitive advantage against future threats to our Nation,” as outlined by General Martin Dempsey, Chairman of the U.S. Joint Chiefs of Staff (Elliiott et al. 2013, 30).

Framing the Problem

The Academy produces graduates who can think dynamically in the ever-changing world described in the quotes from Grassie and Boix Mansilla at the beginning of this article. At West Point, this is accomplished by taking not only a multi-disciplinary approach to education, but also an interdisciplinary one. The Academy’s Core Curriculum describes the required classes that all cadets must complete or validate. The Core Curriculum does not include any classes required for a cadet’s major. Other non-academic requirements include three tactics courses and seven physical education courses. The interdisciplinary aspect is a new addition to the curriculum. In recent years, several committees have recommended promoting interdisciplinary approaches to better meet both the Academy’s and the Army’s goals as outlined in Elliott et al. (2013).

To achieve these goals, several academic departments involved in the Core Curriculum developed an interdisciplinary program for the entering plebe class, the Class of 2016. During the first week of classes, freshmen wrote an essay in their Introduction to Mathematical Modeling course, or MA103, about how they would use concepts from different courses to tackle the challenges that NetZero and the alarming problem of energy consumption in the Army pose to West Point. After 30 instruction sessions (approximately thirteen weeks), the freshmen revised these essays in their Composition course EN101. This time they used the knowledge acquired throughout the semester in the English course and in the other courses they were taking. Faculty from the Department of Mathematical Sciences and the Department of English and Philosophy evaluated these revised essays from different perspectives to emphasize the importance and relevance of multiple disciplines. This led to the realization that it was impossible to adequately compare the essays, since the assignments, rubrics, and faculty were not consistent and there was no common rubric to standardize the grading approach. To mitigate this challenge, the essays were compared in our study using the Flesch-Kincaid test (a formula designed to evaluate the difficulty and complexity of technical writing. It consists of two readings: grade level and reading ease) and a comparison of the final grades for the various essays. Scores for a sample of three essays for 25 students, a total of 75 essays, were used to compare improvement in a measureable, quantitative manner. The test consisted of a null hypothesis that there was no significant difference among the ratings, indicating neither improvement nor deterioration of scores from the different assignments throughout the semester, and an alternative hypothesis that there actually was a difference between scores. A two-tailed t-test yielded p-values ranging between 0.3 and 0.6. This indicated that the Flesch-Kincaid results were inconclusive, meaning that neither the null nor the alternative hypothesis could be rejected.

Despite the inconclusive results of the Flesch-Kincaid test, there was a demonstrated improvement in student work, albeit an improvement that was perceived on the basis of a subjective analysis of the essays. Therefore, a new rubric was developed to re-grade all of the essays in a standardized fashion against the desired elements for that particular set of assignments. To facilitate a comparison, this new and straightforward rubric aimed at grading each assignment from the different departments on the same scale. The grades were on a 1–10 scale, and the rubric can be seen in Table 2. The essays were then re-graded according to the same rubric and the results were compared again using a two-tailed t-test.

The challenge in evaluating interdisciplinary work is that the term “interdisciplinary” is not well-defined or broadly understood. This became even clearer after the Chemistry faculty conducted an interdisciplinary group capstone in the General Chemistry course with the Class of 2016 during the second semester of their freshman year. The capstone presented the students a complex and challenging energy problem that was both current and militarily relevant to their future roles as Army officers. This project required groups of students to write a memorandum summarizing their findings on an experimental, portable, and green battery recharger for soldiers in the field, and then to provide a presentation of their results to their commander. Cadets conducted an experiment on the battery recharger to test its efficiency, to compare it to current recharging methods, and to address the social and leadership challenges that would occur when this new equipment was integrated into a unit. In addition, the capstone leveraged the students’ various courses and experiences to scaffold understanding of key concepts and technology necessary to engage the problem. The freshman cadets were expected to utilize what they learned from math modeling, information technology, general psychology, and general chemistry courses in formulating their solution.

The rubric used to grade these capstones was developed by the Chemistry faculty with input from all the participating courses, and then later utilized by the Chemistry faculty in assessing the capstones. The collaborative rubric identified numerous concepts in each course, and as a result, it was several pages long. Perhaps most significantly, it did not define the term “interdisciplinary” for the faculty and the students in the capstone, nor did it make clear the associated expectations. At the conclusion of the rubric, faculty were asked to rate on a 1–10 scale how interdisciplinary their students’ submissions were. The results, displayed in Figure 3, had a standard deviation of .186 and were inconsistent in both the average instructor rating and the range of different ratings faculty assigned. This indicated that the faculty did not share the same understanding of “interdisciplinary” in assessing student work.

Boix Mansilla and Dawes Duraising (2007) state that student interdisciplinary work should “be well-grounded in the disciplines” “show critical awareness,” and “advance student understanding” (223). These criteria both define the basic learning objectives of an interdisciplinary education and address the need for baseline knowledge in the subjects being addressed in student work. While these criteria may not be included in a rubric or other grading mechanism, they provide more of a defined objective regarding interdisciplinary student work.

Although the idea of graduating interdisciplinary-minded students is appealing to many programs, the challenge of measuring the success of interdisciplinary curriculums in producing these “multi-disciplined” graduates has yet to be addressed. The problem of scaling and measuring interdisciplinary education is itself interdisciplinary in nature and, consequently, an abstract idea for many (Boix-Mansilla and Dawes Duraising 2007, 218). Interdisciplinary education evaluation currently lacks a “sound framework” for assessment since the effects of interdisciplinary efforts on student learning are neither well-defined nor proven (Boix Mansilla 2005, 18). As seen in Figure 2 (Stowe and Eder 2002), the assessment of interdisciplinary work is a non-static scale where the balance between the perspectives and entities is never quite the same from project to project, or from class to class. Stowe and Eder (2002) offer a flexible scale for assessment that allows each interdisciplinary quality to be judged according to faculty expectations: how discovery-oriented versus objective-orientated do they want student assignments to be? Rhoten et al. (2008) do correlate several common learning outcomes of a liberal arts education with their interdisciplinary counterparts as seen in Table 1. Although useful for demonstrating extensive possible outcomes and correlations, the linkages are broadly defined and do not specify objectives; this exemplifies the issues of scale, definition, and the non-quantified nature of interdisciplinary education that currently prevail in academia.

All of the aforementioned problems can be traced to a lack of clarity on standards (Boix-Mansilla 2005, 16). Stowe (2002) explicitly calls for a standard for grading, collecting data, and creating a shared understanding, which he suggests could be found in a rubric. A standardized rubric, which is adaptable to several mediums and is general enough to be applicable to several disciplines, is desperately needed for evaluating and assessing interdisciplinary work. Such a rubric needs to clearly define the necessary elements of an interdisciplinary product and be sufficiently adaptable to align with project requirements; this would resolve several of the problems we have identified. In addition, Stowe and Eder (2002) call for the inclusion of very specific elements in a rubric, so that it can address current problems and properly evaluate interdisciplinary work. Among these requirements are assessing complex intellectual processes, promoting objectivity, reliability, and validity in assessment, clearly defining learning objectives for students, and being flexible and adjustable for course or curriculum progression (96). Although we conducted a thorough search, we failed to find a rubric that adequately fulfills this need.

Interdisciplinary Rubric Development

The goal of the rubric developed at USMA is to create a grading mechanism that can be used in multiple project mediums across multiple disciplines. Simultaneously this rubric maintains the integrity of the interdisciplinary goals by creating a more defined standard with which to grade interdisciplinary student work. The rubric also contains open areas for point allotment as well as weighting for each category, which allows faculty to allot points and focus where they see fit. Developing such a rubric required several steps: defining the term interdisciplinary, identifying the elements that student work needs to demonstrate in order to illustrate interdisciplinary thinking, creating a model that visually represents the interconnectivity of these elements, and then using the defined elements and model to arrive at the rubric categories.

The first step in the rubric development process was to define the term interdisciplinary:

Interdisciplinary: The seamless integration of multi-dimensional, multi-faceted ideas into a clearly demonstrated understanding of an issue’s breadth and depth, with sound judgment and dynamic thinking.

Boix Mansilla’s definition of interdisciplinary understanding provided the starting point for the development of the rubric. Additionally, material from the research discussed above identified missing elements from Boix Mansilla’s definition. For example, the best students’ interdisciplinary work included ideas from multiple disciplines that were integrated to demonstrate the level of understanding that the student has attained.

The second step in the rubric development process was to expand the definition of interdisciplinary, in order to create a shared understanding between students, faculty, and those evaluating the interdisciplinary work. To this end, the feedback and lessons learned from previous student work were used to identify the elements common to successful interdisciplinary work. These principles include: discipline specific knowledge, multi-perspective understanding, integration, practical integrated solutions, reflection, and clarity of purpose. To illustrate the interconnectivity of these principles, a conceptual model of the characteristics was created. Initially, the intention was to create a linear model to represent the core principles. However, several issues, such as missing connections and limited complexity, led to the immediate conclusion that a linear model could not completely describe complex nonlinear problem solving. The resulting model, which illustrates a cyclical thinking process, is shown in Figure 4.

The model begins with the framing and scoping of the problem before the application of discipline-specific knowledge, which as we have seen is an essential starting point for interdisciplinary work. The core principle of the integration of ideas was partitioned into multi-perspective understanding, integration, and practical integrated solutions. Multi-perspective understanding and discipline-specific knowledge are connected by an addition sign, which symbolizes understanding a topic from multiple perspectives. This illustrates that students must be able to use discipline-specific knowledge to make this essential connection. The arrow labeled “integration” in the lower part of the model represents the synthesis of discipline-specific knowledge and multi-perspective understanding into practical integrated solutions. Practical integrated solutions are then connected to reflection via a multiplication sign to show that reflection has a multiplicative effect on interdisciplinary understanding. The arrow labeled “clarity of purpose” represents the cyclical process and shows the compilation of all the previous elements back into discipline-specific knowledge. The knowledge gained from the various parts of the cycle can be used in the further learning of other applicable disciplines. This model’s goal is not to explain the rubric, but to illustrate how interdisciplinary education is cyclical in nature, how the characteristics of interdisciplinary understanding are relevant to interdisciplinary education, and how student learning should continue to build.

Next, the core principles of what makes student work interdisciplinary were established, defined, and examined. The elements in Figure 1 above, taken from Boix Mansilla and Dawes Duraising (2007), were used as a starting point for the development of this rubric’s core principles: be well grounded in the disciplines, show critical awareness, and advance student leaning through understanding (223). For the purpose of this rubric, some elements were modified and expanded to create six core principles. A list of the six core principles that were incorporated into the rubric, along with their definitions, appear in Table 3.

Problem framing and scope are derived from the idea that interdisciplinary work should show critical awareness. The definition used in the rubric is very flexible, so that educators can adapt it for different project mediums and faculty, departments, and/or university requirements. Critical awareness, as defined by Boix Mansilla (2007), includes the definition of purpose as well as the integration of ideas. The definition used for problem framing and scope in the rubric requires that the student’s work have a clearly defined purpose. This was created as a separate category because we had observed a clear trend of misunderstanding among faculty regarding the level of complexity that they expected. This is an important aspect of student interdisciplinary understanding; it allows the faculty to scale assignments according to the expected level of student understanding and allows the student to recognize just how complex and multi-disciplined a product the instructor is seeking. For example, if students were assigned a project on how to effectively stock a warehouse, an instructor would not have the same expectations of a freshman who has taken only introductory courses in mathematical modeling and economics as of a senior who had taken nonlinear optimization, supply chain management, and microeconomics courses. Having this requirement in the rubric makes clear the expectation that students will properly identify what they want to address, and also allows the instructor to have a frame of reference in a project.

The rubric’s second core principle, discipline knowledge is well grounded in the disciplines and is intentionally more open-ended, so that it can be readily adapted to different departments, projects, and situations (Boix Mansilla 2007). Identifying theories, examples, findings, methods, etc. may not be relevant or necessary in a given problem. Therefore, although the evaluator is given an area in the rubric that calls for disciplinary knowledge, the rubric does not explicitly indicate how that knowledge is to be graded. For example, in our warehouse stocking project, a freshman might be expected to mathematically model the effects of changing employee wages on productivity. A university senior, on the other hand, might be expected to produce a business recommendation to stakeholders by addressing the intricacies of supply chain management on warehouse profits as well as its psychological implications for employees. The discipline knowledge area of the rubric enables the evaluator to determine how much knowledge and understanding students are expected to demonstrate, while ensuring that the importance of disciplinary understanding is not lost on an interdisciplinary project.

The integration of ideas principle is really the quintessential element for the interdisciplinarity of this rubric. All six core principles are important interdisciplinary factors, but if this element were removed, the rubric could be used for a project that is not interdisciplinary. Integration of ideas derives its meaning from the critical awareness and advanced student understanding pieces identified above in Figure 1. This rubric defines integration of ideas as multi-dimensional, feasible, practical solutions with multi-faceted and seamlessly connected ideas. It is important to note the difference between being integrated and being seamlessly integrated. The seamless integration of ideas, which can take on different meanings depending on the assignment, is an indicator of true multi-dimensional, multi-faceted understanding. Seamless integration. We define the term seamlessly integrated to mean that ideas are not simply laundry-listed, but instead are connected in an intelligent and logical fashion. The definitional elements of multi-dimensional and multi-faceted identify the need for complexity in student work. It is multi-dimensional when students make use of multiple dimensions of their education or, in other words, use multiple disciplines, in their work. Multi-faceted means that students are able to use evidence and knowledge to back up their multi-dimensional claims. The most important component is that students be able to demonstrate a clear understanding of what they are presenting. This also relates to a student’s ability to demonstrate the span of an issue’s breadth and depth. In other words, students should be able to apply disciplines to an issue or topic with an appropriate understanding of the level of each of the disciplines. The use of extraneous disciplines merely for the sake of incorporating more disciplines does not necessarily make student work interdisciplinary. In fact, it contradicts the idea of advancing the complexity of the student’s thought process.. Students who apply the appropriate level of discipline breadth and depth indicate their ability to use sound judgment or logic, as well as their ability to think dynamically.

The next two core principles, clarity of purpose and reflection, were added to address the students’ failure to internalize what they were learning and understanding; this was revealed during the analysis of the USMA interdisciplinary program. The main challenge was that students did not fully grasp why a given project was interdisciplinary, or why that was important. To alleviate this, the core principle clarity of purpose was added to the rubric to help students understand the “why”; the intent was to motivate them to define the purpose of their investigation and to take an in-depth approach to the problem. This is different from problem framing and scope in a very important way: problem framing and scope focuses on a well-defined thesis statement or purpose statement, whereas clarity of purpose focuses on the content of student work. In other words, problem framing and scope ask whether students have a clearly stated framework for their project, while clarity of purpose asks whether they demonstrate their personal interdisciplinary understanding and then explain it well to their audience. Similarly, the next principle, reflection, calls for a clear and delineated connection of ideas and an indication that students have reflected on the interconnectivity and importance of their areas of study. These two core principles are drivers of internalization and cognitive advancement in interdisciplinary learning. They are particularly important because often students do not reflect on what they have learned. The reflection piece is intended to facilitate a deeper understanding of what they are learning and to encourage students to consider how the material fits into the greater scheme of their education.

The final element of the rubric shown in Table 3 is the presentation principle. This principle calls for information that is presented in a suitable medium with proper tone, word choice, spelling, grammar, etc. In short, did the students address the audience correctly and present their knowledge intelligently while doing so? This section can be adapted to the type of project and course for which the rubric is being used. For example, English faculty would probably expand this section because of its importance to their learning outcomes, whereas chemistry faculty may place more emphasis on the discipline-knowledge portion.

The newly developed rubric was presented to the Math course leaders for use on the freshman’s “mini” capstone exercise in December 2013. The rubric was sent to the faculty with minimal guidance. The feedback from the course director made it clear that the students and faculty did not fully grasp the intention or expectations behind the rubric. A few factors contributed to this: sixty-six percent of the faculty were new to the department; the interdisciplinary expectations were not fully explained to the faculty; although everyone received the rubric, each instructor created his or her own rubric for the mini-capstone; and the students who took the mini-capstone and the faculty who graded their work were under significant time pressure. The mini-capstone in its creation, execution, and grading was not given adequate time due to end of semester requirements at USMA during the November-December time period. An important conclusion from this feedback was that the faculty needed to have a common understanding of what is expected on an interdisciplinary project. To achieve this for the General Chemistry capstone project in the spring of 2014, a grading calibration exercise was conducted. This calibration included good and poor examples of interdisciplinary work from the previous year’s chemistry capstone, and showed faculty how to distinguish between good and poor work and how to use the rubric in assigning a grade.

Implementing the Interdisciplinary Rubric

The first step in implementing the rubric was calibration with the faculty. With such an exercise, the faculty should take away a common understanding of what exactly interdisciplinarity is as well as the knowledge of what constitutes a good final project. The plan for the calibration exercise developed for USMA faculty who would be grading the CH102 General Chemistry capstone in the spring of 2014 was an hour-long presentation and discussion. Prior to the presentation, faculty received a packet of examples of cadet work in each of the major portions of the previous year’s capstone project. The examples included “A” work as well as examples of common integration errors students make: the “laundry list,” the “tacked on at the end,” and the “no real knowledge” integration errors. The “laundry list” is an example of how a student may mention and be knowledgeable in multiple disciplines but does not integrate them, providing instead a “laundry list” of the different disciplines and explaining the relevance of each individually. The “tacked on at the end” error (or whatever we may call it) exemplifies how a student may go in-depth in one discipline, particularly in the discipline for which the assignment was given, then tack on a sentence or two at the end mentioning other disciplines in order to call the project interdisciplinary. The “no real knowledge” example presents a plethora of ideas but does not demonstrate that the student learned or integrated disciplines and/or ideas. With these examples, faculty became more familiar with what correct and incorrect work looked like. The “A” level example was not meant to illustrate the perfect or only solution; it was merely one example. Faculty evaluated each example using the standard A, B, C, D, F grading scale based on how interdisciplinary they felt each project was.

At the start of the presentation portion of the rubric calibration, faculty were introduced to the interdisciplinary characteristics and model from Figure 4. This ensured understanding of interdisciplinary characteristics prior to the introduction to the rubric itself. After the characteristics were covered, the results from the exercise, which the faculty just had completed, were discussed. This clarified any misunderstandings that faculty had about the interdisciplinary characteristics, while the examples of chemistry capstones from the previous year provided a frame of reference. Next the rubric was thoroughly explained, showing how it was scalable, expandable, and concise to meet instructor needs for interdisciplinary student projects.

The General Chemistry capstone rubric for 2014 differs from its 2013 predecessor in two very important ways. First, it is significantly shorter; its two pages (compared to seven pages) emphasize quality over quantity. Instead of listing every detail of the project, the new capstone rubric has five categories that address the math modeling, leadership, information security, oral communication, and the required submission components of the project, all without specific details. This allows the students to be more creative in their answers to the given problem.

The 2013 rubric was not based on any interdisciplinary principles or examples. Instead, it listed specific requirements from the disciplines the students were supposed to integrate. The result was quite the opposite: the 2013 capstone projects tended to be disjointed because of the slew of specific requirements. This year’s capstone rubric incorporates the interdisciplinary principles described in Table 3.Problem framing and scope is addressed in the Project Summary section with the requirement for a bottom line up front (BLUF), or thesis. Discipline knowledge is asked for in the Discrete Dynamic Modeling, Persuasion and Conformity in a Leadership Environment, and Information Security sections. Although the course-specific requirements must be addressed, Integration of ideas is assessed in the Oral Communication and Project Summary sections, which requires that fluid transitions and logically ordered and related ideas be integrated. Appropriate presentation is also adequately addressed in these sections, as the rubric lays out clear expectations of the written and oral presentations for students, including their tone, body language, and level of professionalism. Clarity of purposeand reflectionare asked for in the Project Summary section, which calls for contingency plans and thoroughly explained analysis of the total problem.

Initial instructor feedback on the use of this rubric is that it better defined expectations for the students’ interdisciplinary work, for both the instructor and the students. After using the rubric in the calibration exercise, instructors stated that they felt more confident and prepared than they had in 2013 when there was no such exercise and assessment tool available; this year they understood what was asked of them and of the students. Initial comparisons of the interdisciplinary assessments of the students’ work from 2013 and 2014 are quite positive. On a scale of 0–10, the average interdisciplinary score given by instructors was 5.69 in 2013, with zero being the least interdisciplinary and 10 the most. (See Figure 3 for these data.) In 2014 this improved to 7.79 (actually 15.5/20). There was also less variability between instructors. For example, in 2013 the standard deviation of the mean scores assigned by each of the instructors was 1.86 (Figure 3). In 2014, the standard deviation between the instructors’ mean scores was .98 (1.96/20), a decrease of over 47%.

Future Work and Conclusion

Now that the General Chemistry capstone for USMA Class of 2017 has concluded, several analyses must be completed to evaluate the progress of interdisciplinary education at USMA. At a minimum, an analysis of the grades and feedback from the students and faculty needs to be conducted. The analysis of the grades should include a distribution of grades compared with their expected distribution, as well as a quantitative and a qualitative analysis of the capstones compared to the previous years’ capstones. This could be done using the methods previously employed, including the use of Flesch-Kincaid, paired t-test, the distribution of the faculty’s interdisciplinary rating similar to Figure 3, and/or a cross-course sample of projects re-graded by the course director.

The discussion and research that have taken place at West Point since the first General Chemistry capstone project in 2013 indicate that the results of this year’s changes should be positive. Although there is as yet no statistical evidence to demonstrate improvement, the general understanding of how interdisciplinarity looks, how to produce it, and how to assess it is much more expansive now than in 2013. The reason for this might be that faculty and students at USMA are now experienced with interdisciplinary work and have a clearer understanding of interdisciplinary assessment and its importance over the course of a year.

The world is a complex and rapidly changing place that requires its future scientists, scholars, engineers, teachers, and leaders to think dynamically and across disciplines.. Interdisciplinary assessment is necessary for the future of education, particularly at West Point where we recognize that “adaptive leaders who are comfortable operating in ambiguity and complexity will increasingly be our competitive advantage against future threats to our Nation” (Elliott et al. 2013, 30). Only time will tell whether this interdisciplinary rubric has met its goal of creating a grading mechanism that can be used in multiple project mediums across multiple disciplines. Given the extensive research and analysis done at West Point to create this much- needed and useful tool, the prospects for future interdisciplinary education are promising.

About the Authors

Elizabeth Olcese graduated with a Bachelor of Science degree in Operations Research from the United States Military Academy at West Point, NY in 2014. She served as a student researcher for West Point’s Core Interdisciplinary Team focused on enhancing opportunities for interdisciplinary learning in West Point’s core academic curriculum. Upon completion of the Quartermaster Basic Officer Leader Course at the Army Logistics School in Fort Lee, VA, she will serve as a second lieutenant for the 25th Infantry Division at Schofield, Hawaii.

Joseph C. Shannon graduated with a doctorate in Curriculum and Instruction with a focus in Science Education from the College of Education at the University of Washington, WA. He is a former member of West Point’s Core Interdisciplinary Team that was focused on enhancing opportunities for interdisciplinary learning in West Point’s core academic curriculum. He is a former Academy Professor at the United States Military Academy and Program Director for the General Chemistry Program in the Department of Chemistry and Life Science. He is currently the Dean of Academic Programs at South Seattle College in West Seattle, Washington.

Gerald Kobylski graduated with a doctorate in interdisciplinary studies (Systems Engineering and Mathematics) from Stevens Institute of Technology, NJ. He currently is co-leading a thrust to infuse interdisciplinary education into West Point’s core academic curriculum. He is also deeply involved with pedagogy, faculty development, and assessment. Jerry is an Academy Professor at the United States Military Academy, a Professor of Mathematical Sciences, and a Commissioner for Middle States on Higher Education.

Lieutenant Colonel Charles (Chip) Elliott graduated with a doctorate in Geography and Environmental Engineering from Johns Hopkins University in Baltimore, MD and is a registered professional engineer in Virginia. He is currently the General Chemistry Program Director and the Plebe (Freshman) Director for the Core Interdisciplinary Team at the United States Military Academy. He has previously taught CH101/102 General Chemistry, EV394 Hydrogeology, EV488 Solid & Hazardous Waste Treatment and Remediation, EV401 Physical & Chemical Treatment, and EV203 Physical Geography. He is currently an Assistant Professor in the Department of Chemistry and Life Science.

References

Boix Mansilla, V. Interdisciplinary Understanding: What Counts as Quality Work?” Interdisciplinary Studies Project, Harvard Graduate School of Education.

Boix Mansilla, V. 2005. “Assessing Student Work at Disciplinary Crossroads.” Change 37 (1): 14–21.

Boix Mansilla, V., and E. Dawes Duraising. 2007. “Targeted Assessment of Students’ Interdisciplinary Work: An Empirically Grounded Framework Proposed. The Journal of Higher Education 78 (2): 216–237.

Elliott, C., G. Kobylski, P. Molin, C.D. Morrow, D.M. Ryan, S.K. Schwartz, J.C. Shannon, and C. Weld. 2013. “Putting the Backbone into Interdisciplinary Learning: An Initial Report.” Manuscript submitted for publication, United States Military Academy, West Point, NY.

Grassie, W. (n.d.). “Interdisciplinary Quotes.” Thinkexist.com. http://thinkexist.com/quotation/as-the-pace-of-scientific-discovery-and/1457907.html (accessed November 16, 2013).

Ivanitskaya, L., D. Clark, G. Montgomery, and R. Primeau. 2002. “Interdisciplinary Learning: Process and Outcomes.” Innovative Higher Education, 27 (2): 95–111.

Newell, W.H. 2006. “Interdisciplinary Integration by Undergraduates.” Issues in Integrative Studies 24: 89–111.

Repko, A.F. 2007. “Interdisciplinary Curriculum Design.” Academic Exchange Quarterly 11 (1): 130–137.

Repko, A.F. 2008. “Assessing Interdisciplinary Learning Outcomes. Academic Exchange Quarterly 12 (3): 171–178.

Rhoten, D., V. Boix Mansilla, M. Chun, and J. Thompson Klein. 2008. “Interdisciplinary Education at Liberal Arts Institutions.” Teagle Foundation White Paper.

Stowe, D.E., and D.J. Eder. 2002. “Interdisciplinary Program Assessment.” Issues in Integrative Studies 20: 77–101.

Appendix 1

Download (PDF, 191KB)

 

Download (PDF, 998KB)

 

 

Abstract

The National Science Bowl® emphasizes a broad range of general and specific content knowledge in all areas of math and science. Over 20,00 students have chosen to enter the competition and be part of a team, and they have enjoyed the benefits of their achievements in the extracurricular Science Bowl experience. An important question to ask, in light of the effort it takes to organize and participate in regional or national science competitions, is whether the event makes a difference to the student. And if it does make a difference, does it improve student learning or student attitudes about science? In a preliminary survey, students competing in a Regional Science Bowl Competition report that the event has a positive impact and fosters learning in science and mathematics. These data support findings for other forms of extracurricular academic competitions associated with science and mathematics.

Introduction

Since 1991, the Department of Energy’s (DOE) National Science Bowl® has been sponsoring annual regional and national competitions for high school students across the United States of America, including Puerto Rico and the U.S. Virgin Islands. In addition to seeing the pragmatic value of increasing the “feed” of science-educated personnel into DOE research facilities, the DOE recognized that the improvement of science education, broadly, would be of great benefit to the nation. Expanding its focus beyond formal science education at the college level, the DOE started the Science Bowl program to encourage high school student participation and interest in math and science. The idea was to increase science literacy in general and to encourage science- and mathematics-related careers specifically. The success of the high school competitions resulted in the expansion of the program to include middle schools in 2002.

The competitions feature teams of four to five students answering multiple choice and short answer questions in the areas of science, mathematics, energy, and technology. There are currently 67 regional high school and 36 middle school competitions. The high school competitions involve more than 15,000 students and the middle school contests more than 6,000. The winning team from each regional event is invited to Washington D.C. to compete with other winners.

Participation in Science Bowl involves working as a team, and a team’s level of success is determined not only by scientific knowledge, but also by teamwork and gamesmanship. The students’ engagement in group work directly benefits the individual team members, their social groups, and society as a whole (Greif and Ephross 2011, 6). The actual team formation and function is itself a model for both future community engagement and civic activism. In fact, creating teams is one of the three principal strategies for successfully placing students in service-learning opportunities within communities. (Harris 2009).

The National Science Bowl® emphasizes a broad range of general and specific content knowledge in all areas of math and science. Science Bowl experiences are independent of the classroom environment and generally occur because the students have volunteered to enter the competition and become part of a team. Each team must have a coach, who can be a parent or other interested person, but is usually a high school science teacher. The volunteer aspect of the competition as an extracurricular activity means that it is similar to robotics competitions, the Science Olympiad, and other interdisciplinary, multi-disciplinary, and applied endeavors. All of these programs stress the collaborative and communal nature of the projects over the content, a characteristic shared by other civic engagement and volunteer endeavors (Jacoby and Ehrlich, 2009).

An important question to ask in light of the effort it takes to run regional or national science competitions is whether the event makes a difference to the student. And if it does make a difference, does it improve student learning or student attitudes about science? The literature on science competitions is not extensive. Abernathy and Vineyard (2001, 274) asked students who competed in the Science Olympiad why they did so. The number one reason for participating in the Olympiad was that it was fun. The number two reason was that the participants enjoyed learning new things. These findings held for both male and female participants; they seemed to think learning science and math in this context was enjoyable. Abernathy and Vineyard suggested that competitive events “may be tapping into students’ natural curiosity and providing a new context for them to learn in, without rigid curriculum or grading constraints (2001, 274).”

Competitive events such as the National Science Bowl® may provide the “initial motivation” and catalyst for helping students to discover the joy of learning (Ozturk and Debelak, 2008). Academic competitions can provide motivation for students to study, learn new material, and reinforce previously learned material so that they will be ready to compete (and collaborate) with their peers from other schools both regionally and nationally—not just in games but also in academic and work environments. This type of motivation is difficult to provide in a normal classroom environment. While it can be argued that this is solely extrinsic motivation and that students should not be dependent on it, it can nevertheless serve as the spark that ignites a discovery of the joy of learning science and math.

One of the more important effective benefits of competitions like the National Science Bowl®, is that the participants, who may be the academic elite at their home schools (big fish in a little pond), must test their knowledge and skills against the students from other schools who will be their peers once they get to college and the workplace. Ozturk and Debelak (2008) note that students “learn to respect the quality of work by other children and to accurately assess their own performance in light of the performance of their intellectual peers. They achieve an accurate assessment of where their level of performance stands in the world of their intellectual capacity and, in turn, develop a more wholesome self-concept” (51) . Developing a more accurate and grounded self-concept is an important stage for children to go through on their way to becoming healthy and mature adults. This realistic and comparative self-assessment can be difficult to foster in the case of elite students who have never faced stiff competition or external challenges to their academic abilities in their home institution.

Students in academic competitions also benefit from learning not only how to succeed, but how to accept failure, learn from it, and, “subsequently, grow as a person and improve in performance” (Ozturk and Debelak 2008, 52). This, again, may be one of the most important aspects of intramural academic competitions, one that cannot be easily provided in a typical classroom environment; learning to fail and being able to cope with the emotional aftermath may be riskier in a classroom environment than in a games environment where the experience of failure is shared among the group. Being thrust into a situation where participants must deal with failure (even after they have prepared and done their best) promotes the healthy development of a student’s resilience and self-awareness. Academic competitions like the National Science Bowl® and its many regional competitions may provide the type of environment that helps students to reflect on their knowledge and abilities and self-evaluate their performance, promoting improved personal growth and development for the participants.

Certainly, extreme competitiveness can cause anxiety and undue stress (see for example Davis and Rimm, 2004). Many of us can remember learning in our Psychology 101 course about test anxiety and how it can negatively affect student performance and achievement and lead to low self-esteem. But Davis and Rimm also report that competition can increase student productivity and achievement. Some students seem to need to compete with others in order to push themselves to produce at a higher level. It would follow that socially organized competitions like the National Science Bowl® and its many regional competitions could help to promote high levels of achievement and productivity in the participating math and science students. Some of the increased levels of achievement and productivity may be due to the practice in teamwork and study skills promoted by participation in this type of academic competition. Bishop and Walters (2007) report that the students involved in competition increased their ability to be leaders and team players, especially in the areas of directed studying (“cramming”), communication, and stress management.

Most studies of this nature tend to be based on student reporting of their own perceptions, and Bishop and Walters also discuss the viability of using a self-report, Likert scale survey to investigate how the National Ocean Sciences Bowl (NOSB) influenced the participants’ choice of major and courses in college. They further triangulate their data using follow-up interviews, information on the colleges the students attended, and lists of the college courses the students took following their participation in the NOSB. Their longitudinal study, which took place from 2000–2007, establishes the credibility of the students’ self-reported data using this type of survey (Bishop and Walters 2007).

What Do the Students Get from This Competition?

A brief survey was developed for the students who compete in the Northern New England Regional Science Bowl Competition, for the purpose of gathering information about the students’ perception of the impact the competition has on them and other students. The questions were developed by the Regional Science Bowl coordinators and distributed to the students (also to coaches, volunteers, and audience) on the actual day of the competition, which takes place each year in late February or early March. The students in the Northern New England Regional Science Bowl Competition come from the three northernmost New England states, Maine, Vermont, and New Hampshire. The competition is an extracurricular activity; the students in grades 9–12 have self-selected to be part of a team that practices and competes during non-school hours. The students making up the teams tend to be academically successful. As might be expected, these students usually like mathematics and science and are predisposed to participate in activities involving these subjects. The teams of students compete in a one-day event at the University of Southern Maine, which culminates in a single elimination tournament round. The winning team is offered an all-expenses-paid trip to Washington D.C. to compete with other regional winners for the national championship. Students at the regional bowl are given the survey. Completing and returning the survey is voluntary, although the students and coaches are made aware that their responses will help improve the event.

The Instrument

The first part of the survey was designed to collect general background information about the students and their role in the day’s competition. This section was a simple checklist:

• This is my first experience.
• I’ve been at previous science bowls here.
• I was a volunteer today.
• I am a spectator/guest.
• I was one of the student competitors today.
• I am a coach of one of the teams.

The next set of items was intended to gain insight into the students’ perceptions of how the regional competition affected the students who were taking part in the day’s activities and events. The questions consisted of three Likert-type response choice items:

1. I think this competition had a positive impact on the students:

2. Quiz competitions foster student learning about science and mathematics:

3. Quiz competitions are stressful in a negative way:

Each of these questions had a five-choice scale that ranged from strongly agree to neutral to strongly disagree. There were also two open ended questions:

The thing I enjoyed most about today was:

What I would recommend for next year:

And finally a yes/no question:

I’d like to come back next year.

Findings and Discussion

Data collection began with the 2004 Northern New England Regional Science Bowl Competition and continued through 2009. (After this year the Bowl was restructured and focused exclusively on Maine students, although participants continue to be surveyed.) This six-year longitudinal study has provided data representing a constant mix of new and returning students. Throughout the course of the study, there was an almost equal distribution of first-time and returning students who responded to the survey. Although the survey was distributed to students, coaches, and other volunteers who took part in the events, only the results of the student surveys were used as part of this report. The voluntary nature of conducting the study produced an average of fifteen percent of the students per year completing and returning the survey. Interviews with coaches and students indicate that the low response rate is most likely a result of its collection at the end of a long, intense day, when many teams were eager to start their journeys back to homes throughout northern New England.

Of the students participating in the Northern New England Science Bowl who responded to the survey during the study period, 93 percent either agreed or strongly agreed that the competition had a positive impact on them (Table 1).

Campbell and Walberg (2011) suggest that this type of positive impact follows the students throughout their life. Willingness to participate in events on their own time, especially during the weekend, demonstrates a high level of positive engagement that would foster feelings of positive impact. Akey (2006,16) reports that “student engagement and perceived academic competence had a significant positive influence.” on achievement. The survey results also suggest that the students perceive themselves as academically competent in math and science, and that is why they participate. This mirrors the findings of Abernathy and Vineyard (2001) who report that academic competitions tap into the natural curiosity and inclinations of students and provide an arena for them to learn new things. The science bowl event could provide the platform for these students to excel and receive recognition. Further, Ozturk and Debelak (2008) report that academic competitions may provide the motivation to find the joy in learning. Curiosity and motivation are important aspects of learning that would presumably have a positive impact on the lives of the participants in academic competitions like the National Science Bowl®.

Most (91 percent) of the respondents reported either that they agreed or that they strongly agreed that the Regional Science Bowl Competition fosters student learning in science and mathematics (Table 2).

These data again appear to support the research done by Abernathy and Vineyard (2001), indicating that academic competitions provide a forum to stimulate the students’ natural curiosity about learning new things, as well as the work of Ozturk and Debelak (2008), who have concluded that academic competitions may motivate students to discover the joy of learning.

The high positive response rate of these two questions indicates that the student participants in the Regional Science Bowl Competition are developing a strong positive sense of self. These responses, reinforced by our interviews of participating coaches, indicate that the students are reflecting on their experiences and developing a more complete self-image and perhaps an increased sense of their personal competence. Bishop and Walters report that an enhanced and comparative sense of personal competence or capability “translates as a very high factor influencing career choice” (2007, 69). It may well be that academic competitions such as the National Science Bowl® and its associated regional competitions provide experiences that positively influence student career choices.

Interestingly, the same students who reported that the Science Bowl Competition had such a positive effect on them in general, and a positive effect on their learning, did not necessarily think the competition was unstressful. Only 61 percent disagreed or strongly disagreed that the quiz competition was stressful in a negative way (Table 3).

Perhaps the wording of the question led students to equate “quiz” with “test,” which affected their response. It could also be that the students consider any kind of stress negative, and if they perceived that the competition created even a low level of stress, they would conclude that this was a negative effect.

In the open-ended question that asked what they enjoyed the most about the Science Bowl, the number one response was competition, the second most frequent response was meeting like-minded people, and the third was the hands-on nature of the activities. These students seem to be saying that they feel that testing their knowledge and skills in science and mathematics against other students of similar ability is fun! Maybe this is because they are beginning to form a deeper understanding of and respect for the quality of their work, as suggested by Ozturk and Debelak (2008). Academic competitions (such as the Science Bowl) may give students the opportunity to compete mentally the way athletic competitions allow them to compete physically (Parker 1998). Perhaps these students get the same kind of “high” that athletes get during competition, and the thrill of academic competition releases endorphins much the same way that athletic competition does.

The data indicate that a statistically significant portion of the students competing in the Northern New England Regional Science Bowl Competition report that the event has a positive impact on them and fosters learning in science and mathematics. These data support findings that have been reported for other forms of academic competitions that are involved with science and mathematics (e.g. Campbell and Walberg 2011). Self-reporting indicates that the students have a high level of perceived personal competence, a high level of engagement in mathematics and science activities, and a high level of motivation toward these academic subjects. In addition to increased involvement in the community, competence, engagement, and motivation are factors that have been linked to academic achievement, personal growth, and career choices. If the education community is seeking to increase student interest and participation in science and mathematics majors and in science and mathematics careers, and ultimately in complex science-related public policy discussions, then academic competitions like the National Science Bowl® may be an important part of the overall strategy bringing the nation closer to that goal.

A Proposal for Further Study

A key aspect of the Science Bowl competition is its role in building a social community of contestants, which leads one to wonder whether the competitions contribute to increased involvement in the larger community and whether they encourage participants to become more effective and engaged citizens. Participating schools are likely to return to the event, as are alumni who come back as volunteer officials. Further, with the release of recent studies, such as “Steady as She Goes? Three Generations of Students through the Science and Engineering Pipeline” (Lowell et al., 2009), we (the authors of this paper) feel an ethical responsibility to continue the investigation of whether science competitions represent meaningful contributions to the experience of students and their disposition towards science.

To better understand the impact of the Science Bowls on both STEM learning and civic engagement, we recommend that surveys be administered for all the National Science Bowl® middle school and high school competitions. The surveys should be standardized, with optional regionally based questions, and should be part of a well-designed study that can inform future science bowl decisions. An existing instrument, the Student Assessment of Learning Gains (SALG, http://www.salgsite.org/), has survey questions that are geared towards formal academic courses but are a no-cost, accessible means to obtain data on students’ attitudes about science. Social media also provides opportunities for assessment and self-reporting of students. Surveys can be followed up by focus group interviews that could provide greater depth to our understanding of the findings. Such longitudinal studies could serve to verify whether or not these informal and volunteer learning experiences correlate with continued interest and involvement in science and mathematics, including choice of college majors, careers, and enhanced awareness and involvement in our most pressing science-related civic challenges, including climate change, public health, and technology.

About the Authors

Robert Kuech (Bob) taught middle and high school physics, chemistry, physical science, biology, ecology, computer programming for 20 years before returning to Penn State to work on a Ph.D. in science education. In 1999, when he finished his studies at Penn State, he came directly to USM and has served as the science educator in the Teacher Education Department since that time.

Robert M. Sanford (Rob) is Professor of Environmental Science and Policy and Chair of the Department of Environmental Science and Policy at the University of Southern Maine in Gorham, Maine. He received his M.S. and Ph.D. in environmental science from SUNY College of Environmental Science & Forestry. His research interests include environmental impact assessment and planning, and environmental education. He is a co-director of the SENCER New England Center for Innovation (SCI) and is a SENCER Leadership Fellow.

References

Abernathy, T., and R. Vineyard. 2001. “Academic Competitions in Science: What Are the Rewards for Students?” The Clearing House 74 (5): 269–276.

Akey, T.M. 2006. School Context, Student Attitudes and Behavior, and Academic Achievement: An Exploratory Analysis. New York: MDRC. http://www.mdrc.org/publications/419/full.pdf. (Accessed July 7, 2014.)

Bishop, K., and H. Walters. 2007. “The National Ocean Sciences Bowl: Extending the Reach of a High School Academic Competition to College, Careers, and a Lifelong Commitment to Science.” American Secondary Education 35 (3): 63–76.

Campbell, J.R., and H.J. Walberg, 2011. “Olympiad Studies: Competitions Provide Alternatives to Developing Talents That Serve National Interests.” Roeper Review 33:8–17.

Davis, G.A., and S.B. Rimm, 2004. Education of the Gifted and Talented. 5th ed. New York: Pearson.

Greif, G., and P. Ephross. 2011. Group Work with Populations at Risk. Oxford: Oxford University Press.

Harris, J.D. 2009. “Service-learning: Process and Participation.” In Service-learning and the Liberal Arts, C.A. Rimmerman. ed, 21–40. Lanham, MD: Rowman & Littlefield.

Jacoby, B., and T. Ehrlich, eds. 2009. Civic Engagement in Higher Education. San Francisco: Jossey-Bass.

Lowell, B.L., H. Salzman, H. Bernstein, and E. Henderson. “Steady as She Goes? Three Generations of Students through the Science and Engineering Pipeline.” Paper presented at the Annual Meetings of the Association for Public Policy Analysis and Management, Washington, D.C. http://policy.rutgers.edu/faculty/salzman/steadyasshegoes.pdf. (Accessed July 7, 2014.)

Ozturk, M., and C. Debelak. 2008. “Affective Benefits from Academic Competitions for Middle School Gifted Students.” Gifted Child Today 31 (2): 48–53.

Parker, S. 1998. “At Dawn or Dusk, Kids Make Time for This Quiz.” Christian Science Monitor 90 (116): 49–54.

Download (PDF, 143KB)