Staying SMArT: Introduction, Reflection, and Assessment of an Inquiry-based Afterschool Science Program for Elementary School Students
Our public schools need more STEM infusion. Simultaneously, civic engagement in higher education opens a window for colleges to partner with local communities to inject science into affordable afterschool programs. We offer a description, reflection and preliminary assessment of an enrichment program, "SMArTeams" at Southwestern University (Georgetown, Texas). Using a pre/post-test design, our study demonstrated that elementary school students exhibited gains in confidence, experimental design, curiosity and science enjoyment in ten weeks. Surprisingly, they did not show similar gains in drawing conclusions or imagining future STEM careers. However, extending beyond survey results, reflections of SMArTeams' Day Coordinators confirmed that young students successfully presented projects and responded positively when asked about future endeavors. Our assessment identified the need for increased discussion of STEM career pathways to broaden perspectives of elementary school students. Overall, we present SMArT as a cost-effective, engaging outreach program for creating partnerships between colleges and local school districts.
Despite increased efforts, the U.S. struggles to achieve proficiency in science, math, engineering, and mathematics (STEM) education and related careers. According to the 2011 National Assessment of Educational Progress Exam, thirty-five percent of eighth graders scored below "basic" on science, and twenty-seven percent scored below "basic" on math (NAEP 2011). Of the students who scored at least "basic" in science and math, the majority did not score above "basic" proficiency (NAEP 2011). According to the 2009 Program for International Student Assessment, the U.S. fluctuates from nineteenth to twenty-sixth place (out of sixty-five countries) in science, and between twenty-sixth and thirty-sixth place in math (PISA 2009). Such rankings prompt the need to examine where and how elementary school students can make a strong start in science and math.
To encourage other universities and colleges to establish a SMArT (Science and Math Achiever Teams, hereafter "SMArT") program, we offer an in-depth look at SMArT at Southwestern University (SU), a small liberal arts college in Georgetown, Texas, with approximately fifteen hundred undergraduate students. With the elementary students traveling to Southwestern for the activity, SMArT operates as a once-a-week afterschool program for third through fifth graders. Students engage in STEM education through individual implementation of the scientific method with the guidance of an undergraduate mentor. Student-mentor pairs complete a project chosen by the student over the course of nine to ten weeks and give a poster presentation at the program's conclusion.
From Spring 2007–Spring 2011 (i.e. time of this program assessment), SMArT served approximately one hundred and five students, working with ten to twelve students each semester and in one semester running two teams simultaneously. Since 2011, the SMArT program has continued these enrollment practices and just concluded its 14th semester in Spring of 2013 (> 150 students). Participating elementary schools thus far include Rae McCoy Elementary, Dell Pickett Elementary, Everett L. Williams Elementary, Patricia Webb Cooper Elementary, Joann Ford Elementary, James E. Mitchell Elementary, and Village Elementary, all within the Georgetown Independent School District (Texas).
While we hypothesize and have anecdotally observed STEM education, we sought an additional objective and qualitative assessment of SMArT's ninth iteration. The assessment analyzed the program's routine curriculum and practices to 1) measure the success of SMArT's core objectives; 2) improve the program; and 3) present a science-based civic engagement program model for other institutions of higher learning. Our assessment included problem solving prompts, surveys, and interviews before and after the program to measure student outcomes and identify areas of improvement.
Within the public educational system, many teachers face three main struggles: (1) the pressure to "teach to the test" for state assessments (Jehlen 2009), (2) the difficulty in moving beyond lecture-classroom methods (NAEP 2011), and (3) being personally uncomfortable in teaching math (Epstein and Miller 2011). The average hours per week spent teaching core science in elementary schools dropped from three hours in 1993–1994 to only two and a third hours in 2007–2008 (United States Department of Education 2008). With its focus on test-based learning, the education system has reduced the importance of basic science with the loss of exploration and inquiry-based learning opportunities.
Afterschool programs provide a good vehicle for enrichment and additional educational benefits not found during normal school hours. For example, inquiry-based STEM projects, activities, and experiments can be performed during afterschool programs without the constraints of traditional school schedules. Additionally, children who regularly attend quality afterschool programs usually do better in school and have fewer behavioral problems (Durlak et al. 2010). Therefore, if conducted properly, afterschool programs can be used as an outlet for increasing educational enrichment.
Despite the documented benefits of structured and enriching afterschool programming, an enormous need exists. Although 8.4 million or fifteen percent of children in grades kindergarten through high school (K-12) are enrolled in afterschool programs, 18.5 million more would enroll if local programs existed (Afterschool Alliance 2009). The great need for quality afterschool programs opens a window for universities to create affordable afterschool programs within local communities to enrich K-12 education, especially in the sciences. Often tied to the socioeconomic base of the taxpayers, funding for extracurricular or afterschool programs varies widely across school districts. Reciprocal community-university partnerships can offer modest resources to support local education in districts with limited funding.
Increasing resources and innovative STEM education via college-student led organizations may inspire grade-school students to join STEM programs. Teacher recruitment may also occur among the university student participants. Through college and donor sponsored programs like SMArT, school districts and colleges can partner to fill the gap between education and experience. The smaller age gap between grade-school students and college mentors provides more immediate role models for children disinterested in or disheartened by math and science. Currently, only nineteen percent of K-12 students take advantage of available STEM-related afterschool programs when they occur in their school districts (CTEq 2012). The lack of participation in STEM programs likely arises from the cultural bias that math-related subjects and sciences are difficult or impossible to comprehend (Epstein and Miller 2011).
Furthermore, lower-income students typically encounter fewer opportunities to participate in STEM programs and understandably tend to develop less of an interest in the sciences and related careers (Epstein and Miller 2011; Museus et al. 2011). During regular school hours, only thirty-two percent of low-income students reported that teachers possessed the necessary supplies to complete lab activities (NAEP 2011). Across all income levels, only fifty-six percent of students participate in hands-on science activities once or twice a week (NAEP 2011). However, students who participated in lab exercises scored fourteen percent higher on the National Assessment of Educational Progress in science, equivalent to one grade level better (NAEP 2011). It is apparent that afterschool programs can help narrow the gap in STEM exposure.
SMArTeams – History and Mission
Responding to the community's need for better STEM education, in an effort to bolster career interest in STEM fields, Southwestern University implemented the Science and Math Achiever Team program model (i.e. SMArT) in a partnership with Georgetown Independent School District (GISD). Even before educational outreach and civic engagement started to gain a strong foothold in higher education, the basic framework for SMArT was established at Yale University in the early 1990s with founder Rowan Lockwood (Burks, personal communication). We use the term civic engagement holistically to include a wide range of activities that develop a person's sense of public responsibility and encourage a desire to contribute to the common good (Jacoby 2009). SMArT seeks to provide an alternative to the negative perception of STEM by providing elementary school students an engaging, dynamic, and fun inquiry-based learning experience. SMArT brings the process of discovery and the scientific method to each participating elementary student in a personal, individual, non-competitive format. The program fuels children's innate scientific curiosity, which—although this is not the direct intention—could develop into an interest in STEM subjects and potential future STEM-related careers.
In partnership with the local public schools, SMArT allows children across socioeconomic backgrounds to participate in an extracurricular, individualized, interactive science program, giving many children from lower-income families an otherwise unlikely experience. The SMArT model relies on three positive factors identified by the ASHE Higher Education Report to promote STEM education success in racial and ethnic minorities: (1) providing early exposure to STEM careers; (2) increasing STEM interest; and (3) bolstering self-efficacy in STEM subjects (Museus et al. 2011). SMArT integrates these three factors into a one-on-one mentoring program between an undergraduate mentor and an elementary school student, where the pair pursues questions driven by the elementary student's interests. The program indirectly touches on STEM careers through casual discussion between mentors and students. SMArT excels at increasing STEM interest by encouraging the students to choose their individual projects based on personal interest, create a project design, and take ownership of their projects at the end of program project presentation session. The Achievement Party, where the students present their projects, remains an integral cornerstone of the program. It celebrates the students' increased scientific knowledge and project accomplishments, and bolsters their self-confidence in math and science.
SMArT began at SU in the spring of 2007 and has fielded a "team" every semester since its inception. Backed by university support and a recent endowment (expenditures amount to approximately $1200/semester), SMArT offers free programming to third through fifth graders enrolled in the local school district's afterschool enrichment program, Extended School Enrichment (ESE). ESE charges a modest tuition for its services, but its demographics reflect the substantial diversity found within GISD (GISD 2011). While students have to pay to enroll in ESE, it is an economical and educational childcare option, and students incur no additional costs when participating in the SMArT program. The partnership between GISD, ESE, and SU facilitates parental consent, liability concerns, and transportation to SU from the elementary schools. The SMArT program primarily depends on a four-member team: a faculty advisor, a university civic engagement coordinator, a budget contact person, and a day coordinator.
Day coordinators train the mentors, recruit participants, run the one-hour weekly SMArT sessions, gather project supplies, and organize the end-of-program Achievement Party. Usually, only one day coordinator works at any one time, and the position gets handed down from graduating to incoming student leaders. After hearing a presentation about the program by the SMArT day coordinator, interested elementary students receive program applications. The application includes questions about the child's interest in science, favorite topic in science or math, past STEM experiences, and invention ideas. Evaluation of the applications is based on the child's creativity, expression of interest, ability to commit to attending the weekly sessions, and whether he or she has had the opportunity to participate in a similar program before.
The program optimally operates with ten to twelve elementary student-college mentor pairs, making a selection process necessary. Applications that exude enthusiasm and independence receive the most positive reviews. We also check with ESE site coordinators to avoid attendance issues. We want as many new students to have the opportunity to participate as possible, so we also prioritize new applications. The program rotates campuses every semester to give students across the district enrolled in ESE a chance to participate within their third through fifth grade window. To increase interest in attending college, and to facilitate access to laboratory resources, SMArT meets in a general biology lab at SU.
Each elementary student-college mentor pair works on an inquiry-based project or experiment of the elementary student's choice for one hour, once a week, for nine to ten weeks to improve the student's working understanding of the scientific method, encourage self-confidence in science, and increase interest in scientific careers. Students choose projects based on: (1) personal interest; (2) brainstorming with their mentors using STEM books and internet research; and (3) feasibility as discussed with the day coordinator and faculty advisor. Each week, the one-hour routine of the program includes five minutes of snack time at the beginning, forty-five minutes of project time, and concludes with a brief discussion (i.e. five to ten minutes) of what each pair did that day. The brief discussion allows the elementary school students to practice speaking aloud about their projects to prepare for the end-of-program Achievement Party.
Projects range across the scientific disciplines, varying in complexity based on the child's grade level, interest, and critical thinking skills. SMArT entertains its fair share of basic volcanoes, dissections, and robots, but each of the projects includes enough depth to last several weeks (Table 1). At the Achievement Party, during the last week of the program, the elementary student-college mentor pairs present their projects to the university, the community, and the children's parents in a celebratory, noncompetitive format. The elementary students, supported by their college mentors, stand by their posters with demonstration items in an open hall to explain their projects and answer questions from SU students, faculty, and parents about their posters. No ranking or prizes accompany the projects. The students do not present formally, but instead answer questions from a small circulating audience. We end the celebration with cake, a slide show of mentor-student pairs, and a short speech praising the students for their leaps in learning. We believe the noncompetitive atmosphere of the Achievement Party assuages presentation anxiety, supports a collaborative atmosphere within the program, and encourages students to focus on their individual explorations and learning.
With IRB approval (IRB number: F10-27), ten GISD-ESE elementary school students in the SMArT program (ages 8–11, grades third–fifth) participated in the Spring 2011 assessment with written parental consent. Researchers used three anonymous assessment tools: a written prompt, a written survey before and after the program, and an individual interview after the program. We administered the written prompts individually to the students before and after the program to assess scientific problem solving skills, asking the student to design a hypothetical experiment to test a simple question. The pre-program prompt asked students to think about "what makes plants grow taller?" and the post-program prompt posed the question "what makes a paper airplane fly further?" To facilitate multiple modes of communication, researchers encouraged the students to draw, diagram, and write out their responses. They did not limit the time available to complete the task. Mentors clarified student questions or read the prompt aloud but did not help the students answer.
We evaluated the prompts with a positive/negative score based on the presence or absence of seven scientific method objectives (Figure 1). We administered the same individual written survey pre- and post-program (Figure 2), with five additional questions in the post-program survey (Figure 3) that explored student attitudes towards science, the scientific method, STEM career interests, and SU as the place where we met for SMArT. Students responded through a positive, negative, or neutral agreement scale towards positive statements. We administered an interview individually post-program. The audio-taped interview lasted ten minutes. We transcribed the results for later analysis. The students responded to eight open-ended verbal statements similar to the survey. We summarized six of the eight questions into agree/disagree (Figure 4). We included two other two questions, "What was your favorite part?" and "What was your least favorite part?" to look for consensus among participants in the program.
Lastly, to gain a college student perspective about the program, we asked our five day coordinators (Figure 5) to reflect on what they observed about the student learning of the elementary school students (Table 2). Over the six years that the program has been running, SMArT has employed five day coordinators (Frankel, Navaira, Mohammed, Liebl, Lowther) to help manage the program. While the directors (Burks, Pukys) managed the financial and logistical arrangements with GISD-ESE, the day coordinators did everything from recruiting elementary and college students to helping brainstorm project ideas to gathering supplies and running the day-to-day activities. All day coordinators (Table 2, Figure 5) started off as mentors working with an individual elementary school student. The day coordinators provide a longitudinal perspective on SMArT to compare with the single semester assessment study conducted by our assessment researcher (Roberts).
Experimental design prompt
Before the program year started, all elementary students received a prompt to design a hypothetical experiment to answer the prompt question. Throughout SMArT, the elementary students successfully designed and completed their own experiments. As a result, in the post- program prompt exercise, all participants successfully described an experimental procedure, compared to only seventy percent before the program (Figure 1). The percentage of students who effectively described how to interpret results also increased. [Figure 1 about here] Recognition of problem and variables remained the same, while the description of variable measurement, description of tools for data collection, and secondary evaluation of the experiment prompt demonstrated little or no improvement (Figure 1). Secondary evaluation of the experiment included what the student would hypothetically change, do differently, or what aspect he or she would evaluate in more detail in the hypothetical "next" experiment.
According to the survey, before the program, most students expressed either uncertain or negative responses in how to use science to answer a question about the world (Figure 2). After the program, confidence in using science to answer a question and in designing experiments improved forty and sixty percent, respectively (Figure 2). Contrary to expectations, uncertainty towards science-related careers increased after the program by thirty percent (Figure 2). Enjoyment of science remained high throughout the program (Figure 2).
Post-Program Survey, Additional Questions
Ninety percent of students said their confidence level towards science in school improved, and they enjoyed science more. Similarly, most students said their curiosity about science increased (Figure 3). Surprisingly, most students expressed no change in interest in future science activities, or the amount they talk about science outside of school (Figure 3).
Post-program interview responses mirrored the responses in the post-program survey's additional questions. All students enjoyed participating in SMArT and working with their mentors (Figure 4). Similar to the post-program survey, most students reported that SMArT had a positive impact on their science experience at school and on their attitudes about science (Figure 4). Almost all students expressed increased knowledge about science, how to develop experiments, and how to answer questions (Figure 4). When asked about their favorite or least favorite aspects of the program, eighty percent of students pointed towards working on their projects as their favorite part, and a high proportion (forty percent) of students failed to identify a least favorite part. A couple of students identified poster making and waiting for their mentors to arrive so they could get started on that week's portion of the project as their least favorite time spent in SMArT.
Day Coordinator and Director Reflections
Day coordinators and directors of SMArT attest to the success of the program in ways that our quantitative study could not (Figure 5, Table 2). Directly interacting with the elementary students on a week-to-week basis, those directly involved in the program witnessed the intellectual and critical thinking leaps the students made. Whether or not the experience of SMArT influences future outcomes in STEM career choices is difficult to determine. The day coordinators' and directors' testimonies make it apparent that SMArT works on a deeper level to improve the quality of the education the participants receive by challenging them to think harder about the world around them.
Partnerships like SMArT between institutions of higher education and local school districts benefit elementary students in their cultivation of long-term critical thinking skills by providing them college science student role models. Our first detailed assessment evaluated the program's success in teaching the scientific method to elementary school students (moderately successful), increasing STEM career interest (unsuccessful), and raising student self-confidence about learning science (highly successful). Even though the sample size of our assessment is not large enough to draw statistically significant conclusions, we can identify trends of success and areas of improvement for our internal program development and advocate for the use of SMArT as a model for other STEM civic engagement programs. Based on our assessment and anecdotal testimony, we sincerely believe SMArT exemplifies a strong model for developing student inquiry.
With STEM careers comprising one-fifth of all U.S jobs, STEM education needs to increase starting from elementary through high school (Morella and Kurtzleban 2013). Even though seventy-five percent of high school girls are interested in STEM careers, women make up only one-quarter of the STEM workforce (Morella and Kurtzleban 2013). Clearly, increasing the caliber of STEM education, student confidence to enter STEM careers, and supportive role-models like those in SMArT represent necessary actions to reinforce improvements in the way education emphasizes science and math. In an era driven by STEM education, SMArT provides an engaging, hands-on STEM experience that could be better targeted to increase interest in related careers.
We attribute the increased participant understanding and implementation of the scientific method, especially experimental procedure description and result interpretation, to the SMArT model. The completion of a project represents the cornerstone of the SMArT formula. The elementary student-college mentor pair either designs an experiment, an interactive model, or an experiment-model combination. The inquiry-based learning model encourages the participants to follow the scientific method by recording their hypotheses, procedures, observations, results, conclusions, and future research ideas.
While the correlations between physically designing experiments, completing an experiment, and understanding the scientific method appear clear, it remains uncertain whether the exploration of a topic through modeling reaches the same educational goals as an experiment. In a study by Vattam et al. (2011), modeling successfully illustrated and taught the structure, behavior, and function of a relationship better than a lecture. Such results support SMArT's approach of using models for teaching difficult, abstract, or unreasonably expensive subjects. For example, past SMArT topics that necessitated modeling included dolphin versus whale anatomy, large-scale robotics, how horses run, tectonic plates, the life cycle of a bird, and jungle camouflage. Modeling can include actual dissections of commercially available specimens that have similar function or body structure to the subject of interest (such as a dogfish shark instead of a great white shark), constructing robots from kits, growing crystals, and doing research to construct ecological biome models. Because we administered the written assessment tools anonymously, we do not know whether the two students who exhibited no improvement in interpreting the prompt's experimental results did a model or experiment-based inquiry model.
According to the pre/post-test prompt results, participants failed to demonstrate secondary evaluation of their hypothetical experiments in the prompt exercises both before and after the SMArT program. We defined secondary evaluation as explaining how they would alter their experiment if allowed to do it again. Possible confounding issues of this specific objective include the age of the participants, the indirect expectation of secondary evaluation in the prompt, and the lack of a separate secondary assessment trial. During the Achievement Party, personal accounts and anecdotes by mentors, day coordinators, and faculty (Table 2) support the observed improvement of the students in their proper use of scientific vocabulary, interpreting the results of their own projects, explaining how they would alter their experiment for next time, and what they would like to learn about next.
Despite the verbal and unquantifiable observation of advanced critical thinking skills, it remains understandable that the written communication and questioning of critical thinking remains difficult for elementary school students. In a study by Wan (2010), seven-year-olds effectively described the procedure of how to make Jell-O but found it difficult to explain the reasoning behind the procedure. In a similar study, third graders struggled to present a sophisticated, abstract explanation about magnetism, relying mostly on intuition and observations (Cheng and Brown 2010). Both studies support our assessment findings; the intuition to think critically lies just below the surface in this young age group. Our participants excelled in describing a procedure and interpreting results in the post-program prompt, but struggled to reach and communicate the next step of analytical thinking. Similarly, none of the assessed participants demonstrated secondary, abstract evaluation in the prompt exercises.
The similarities between these two studies and our results support the hypothesis that secondary evaluation and higher-level analytical thinking depend on age. The SMArT curriculum revolves around each individual elementary student. Depending on the students' educational and critical thinking level, the mentor guides the project to be more or less complicated, so as to create an appropriate challenge. As a result, there can be large differences in complexity between individual children and their ability to critically and abstractly analyze their projects. In future assessments, we intend to perform an assessment at the end-of-program Achievement Party; an audio recording of their project explanations, for example, would provide a concrete opportunity for the students to apply secondary, abstract evaluation of an experiment. Based on informal observations over the years, we noticed that some of the younger students struggled with this challenge while more of the older participants answered more confidently when asked to further analyze their experiments during poster presentations. We believe most of the SMArT participants execute some level of secondary evaluations of their projects and should be assessed through direct oral questions like those the students receive at the Achievement Party poster presentations.
The assessment results concerning the second major goal of SMArt, to increase confidence in STEM subjects and interest in STEM-related careers, illuminated areas to improve within the program. Student interest in participating in science-related activities in the future remained fairly constant for the duration of the program. We attribute this partly to the non-random nature of the SMArT program, given that participants have already demonstrated an enthusiasm for science in their applications. Unexpectedly, the elementary students expressed a higher degree of uncertainty towards STEM careers after the program than before, despite a demonstrated increase in STEM subject interest.
We expect that the short duration of the program, as well as the students' young age, contributed to the perceived limited impacts of the program on STEM career interest as extrapolated from the survey results. However, personal experience of day coordinators who talked with the students helped solidify our assertion that the program does extend students' experiences of science outside the classroom. While the students enjoyed the laboratory experiences, we believe the students did not make connections between SMArT activities and STEM career paths. In the future, short STEM career presentations to the elementary students by their mentors or science faculty may introduce and encourage science-related careers; for example, mentors could give short presentations on their personal educational, and career goals, and current research project.
Overall, the SMArT program stands as a strong model for colleges and universities seeking to make a lasting impact on the next generation of STEM professionals in their community, by providing a unique hands-on experience and dynamic exploration of the scientific method, the core foundation of any STEM education. Ahead of its time, the SMArT model supports the 2013 Next Generation Science Standards (NGSS) issued by the National Research Council, which has been adopted by twenty-six states (Cardno 2013). Like the SMArT model, the NCSS encourages the use of science and engineering hypothesis-testing learning environments where students engage in problem solving activities, modeling, investigations, data analysis, and math skills (Cardno 2013).
As excitement for STEM education reaches state-level curricula, it will take time to teach teachers how to implement the new classroom standards and increase the number of educators comfortably teaching STEM. Currently less than half of elementary school teachers express confidence in teaching science, and only four percent feel prepared to teach engineering (Banilower et al. 2013). Joint programs like SMArT between post-secondary institutions and school districts can amend the disparity between the science and math education students sometimes receive at school, and what they need if they are to succeed. Some school districts already turn towards business professionals, community specialists, and universities to bring science to life in the classroom through real-world examples (DeNisco 2013). Programs such as SMArT demystify science and math, bringing a highly diverse and opportunity-rich field to the fingertips of elementary students. There will always be a place for civic engagement programs like SMArT to bring the enthusiasm of college students to elementary students, but perhaps we need programs like SMArT now more than ever to bridge the education gap.
SMArT began as a project through a Brown Fellowship for Innovative Teaching supported by the Provost's Office at Southwestern University. We acknowledge additional internal financial support from The Paideia Program, The Office of Civic Engagement, and most recently, external funding from the W. D. Kelley Foundation. Linda Southwick and Christy Schaller in the Department of Biology graciously help with supply acquisition and project development. Sarah Hensley also helped get the SMArT program up and running in 2007. We appreciate the efforts of many Natural Science faculty at Southwestern to provide advice for SMArT projects. Jan Williams and the entire staff of Extended School Enrichment make the partnership possible. We appreciate the comments of Dr. Sarah Brackmann, SU's Director of Civic Engagement, on this manuscript.
About the Authors
Afterschool Alliance. 2009. "America after 3pm: The Most In-depth Study of how America's Children Spend Their Afternoons." http://www.afterschoolalliance.org/documents/AA3PM_National_2009.pdf (accessed July 28, 2013).
Banilower, E.R., P.S. Smith, I.R. Weiss, K.M. Malzahn, K.M. Campell, and A.M. Wels. 2013. "Report of the 2012 National Survey of Science and Mathematics Education." http://www.horizon-research.com/2012nssme/research-products/reports/technical-report/ (accessed July 30, 2013).
Cardno, C.A. 2013. "New K-12 Science, Engineering Standards Unveiled." Civil Engineering 83 (6): 26-27.
Cheng, M.F, and D.E. Brown. 2010. "Conceptual Resources in Self-developed Explanatory Models: The Importance of Integrating Conscious and Intuitive Knowledge." International Journal of Science Education 32 (17): 2367-2392.
CTEq, Change the Equation. 2012. Vital Signs, Reports on the Condition of STEM Learning in the U.S. "Lost Opportunity, Few U.S. Students Participate in STEM Out-Of-School Programs." http://changetheequation.org/sites/default/files/CTEq%20Vital%20Signs%20Lost%20Opportunity.pdf (accessed July 28, 2013).
DeNisco, A. 2013. "Survey: Districts Need Vision for Teaching Science, Math." District Administration 49 (5): 20.
Durlack, J.A., R.P. Weissberg, and M. Pachan. 2010. "A Meta-analysis of After-school Programs that Seek to Promote Personal and Social Skills in Children and Adolescents." American Journal of Community Psychology 45 (3-4): 294-309.
Ehrich, T., and B. Jacoby. 2009. Civic Engagement in Higher Education: Concepts and Practices. San Francisco: Jossey-Bass.
Epstein, D., and R.T. Miller. 2011. "Slow off the Mark: Elementary School Teachers and the Crisis in STEM Education." Educational Digest 70 (1): 4-10.
GISD, Georgetown Independent School District. 2011. Georgetown ISD School Profile as of 10/28/2011. http://www.georgetownisd.org/cms/lib/TX01001838/Centricity/Domain/60/GISD%20Student%20Profile%2010-28-11.pdf (accessed January 6, 2014).
Jehlen, A. 2009. "NCLB: Is it Working?: The 'Scientifically-based' Answer." National Education Association Today Magazine. http://www.nea.org/home/20755.htm (accessed August 5, 2013).
Morella, M., and D. Kurtzleban. 2013. "The State of STEM." U.S. Digital Weekly 5 (26): 11.
Museus, S.D., R.T. Palmer, R.T. Davis, and D. Maramba. 2011. "Racial and Ethnic Minority Student Success in STEM Education." ASHE Higher Education Report 36 (6).
NAEP, National Assessment of Educational Progress. 2011. "The Nation's Report Card." http://nationsreportcard.gov/science_2011/science_2011_report/ (accessed August 19, 2012).
Organization for Economic Co-operation and Development. 2010. Program for International Student Assessment 2009 Results: Executive Summary. OECD.
U.S. Department of Education. 2008. National Center for Education Statistics. "Schools and Staffing Survey." http://nces.ed.gov/surveys/sass/tables/sass0708_005_t1n.asp (accessed July 23, 2013).
Vattam S.S., A.K. Goel, S. Rugaber, C.E. Hmelo-Silver, R. Jordan, S. Grey, and S. Sinha. 2011. "Understanding Complex Natural Systems by Articulating Structure-Behavior Function Models." Educational Technology & Society 14 (1): 66-81.
Wan N. 2010. "Jelly Making: Children's Science Discourse and Thinking." Teaching Science: Journal of the Australian Science Teacher's Association 56 (3): 13-18.