Lifelong Learning and Information Literacy Skills and the First Year Engineering Undergraduate: Report of a Self-assessment – Meagan C. Ross with Michael Fosmire, Ruth Wertz, M.E. Cardella, and S. Purzer
Meagan C. Ross, discussed first year IL and assessment projects at Purdue University. This particular project was funded by an Engineer 2020 grant. Ross, et al found engineering students self-reported lack of gains in “lifelong learning.” ABET Outcome 3i Lifelong Learning and information literacy are connected. Librarians found they had a hard time teaching IL skills as well as a difficult time assessing these skills. Looking to develop an easy to administer test for librarians and engineering faculty to use. Guglielmino developed a self-directed learning readiness scale (SDLRS), a SDLRS for Nursing education, as well as Shinichi, et al who created a more generic 13 question instrument.
Purdue wished to develop an instrument that would be more focused on engineering. They also desired to bring in Kuhlthau (2004) Information Search Process (ISP). First year students who took their information skills assessment reported being good at task definition, citation, reflection/self-assessment. The weakest areas included exploration of alternative sources, ability to locate information effectively, and these correlate with Kuhlthau’s ISP.
Junior level data shows they are more humble than the first year students. Instructors created an authentic “memo” assignment where they found one weak area for the juniors was citing sources.
They are looking for partners, they have designed three instruments and will be further developing them. They are looking for partners, so feel free to contact them. See also other presentations by this team at ASEE.
Implications
• Future work, continue developing instrument (circumvent ‘novice effect’)
• Librarians should address beginning part of ISP
Embedded Assessment of Library Learning Outcomes in a Freshman Engineering Course
Larry Schmidt and Melissa Bowles-Terry from University of Wyoming described their experience with IL inclusion within their engineering first year courses. Students are required to perform research on an assigned topic such as autonomous robots then work on a related engineering challenge. Librarians have one-short session and they now have electronic classroom. Short in-class assignments, but they did not know what students were coming away with. They narrowed down to 3 learning outcomes: identifying appropriate research databases, using appropriate vocabulary/keyword choices, and differentiating source types. Started with pre-test (online form), developed worksheet and rubric to assess student keyword choices, and a post-test (online form) to assess databases choices and defining scholarly sources.
Using a constructivist learning approach, they start where the students are by teaching Google then move to Wonderwheel and Scholar, then to an engineering database. Showing search process from general to more specific, and how to modifying search terms.
On the pre-test, almost half of 192 responses had not ever used a database and half were familiar with Academic Search, but this is offered in the Wyoming schools/libraries. For learning outcome 1 (databases) librarians categorized into beginning, developing, exemplary based on the students’ reasons for choosing a specific database. For keywords, librarians created a small rubric, based on quantity of keywords (will be moving to quality in the future, this rubric will be revised). They found most of the students with beginning proficiency in keyword selection were non-native speakers. For learning outcome 3 (source differentiation) students were asked to identify characteristics of a scholarly article, they found the students did not do well. Self-reported confidence levels increased during the 1-hr session. Future idea: taking research paper samples and matching with student confidence levels. They would also like to give a presentation to the engineering faculty with these findings.
The Research Studio: Integrating Information Literacy Into a First Year Engineering Science Course
C. Michelle Baratta, Alan Chong and Jason A. Foster
At University of Toronto, two engineering design instructors worked with Baratta, a librarian to develop new active learning method of incorporating IL skills, which they call a “research studio.”
Students have three major design projects within this course. One project has a research assignment with real focus, to design pedestrian bridge to cross a ravine in Toronto. Students need to incorporate technical load/structural engineering concepts and go beyond and think about issues of usability and sustainability. This assignment involves site visits as well as secondary research. Instructors wanted to introduce students to reference handbooks, building codes so they brought in the librarian to assist.
LOGISTICS: Three hundred students visited the library over 3 days, one hundred at a time. They were required to visit up to six stations per team (minimum of 3 spending 30 minutes at each stop). Part of the goal is to foster mutual interdependence, not all teams visited the same stations. Instructors developed a 28 page handout.
STATION EXAMPLE: Evaluating Information
Students were asked to view and evaluate websites about specific bridges. Reflections questions such as “would they use these web sites in their daily lives,” or “would they use this site for an academic research project.”
At the Search Strategies stop students constructed searches and learned about/constructed Boolean searches. For non-traditional sources, students wandered the library to find examples of non-traditional sources.
Instructors found students are now providing longer reference lists, more credible sources, but they feel that the students still do not strategize (but this is a program level issue). Question about how the students are using the references, but have not had opportunity to do this yet.
In the future they will be adding codes/standards, trying to slow students down, reducing structure/overload (like their 28-page handout), and they plan to “gamify” this activity. Students did not like the mutual interdependence, they wanted to know the information themselves. Progressive disclosure based on attainment of skill, moving from using a screwdriver to a Dremel tool.
Showing posts with label first_year. Show all posts
Showing posts with label first_year. Show all posts
Tuesday, June 28, 2011
Tuesday, June 16, 2009
Assessment of IL at Smith College: ABET, ACRL, & within First Year Engineering Course
Assessing Information Literacy In Engineering: Integrating A College-Wide Program With Abet-Driven Assessment
Donna Riley, Smith College, described how they formalized their information literacy program in fall 2003 and are moving towards an institutional-wide program that incorporates assessment of discipline-specific measures. At first they tapped into the first year writing intensive course. Their IL program has grown into a discipline-by-discipline curriculum-integrated approach based on ACRL information literacy standards and science and technology standards. Around 40% of the departments have developed department-specific IL outcomes already, others are working on theirs. They are now at phase two and collecting data to assess student learning. The engineering faculty wanted their outcomes to dovetail with ABET criteria so they have hybridized the ACRL/ABET. ILST performance indicators are highly detailed and ABET outcomes are broader so faculty needed to design an assessment plan that could work for both. The literature shows that each institution develops their own outcomes even though there have been some papers, including some within ASEE-ELD in past years that have dealt with alignment, or mapping ABET to ACRL standards.
At Smith, they decided to map their own standards with ABET/ACRL in various focus areas related to information literacy. First mentioned is lifelong learning which Smith has developed more detailed performance goals that are measurable. The second area of focus on is ethics as stated in ABET broadly as “an understanding of professional and ethic responsibility” which related to ACRL 4/ILST4. They developed their own outcome and again, performance criteria that embodies this. Communication is the third focus area Riley mentioned and showed specific performance criteria that includes “student exhibits clear writing style,” etc, see paper for details. The final area of focus for Smith Engineering is experimentation which does not map well with ABET so they added wording into their performance criteria “finding and using information” in addition to “data.”
Riley discussed how their use of e-portfolios allows for assessment of their performance indicators in the aforementioned categories. Student assessment also occurs within courses, for instance students produce final portfolio instead of taking a final exam in her course. At the program level, assessment occurs after sophomore year by review of the portfolio and later near graduation a panel of faculty review the e-portfolios. Possible evidence for IL: annotated bibliographies, ethics case analyses related to information, reports from design projects, and so on.
Riley thinks that ABET should revise 3(b) to include language that addresses need for information literacy and makes suggestions for how they could do this in their paper.
Riley followed up by presenting another paper, Integrating Information Literacy Into A First-Year Mass And Energy Balances Course, co-authored Smith College librarian Rocco Piccinino. Smith’s curriculum-integrated approach to IL is sequenced throughout students college career, however this paper focuses on one specific course, a first year second semester course which is required of all engineering majors. Course objectives include engineering calculations, mass/energy balances, as well as engineering ethics, and information literacy which revolves around a life cycle assessment project.
Riley assigned a reading and held discussion with her students prior to the library research session. The reading (Graham, L. and Metaxas, P.T. (2003). “Of course it’s true, I saw it on the Internet”: Critical thinking in the Internet era. Communications of the ACM 46 (5):70-75.) is about students’ performance on an information literacy test showed that they tend to be over confident in their research abilities. Riley felt this helped her students check their own over confidence. She also asked students to do a homework assignment to practice information retrieval and access, in addition incorporated a question on her mostly content-intensive mid-term exam about IL. She mentions the importance for faculty to integrate and reinforce IL skills but throughout the course. She has her students put skills into immediate practice and makes them accountable by ensuring students are using appropriate documentation, creating annotated bibliographies, and so on in an iterative way.
Assessment of student learning included various components. First, they used a one minute paper at the end of the session, which students rated learning experience as excellent or good, mentioned that highlights were learning about databases, navigating web site, full text icon, etc. but students did not mention the in-class group activity or evaluation of sources, so they may revise assessment to determine value of these components.
Second, Riley performed focus groups and of the 24 students invited, 9 participated in 2 sessions. She had three guiding questions not specifically related to information literacy and 4 of 9 mentioned information literacy and the value to their learning and at least one related IL to critical thinking.
Third, a course survey, or student self-assessment of initial course objectives, which showed that IL was on high end, though “it wasn’t most central it made an impression on them.”
Fourth, analysis of Student Work including homework, tests and projects showed from an initial quiz where students performed poorly on information literacy after research session and projects, students showed significant improvement. Still one of their biggest difficulties was determining holding for a journal with both online and print formats.
Riley feels factors contributing to student learning include:
- Librarian inclusion
- Reinforcement by faculty member
- Iteration
- Integration – accountability across coursework
This approach may be more resource intensive but authors recommend featuring and sharing faculty work in these areas, gaining faculty buy-in and creating incentives for participation. Most important is developing relationships to make this happen.
Donna Riley, Smith College, described how they formalized their information literacy program in fall 2003 and are moving towards an institutional-wide program that incorporates assessment of discipline-specific measures. At first they tapped into the first year writing intensive course. Their IL program has grown into a discipline-by-discipline curriculum-integrated approach based on ACRL information literacy standards and science and technology standards. Around 40% of the departments have developed department-specific IL outcomes already, others are working on theirs. They are now at phase two and collecting data to assess student learning. The engineering faculty wanted their outcomes to dovetail with ABET criteria so they have hybridized the ACRL/ABET. ILST performance indicators are highly detailed and ABET outcomes are broader so faculty needed to design an assessment plan that could work for both. The literature shows that each institution develops their own outcomes even though there have been some papers, including some within ASEE-ELD in past years that have dealt with alignment, or mapping ABET to ACRL standards.
At Smith, they decided to map their own standards with ABET/ACRL in various focus areas related to information literacy. First mentioned is lifelong learning which Smith has developed more detailed performance goals that are measurable. The second area of focus on is ethics as stated in ABET broadly as “an understanding of professional and ethic responsibility” which related to ACRL 4/ILST4. They developed their own outcome and again, performance criteria that embodies this. Communication is the third focus area Riley mentioned and showed specific performance criteria that includes “student exhibits clear writing style,” etc, see paper for details. The final area of focus for Smith Engineering is experimentation which does not map well with ABET so they added wording into their performance criteria “finding and using information” in addition to “data.”
Riley discussed how their use of e-portfolios allows for assessment of their performance indicators in the aforementioned categories. Student assessment also occurs within courses, for instance students produce final portfolio instead of taking a final exam in her course. At the program level, assessment occurs after sophomore year by review of the portfolio and later near graduation a panel of faculty review the e-portfolios. Possible evidence for IL: annotated bibliographies, ethics case analyses related to information, reports from design projects, and so on.
Riley thinks that ABET should revise 3(b) to include language that addresses need for information literacy and makes suggestions for how they could do this in their paper.
Riley followed up by presenting another paper, Integrating Information Literacy Into A First-Year Mass And Energy Balances Course, co-authored Smith College librarian Rocco Piccinino. Smith’s curriculum-integrated approach to IL is sequenced throughout students college career, however this paper focuses on one specific course, a first year second semester course which is required of all engineering majors. Course objectives include engineering calculations, mass/energy balances, as well as engineering ethics, and information literacy which revolves around a life cycle assessment project.
Riley assigned a reading and held discussion with her students prior to the library research session. The reading (Graham, L. and Metaxas, P.T. (2003). “Of course it’s true, I saw it on the Internet”: Critical thinking in the Internet era. Communications of the ACM 46 (5):70-75.) is about students’ performance on an information literacy test showed that they tend to be over confident in their research abilities. Riley felt this helped her students check their own over confidence. She also asked students to do a homework assignment to practice information retrieval and access, in addition incorporated a question on her mostly content-intensive mid-term exam about IL. She mentions the importance for faculty to integrate and reinforce IL skills but throughout the course. She has her students put skills into immediate practice and makes them accountable by ensuring students are using appropriate documentation, creating annotated bibliographies, and so on in an iterative way.
Assessment of student learning included various components. First, they used a one minute paper at the end of the session, which students rated learning experience as excellent or good, mentioned that highlights were learning about databases, navigating web site, full text icon, etc. but students did not mention the in-class group activity or evaluation of sources, so they may revise assessment to determine value of these components.
Second, Riley performed focus groups and of the 24 students invited, 9 participated in 2 sessions. She had three guiding questions not specifically related to information literacy and 4 of 9 mentioned information literacy and the value to their learning and at least one related IL to critical thinking.
Third, a course survey, or student self-assessment of initial course objectives, which showed that IL was on high end, though “it wasn’t most central it made an impression on them.”
Fourth, analysis of Student Work including homework, tests and projects showed from an initial quiz where students performed poorly on information literacy after research session and projects, students showed significant improvement. Still one of their biggest difficulties was determining holding for a journal with both online and print formats.
Riley feels factors contributing to student learning include:
- Librarian inclusion
- Reinforcement by faculty member
- Iteration
- Integration – accountability across coursework
This approach may be more resource intensive but authors recommend featuring and sharing faculty work in these areas, gaining faculty buy-in and creating incentives for participation. Most important is developing relationships to make this happen.
Subscribe to:
Posts (Atom)
