Academic Quality Improvement Plan

Assessment Plan

WeaveOnline Program Learning Outcomes

Directly addressing preparedness, the Program Learning Outcomes were developed in response to a major initiative undertaken by the University in response to SACS (see next section). In combination with the surveys above, the Program Learning Outcomes provide a mechanism to evaluate the achievement of the Program Educational Objective.

The current process for establishing and revising Program Learning Outcomes began in 2003 with a university initiative in response to SACS accreditation.  At that time the Office of Institutional Effectiveness (IE) was established to guide all programs in the development and implementation of program learning objective plans and reports.  With help from IE, the faculty of the School of Construction developed a minimum of five major objectives with two measures each, established targets for each measure, and linked the measures to various data that could be extracted from courseware and surveys.

Individual instructors charged with the facilitation of each course developed learning outcomes.  Instructors were also tasked with developing course outcomes that demonstrate a student mastery of the required performances, as well as being specific and measurable in nature.  Once learning outcomes had been developed, they were reviewed by at least one other instructor within the program.  Typically, this instructor was one that had been identified as teaching a course that is immediately subsequent to the former course.  It was the intent of this process for instructors to provide feedback toward the development of a comprehensive range of learning outcomes that will facilitate a cohesive transition into higher-level program curriculum.  The program coordinator also provided a support role in the development of course objectives.  The coordinator reviews all revisions of course objectives to determine if they actively contribute to the advancement of the program’s overall goals and objectives.

 Assessment Plan (PDF)

Continuous Improvement Initiatives

The Engineering Technology programs in the School of Construction underwent a 6th year ETAC-ABET accreditation visit in fall 2010.  From that visit, it was apparent that the program objectives in WeaveOnline did not provide adequate resolution from course level to program level.  The organization of supporting materials and student samples of work was also extremely difficult to collect and organize in a meaningful manner.  It was decided then to reorganize the program learning outcomes to exactly map to the ETAC-ABET general and program specific criteria with direct linkages from each course in the program that supported a particular criterion.  For the Architectural Engineering Technology program, these criteria are:

Criteria Specific to Construction Engineering Technology

Associate degree programs (and our corresponding lower-division) must demonstrate that graduates are capable of:

a.       utilizing modern instruments, methods and techniques to implement construction contracts, documents, and codes;
b.      evaluating materials and methods for construction projects;
c.       utilizing modern surveying methods for construction layout;
d.      determining forces and stresses in elementary structural systems;
e.       estimating material quantities and costs;
f.       employing productivity software to solve technical problems
Baccalaureate degree programs must demonstrate that graduates, in addition to the competencies above, are capable of:

a.       producing and utilizing design, construction, and operations documents;
b.      performing economic analyses and cost estimates related to design, construction, and maintenance of systems in the construction technical specialties;
c.       selecting appropriate construction materials and practices;
d.      applying principles of construction law and ethics;
e.       applying basic technical concepts to the solution of construction problems involving hydraulics and hydrology, geotechnics, structures, construction scheduling and management, and construction safety; and
f.        performing standard analysis and design in at least one recognized technical specialty within construction engineering technology that is appropriate to the goals of the program.

  

Faculty then mapped each of their course objectives to the ETAC-ABET criteria using a listing of the assessment tools/methods for assessing each objective/criterion.  This provided evidence of which courses in the program inventory were supporting any given ETAC-ABET criterion and also provided a simple index system for staff to organize supporting materials by criteria for inspection.  And, while ETAC-ABET only requires summative evidence, this approach easily supports formative inspection of the curriculum.

WeaveOnline Objectives reflect the exact ETAC-ABET criteria with two measures for each criterion: one direct and one indirect.  The direct measures are the aggregated assessments for all student work samples (projects, exams, quizzes, papers) as determined by the faculty in their mapping exercise.  The indirect measures will be the graduate exit surveys and alumni surveys rewritten to also reflect the ETAC-ABET criteria; these have not yet been implemented for this cycle. 
 
Faculty then reported their findings for each section of their courses for fall 2010 and spring 2011.  At the course level, it was decided to begin this process using targets of 80% of students would achieve 70 (out of 100) on the assessments.  The findings were separated by program area the course might serve; for example, a course might have Architectural Engineering Technology (ACT), Construction Engineering Technology (BCT), Industrial Engineering Technology (IET), or other (OTHER) students.  These findings were organized in a master spreadsheet organized so that the findings for each criterion for each program by semester and by delivery type (online or face-to-face) could be summed.  This provides the total number of student samples for each criteria meeting the performance target versus total number of students being assessed.  The findings for each criterion were then entered in WeaveOnline as annual summation values as well as being reported by semester and by type of site or delivery method.  This system allows the program faculty to see the impact of their courses as a whole and individually on each criterion. 

Beyond the reporting system for SACS, ETAC-ABET and ACCE, the faculty now have a systematic approach to evaluate each of their course objectives using the defined performance target levels to look at weaknesses in each course, develop action plans at the course level, and “close the loop”.

 

Closing the Loop BCT
At the program level, all performance targets were met.  In the Construction Engineering Technology (BCT) program, this is represented by 22,123 student work samples (out of 24,768) that were evaluated as better than or equal to 70 (out of 100).   The percentage of samples better than or equal to 70 is 89% which exceeds our stated level of performance of 80%.   These findings were derived from 21 of 22 courses in the curriculum; the findings from the remaining course is still being pursued but was taught by an instructor that lost all data from hard-drive failure. (This issue will lead to an improved reporting system). 

Since the data is driven from the ground up (that is, from the faculty), the value of this assessment approach is that all faculty are involved rather than a select few as previously.  The faculty are able to review their course level findings with respect to either the ETAC-ABET  criteria or the course objectives (which are generally more important to them).  Although we have met all performance targets at the annual program level, there are findings (also reported in WeaveOnline) where the semester based report for either face-to-face or online might not have met the performance target.  It is a simple matter to drill back down to the course level and determine which assessment tools the students were having difficulty with. 

When the faculty submitted their findings, they were asked to provide an assessment of any finding that went below the 80% threshold and develop action plans as needed.  In some cases, the issue was too few students in a section; these sections did not require an action plan but would be monitored.  Sections with significant student numbers that had assessments below targets were added to the action plan section in WeaveOnline.

 

Employer survey of interns
The Program Educational Objectives (developed from the Mission statements) were designed to target the industry shortage in graduates going into management or management support positions.  After 2004, the programs began requiring internship and now use employer surveys as one measure of satisfaction in the “product” they witness. 

Exit and Alumni Surveys
The School of Construction (SoC) also correlated the exit and alumni surveys to provide longitudinal analysis of our graduates; this provided a mechanism to evaluate satisfaction of our primary constituent with respect to items such as critical thinking and preparedness.