Revi R iew w of Fo ort Ben nd Inde I epe ende ent Scho ool Dist D tric cts iA Achieve e Pr rogr

ram m

Submitte ed by

September

Fort Bend ISD iAchieve Program Review

Executive Summary
On February 13, 2012, the Fort Bend Independent School Districts (FBISDs or the Districts) Board of Trustees (the Board) authorized the administration and staff to transform classroom science teaching and instruction . . . using integrated emerging one-to-one handheld technology. In the fourteen months since that meeting (as of April 30, 2013), the District had expended or encumbered nearly $16 million in bond, grant, E-Rate, and general operating funds; acquired and deployed 6,300 Apple iPads; and taught science lessons to approximately 600 students in 4th, 5th and 8th grades over 60 class days. Concerns about this program, its implementation and its cost have been raised by FBISD board members and the new Superintendent, prompting this review by Gibson Consulting Group, Inc. The iAchieve program was launched in February 2012 with the initial objective of developing a software platform and a wireless network for the delivery of interactive science curricula for 2nd through 8th grade levels at fourteen FBISD schools. The program fell short of its mission due to a combination of unrealistic goals, insufficient planning and project management, lack of consistency with existing FBISD curriculum development standards, and poor contract management practices. To date, the program has succeeded in deploying 210 iPad carts (containing 30 devices each) at fourteen schools and, using E-Rate and other District funds installing equipment to enhance wireless connectivity at 34 schools. This hardware and network investment represents the vast majority of iAchieve expenditures, and will likely benefit FBISD in other ways in future years. In this context, the key findings of the iAchieve program review are presented below. Unrealistic goals. FBISD established an overly aggressive timetable for the design, testing and implementation of iAchieve. Even under optimum project management practices, the District would likely have fallen short of its projected timetable for such a complex project. Insufficient project management time and expertise. The iAchieve program was hindered by not having a qualified, fully-dedicated staff member with expertise in large-scale project management, curriculum development, and instructional technology to coordinate the various teams and contractors involved in the program. The Districts Chief Information Officer, responsible for all FBISD administrative and technology functions, was named project manager for iAchieve. The management of the iAchieve program implementation required a full-time project manager. Insufficient technology and instructional supervision. FBISD created three special project coordinator positions (SPCs) to support the program implementation, but the skill requirements on the job descriptions did not include elements required for project success. SPC job requirements reflected limited experience in classroom technology integration to oversee the complete re-writing of elementary and middle school science curricula for a new technology media. The SPCs were expected to supervise and critique the work of over 70 science teachers

Fort Bend ISD iAchieve Program Review

writing content for over 700 lessons, while working directly with software engineers to develop 100 interactive iPad simulations. The job specifications simply did not match these expectations. Poor contract management practices. FBISD employed poor contract management practices in the selection and management of its primary contractor for iAchieve platform development. Project deliverables and payments did not match the contract requirements, and highly unrealistic timetables were negotiated and agreed to by both parties. Lack of documentation to support system development progress. Project documentation was not available to demonstrate evidence of the iAchieve system development through the various phases of technology implementation including: planning, design, testing, implementation, and acceptance of the iAchieve system. Lack of consistency with FBISD curriculum standards. The methodology used in the writing the science curricula, which emphasized specific scripts for teachers and did not follow the Districts lesson-building standards, resulted in content that teachers and Curriculum Department leaders felt was unusable without substantial changes.

The remainder of this report will detail the scope of our review and our findings regarding the iAchieve program, and is organized into the following areas: Timeline of Events .................................................................................. Page 3 iAchieve Program Funding .................................................................... Page 11 Hardware / Network Implementation ................................................... Page 14 Platform Development ......................................................................... Page 18 Project Management ............................................................................ Page 28 Gibson Consulting Group, Inc. (Gibson) was engaged to perform a review of the iAchieve program for the purpose of providing the Superintendent with information regarding the history of the program and its current status. Our review included interviews of District staff and review of requested documents related to the iAchieve program. We did not conduct an independent audit of the programs finances or the Districts compliance with E-Rate regulations. Gibson conducted interviews of District staff and others involved in the iAchieve program. These staff members are listed in Appendix A. We also conducted two focus groups of five teachers and five principals, respectively, who participated in the pilot phase or lesson writing aspects of the program.

Fort Bend ISD iAchieve Program Review

Timeline of Events
The first objective of this review was to provide an account of the activities and timetable that took place since the inception of the iAchieve concept and program. The timeline in Figure 1 shows the key dates for the iAchieve program tasks as they actually occurred. This information was compiled from project documentation and corroborated with interviews of FBISD and contractor staff. Following this timeline are descriptions of the major tasks. Figure 1. iAchieve Timeline

Source: Gibson Consulting Group based on data received from FBISD

Fort Bend ISD iAchieve Program Review

Background
In January 2011, the Board approved Superintendent/Board Goal No. 3 (SBG 3) to incent the Superintendent to develop a model for virtual teaching of science as an alternative to the brick and mortar concept. The roots of the iAchieve program grew from the original vision for a science and learning center that would be a permanent facility for FBISD students and others outside the District to participate in interactive science experiences. The proposal for this Center, budgeted at approximately $25 million was presented to the FBISD Board of Trustees at the June 8, 2009 meeting. Between 2009 and 2010, a number of factors caused the Board to reconsider the Science and Technology Center concept. In May and June 2011, meetings were held at the District to discuss the virtualization of the science and learning center. The idea was to incorporate the experiences envisioned with the center in an online medium as part of the Districts science curricula. The Superintendent chaired these meetings, at which District representatives, including the Districts Chief Information Officer and Chief Academic Officer, discussed their ideas for science simulations with an external consultant from Louque & Associates. Louque & Associates had previously provided consulting and character training programs pursuant to the Districts certification as a national school of character in 2011 by the Character Education Partnership. At these meetings, the company principal introduced associates who had experience developing software applications which might be helpful in advancing the virtual science center concept.

Timeline of Events
August 2011 1. Concept presented to the Board of Trustees: August 22, 2011 Although it did not yet have the iAchieve brand, the concept of the Teaching and Learning Prototype for interactive science instruction was first presented to the FBISD Board on August 22, 2011. The rationale for the program was similar to that supporting the bricks and mortar science and learning center to address lower student academic performance in science and to utilize new technologies to effectively engage 21st century students. At the August meeting, two science teachers demonstrated to the Board how the interactive program would function in the classroom, with the teacher and students utilizing hand-held electronic devices that can interact with other classroom devices such as smartboards. A simulation of the phases of the moon was demonstrated as part of this presentation. Feedback from board members was positive.

Fort Bend ISD iAchieve Program Review

September 2011 2. Contract awarded to Louque & Associates: September 15, 2011 In September the Board approved a contract with Louque & Associates for $135,000 to develop the prototype system to pilot the interactive lessons in 4th and 8th grade science classes. This contract was paid for by the American Recovery and Reinvestment Act ARRA funds. 3. Platform development RFQ issued: September 16, 2011 Also in September, FBISD issued a Request for Qualifications (RFQ) for the development of a fully functional interactive learning platform. FBISD senior leadership demonstrated the capabilities of the interactive science learning system at a meeting of the State of the Schools in September 2011. The content for this demonstration was similar to that presented to the Board in August. October 2011 Four vendors submitted responses to the RFQ by the October 14th submission deadline. November 2011 4. Purchase of iPads approved by Board of Trustees: November 11, 2011 At its November 22, 2011, regular meeting, the Board approved $250,000 for the purchase of Apple iPads for use in the pilot project (4th and 8th grade science classes) for the spring 2012. December 2011 5. Board of Trustees received project progress report: December 12, 2011 At its regular meeting in December, the Board received information regarding the following planning for of the program: Issuance of the RFQ to solicit information from vendors to develop and deploy the platform Re-evaluation of wireless deployment options Development of cost analyses of hardware and software for District infrastructure, curriculum, and the platform necessary to support the program Creation of a timeline for the project

Fort Bend ISD iAchieve Program Review

January 2012 6. George Foundation approves $1 million grant for platform development: January 24, 2012 The key aspects of the plan were presented to the Board in December with the expectation that the program would be presented to the full board for consideration at the January 2012 regular meeting. This timing was contingent on one component of funding for the project, a grant from the George Foundation (the Foundation). Approval by the Foundation of the $1 million grant after the January Board meeting (January 24, 2012) necessitated the delay in presenting the program for FBISD board approval until the February 2012 board meeting. February 2012 7. iAchieve program approved by Board of Trustees: February 13, 2012 At its February 2012 meeting, the Board received a presentation on iAchieve. This is the first instance we found of the use of the iAchieve brand. The presentation detailed the hardware and software requirements, personnel, costs and funding estimates, and the projects timeline broken down by system development, curriculum development, training and infrastructure needs. The timeline included the 4th and 8th grade pilots in February and March 2012 in advance of the Go/No-Go decision date on May 7, 2012, when the first hardware purchases for Wave 1 were planned. Up to that date, some system design and curriculum development costs would have already been incurred, but the more significant infrastructure, hardware and implementation costs would not. The Board approved the program by a vote of 4 to 2. 8. The iAchieve program is launched: February 13, 2012 The iAchieve program was launched with the Chief Information Officer (CIO) as project manager and the Chief Academic Officer (CAO) as project sponsor. The initial team consisted of the CIO, the CAO, two Science Curriculum Directors, and a Curriculum Coordinator. Representatives from Technical Services and Instructional Technology served as project support for the team. The project plan also called for the hiring of three Special Project Coordinators (SPCs) to serve on the project team. 9. Contract with Curriculum Ventures awarded by Board of Trustees: February 13, 2012 The Board approves Curriculum Ventures (CV), FBISD managements recommendation, for the iAchieve platform development at a cost of $1 million. This contract is funded by a grant from the George Foundation.

Fort Bend ISD iAchieve Program Review

February May 2012 10. 11. 12. 13. 14. Curriculum writers engaged and trained: February 13, 2012 through March 12, 2012 Curriculum developed: February 13, 2012 through July 11, 2012 Initial version of platform developed: February 13, 2012 through September 12, 2012 Fourth grade pilot: March 1, 2012 through March 30, 2012 Eighth grade pilot: April 2, 2012 through April 30, 2012 During late February through May 2012, the project included the following actions: Recruiting and training science curriculum writers. Teachers were recommended for curriculum writing by elementary and middle school principals. Approximately 80 teachers were engaged to write lessons for over 1,100 days of science classes for the 2nd through 8th grade levels. Developing the interactive platform including: content entry tools, so called, widgets, which gave the system functionality for sending content to classroom smartboards and interacting with students iPads. Planning and preparation for the 4th and 8th grade pilots. Interactive lessons were developed for two grades to include simulations. The 4th grade pilots incorporated simulations for weather and moon phases; the 8th grade pilots included a chemical reaction simulation. The pilot for the 4th grade was held in March 2012, and the 8th grade pilot was conducted in April. Planning the purchase and implementation of network hardware and devices for the Wave 1 roll-out in fall 2012. Contracts with Apple Computer (iPads) and Inx, LLC (hardware and wiring) were executed in April 2012. April 2012 15. Purchase orders issued for iPads and wireless equipment: April 24th and 25th, 2012 Purchase orders to Apple Corporation for the purchase of Wave 1 iPads were issued in April. The district also contracted with Inx, LLC for the installation of wireless equipment at Wave 1 schools.

Fort Bend ISD iAchieve Program Review

June 2012 16. Status report presented to the Board of Trustees: June 11, 2012 At the Board workshop and subsequent regular meeting in June, staff summarized the results of questionnaires completed by teachers and students participating in the pilots in 4th and 8th grade. The questionnaire data was summarized as generally positive. Some concern about the program and the funds committed to it were expressed by Board members during the workshop. A resolution to temporarily suspend the program was discussed but withdrawn during the regular meeting. In the absence of a no-go vote, FBISD moved forward with the implementation of iAchieve. The Superintendent under whose leadership the iAchieve program was developed left FBISD. July 2012 17. Installation of wiring, wireless hardware and iPad devices: July 2, 2012 through July 31, 2012 In July 2012, the District began the installation of the wireless infrastructure to support the iAchieve program and the deployment of iPad devices (in carts) to the 14 Wave 1 participating schools. The CAO, the Project Sponsor, left FBISD. An acting CAO is named and assumes role as Project Sponsor. August 2012 18. Baseline survey of science students and teachers: August 15, 2012 through September 15, 2012 At the August 13, 2012 regular business meeting, the Board passed a resolution for the district to conduct a study of iAchieves Wave 1 implementation based on the What Works Clearinghouse Procedures and Standards Handbook (Version 2.1). District administration was to provide the results of the study to determine whether the iAchieve program has a significant, positive impact on student achievement prior to any Board decision to approve Wave 2 of the program. In accordance with directives from the Board, the Department of Accountability conducted baseline surveys of teachers and students in science in order to assess the impact of the iAchieve program. Approximately 400 teachers and 17,300 students from the 14 iAchieve and from 14 comparable non-iAchieve campuses were surveyed. Due to the fact that roll-out of the program has been delayed, no follow-up assessments have been conducted.

Fort Bend ISD iAchieve Program Review

September 2012 19. Modification of scripted lessons to 5E model: September 3, 2012 through September 28, 2012 Beginning in September 2012, the acting CAO assigned staff to convert selected iAchieve lessons to a 5E model. The 5E model is designed to include in each lesson the elements of: engage, explore, explain, elaborate, and evaluate. This model is used in other FBISD academic areas and was felt to be more consistent with teachers expectations regarding curriculum structure. Existing iAchieve content was revised to include the 5E elements concentrating on grade levels 5 and 8. Rather than begin the Wave 1 implementation originally planned for the fall 2012 semester, efforts were made to: (1) revise existing content, (2) continue to develop and enhance the platform, and (3) conduct another pilot of the science learning program in grades 5 and 8. Two elementary schools (Austin Parkway and Blue Ridge) and one middle school (McAuliffe) were selected, and the SPCs and other Curriculum Department staff worked with selected teachers to integrate the iAchieve program in those science classrooms. October November 2012 20. Hired Apple contractor: October 5, 2012 The District contracted with Apple, Inc. for a full-time Consultant to provide professional development. The consultant received direction from and reported to the CAO. The Consultants duties included providing one-on-one training with District teachers, consulting with school staff regarding Apple applications, and providing instruction to individual classes. Through April 2013, Apple was paid $125,768.70 ($20,961.45 each month for six months) for the services of this individual. 21. Fifth and eighth grade pilots conducted: November 1, 2012 through November 30, 2012 The results of the second group of pilots, which were conducted primarily November 2012, were not encouraging. According to the SPCs, feedback from the teachers included the following general comments: The quality of the content was still very poor The scope and sequence of lessons was inconsistent with District lesson plans The content contained numerous errors, including spelling mistakes The platform did not function properly The assessment function was not in place

Unlike the pilots conducted in the spring, no formal questionnaires or other assessment tools were developed for teachers and students involved in the fall pilots. Interviews also indicated that the relationship between the SPCs and the teachers and administrators involved in the fall pilot was becoming increasingly strained. Principals at two of the participating schools requested that the SPCs supporting their campuses be re-assigned by the CAO.

Fort Bend ISD iAchieve Program Review

10

December 2012 - January 2013 22. Science helping teacher assigned to work with developers: December 3, 2012 In December, following the second round of pilots, the consensus within the Curriculum Department was that CV needed to correct the functionality of the platform and enhance the assessment component. A Science Helping Teacher was assigned to review the platform and suggest improvements, including the assessment tools. The Helping Teacher worked directly with CV programmers to make changes to the platform during December and January. 23. Presentation to Curriculum department leaders of revised platform and content: January 14, 2013 In January 2013, a demonstration of the platform and selected, revised lessons was conducted for Curriculum and Technology Department leadership and staff. The consensus of that meeting was that additional improvements were needed before attempting to implement the program at additional schools. February April 2013 24. 25. 26. Focus shift from science to broader use of devices: February 2013 Team 21 inter-disciplinary group formed: March 2013 On-going discussions of the project direction: April June 2013 During the months of February through April 2013, CV continued to make modifications and improvements to the platform at the direction of the project manager and the science helping teacher. During this same period (February - April 2013), the emphasis of the iAchieve program shifted from science instruction using a dedicated system/platform to a broader use of the iPads. Although no further pilot classes were conducted, FBISD leadership continued to discuss the direction of the iAchieve program and future deployment of the platform.

Fort Bend ISD iAchieve Program Review

11

iAchieve Program Funding

As of April 30, 2013, FBISD has expended $9,310,108 related to iAchieve in bond, grant and general funds. The District also utilized $6,542,910 in eRate funds to install hardware and wiring in both iAchieve and non-iAchieve schools for wireless connectivity. Table 1 shows the amount expended in non-eRate funds since the initiation of the project. Table 1. iAchieve Program Funding, through April 30, 2013. Source Vendor Apple Computer CDW 2003 Bond Zagg San Jacinto Staple Inx, LLC (Inx) 2007 Bond George Foundation Grant Title2, Part A Trio Electric Description Hardware and Software Hardware Hardware Supplies 2003 Bond Totals Wireless Hardware Wiring 2007 Bond Totals System Design, Development and Implementation $1,414,742 $1,000,000 $180,000 $750,000 $1,246,557 $1,868,765 $19,460,064 $13,000,000 Budgeted Encumbered and Expended $4,486,650 $164,396 $19,998 $211 $4,871,255 $819,044 $238,342 $1,057,386 $1,000,000 $173,377 $283,312 $153,103 $167,450 $320,553 $1,468,765 $135,460 $1,604,225 $9,310,108 N/A N/A $926,004 N/A $9,412,105 $357,356 $8,128,745 Remaining

Curriculum Ventures (CV)

Training Training Coordinators Curriculum Fund 199 Curriculum Writers iAchieve Staff Technical Services Positions Fund 199 - Wireless Staff Technical Services Technical Positions Fund 199 Technical Totals Inx Technology Allotment PC Mall Wireless hardware Mobility software management Technology Allotment Totals Grand Total
Source: FBISD Business & Finance

Fort Bend ISD iAchieve Program Review

12

As of June 2013, FBISD has paid CV $670,000 of the total $1 million contract amount. CV submitted an invoice in April 2013 for $130,000, representing the amount allocated in the contract for contingent costs. That invoice is still outstanding. An additional $200,000 has not been invoiced and represents the amount allocated in the contract for simulations. None of the 100 simulations called for in the contract have been developed or loaded to the platform. The amount paid to Inx of $2,287,809 includes approximately $1 million for wireless installation at noniAchieve schools. The original estimates from Inx used to develop the eRate funding application included both iAchieve and non-iAchieve campuses. The Districts portion of the amount paid Inx (20%) was paid from the 2007 Bond and the Technology Allotment funds. Eighteen positions (Fund 199 - Technical) were approved by the Board related to the iAchieve program; however, 17 of which are filled. One position, a platform programmer budgeted for $60,000 annually, is not expected to be filled. Approximately $1.25 million was budgeted for these positions in fiscal year 2012-13. The difference between the amount budgeted and amount actually expended in 2013 is due to the timing of filling those positions after the start of the fiscal year. All seventeen positions filled are permanent. Of the 17 filled positions, 12 were for iAchieve technicians to support Wave 1 schools. As noted above, the iAchieve program has not been rolled out to any schools and has been piloted in only four schools. FBISD applied for $9.8 million of Universal Service Support (E-Rate) for the 2010 Funding Year for wireless hardware and infrastructure services from Inx. Of this amount, $9.4 million ($7.5 million and $1.9 million in September 2011 and December 2010, respectively) was allowed and $7.7 million ($6.0 million and $1.7 in 2011 and 2010, respectively) was funded. Funds paid to Inx were used to install wireless connectivity at both iAchieve and non-iAchieve schools.

Fort Bend ISD iAchieve Program Review

13

Table 2. E-Rate Funding for 2010 Funding Year September 2011 Inx Estimate Reductions by USAC: $7,824,868.87 51,439.68 2,057.59 6,435.32 257.41 19,305.96 772.24 20,387.00 815.49 20,387.00 815.49 224,658.34 Total Reductions by USAC Net Eligible by USAC Actual Funding Commitment 347,331.52 $7,477,537.35 $5,982,029.88 December 2010 $1,995,206.94 12,859.92 514.40 52,479.02 65,853.34 $1,929,353.60 $1,736,418.24 Total $9,820,075.81 64,299.60 2,571.99 58,914.34 257.41 19,305.96 772.24 20,387.00 815.49 20,387.00 815.49 224,658.34 413,184.86 $9,406,890.95 $7,718,448.12

Source: FBISD Information Technology Department. Note: E-Rate is administered by the Universal Service Administrative Company (USAC).

Of the $7.7 million funded, FBISD has spent to date $6,542,910, or 84.8 percent of the total funded.

Fort Bend ISD iAchieve Program Review

14

Hardware / Network Implementation

Work began on the installation of wireless infrastructure and devices in June 2012 and was completed in July. The actual timing and costs of the installation did not differ significantly from projections. At the completion of the project, wireless connectivity was installed at the schools shown in Table 3. Table 3. Roll-out of Wireless Connectivity School Armstrong ES Austin Parkway ES Baines MS Barrington Place ES Blue Ridge ES Briargate ES Burton ES Bush HS Colony Meadows ES Cornerstone ES E.A. Jones ES Fleming ES Glover ES Goodman ES Hodges Bend MS Holley ES Hunters Glen ES Lake Olympia MS Lantern Lane ES Marshall HS McAuliffe MS Mission Bend ES Mission Glen ES Mission West ES Missouri City MS iAchieve School 90% E-Rate Reimbursement 80% E-Rate Reimbursement

Fort Bend ISD iAchieve Program Review

15

School M.R. Wood SS Oyster Creek ES Parks ES Progressive HS Ridgegate ES Ridgemont ES Sequin ES Townewest ES Willowridge HS
Source: FBISD Information Technology

iAchieve School

90% E-Rate Reimbursement

80% E-Rate Reimbursement

At E-Rate schools, wireless connectivity was installed school-wide. At the eight non-E-Rate schools, wireless was installed above the drop-ceiling in each classroom to which the iAchieve carts were assigned. Therefore, devices can access the wireless network only in these classrooms. A total of 5,910 iPad devices were deployed at the 14 iAchieve schools. An additional 390 devices on 13 carts were held in the warehouse for replacement of lost, stolen or defective devices. At the time of this review, 11 carts (330 iPads) were deployed at two high schools for teacher professional development. According to IT staff, the actual loss rate for deployed iPads is significantly lower than anticipated and additional devices held in the warehouse may be deployed in the 2013-14 school year. During the fall 2012 semester, the Instructional Technology group gathered data regarding the usage of iPads at each iAchieve school. The usage data is presented in Table 4 below. Figures 2 and 3 are based on the usage data. Reliable spring 2013 usage data were not available at the time of this report. Table 4. iPad Usage at iAchieve Schools, Fall 2012. School Name McAuliffe Cornerstone Ridgegate Fleming Ridgemont Oyster Creek Colony Meadows Type MS ES ES ES ES ES ES No. of Carts 4 21 20 14 22 17 13 No. of Teachers 3 18 9 10 14 16 14 Total No. of iPad Check-outs 862 6,030 3,102 3,763 5,772 7,264 6,716 Check-outs per Teacher 287.3 335.0 344.7 376.3 412.3 454.0 479.7 Check-outs per Teacher per Day 4.0 4.7 4.9 5.3 5.8 6.4 6.8

Fort Bend ISD iAchieve Program Review

16

School Name Blue Ridge Hodges Bend Barrington Place Austin Parkway Seguin Lake Olympia Baines

Type ES MS ES ES ES MS MS

No. of Carts 16 9 15 16 14 9 9

No. of Teachers 15 9 15 15 14 8 9

Total No. of iPad Check-outs 8,364 5,755 12,438 12,638 13,522 12,328 30,842

Check-outs per Teacher 557.6 639.4 829.2 842.5 965.9 1541.0 3426.9

Check-outs per Teacher per Day 7.9 9.0 11.7 11.9 13.6 21.7 48.3

Source: FBISD Instructional Technology

At 12 of the 14 schools, the total number of carts exceeds the number of teachers assigned to use the carts. Only at Barrington Place and Colony Meadows does the number of teachers using the carts equal or exceed the total number of carts available. The excess number of carts and devices was significant during the fall semester, because each cart was locked for use in science classes only. Beginning this past spring use of the devices was expanded beyond science classes; however, the installation of wireless in certain rooms limits the use of the carts in other areas of the schools. Figure 2 shows on an average basis the number of devices used by each teacher each day. Except for Baines and Lake Olympia middle schools, no school saw daily usage per teacher exceed 20 iPads. Figure 2. Check-outs per Teacher per Day

Student Check-outs per Teacher per Day

60.0 50.0 40.0 30.0 20.0 10.0 0.0 Student Check-outs per Teacher per Day
McAuliffe MS Oyster Creek ES Austin Parkway ES Cornerstone ES Colony Meadows ES Seguin ES Ridgegate ES Blue Ridge ES Lake Olympia MS Fleming ES Hodges Bend MS Baines MS Ridgemont ES Barrington Place ES

48.3

21.7 4.0 4.7 4.9 5.3 5.8 6.4 6.8 7.9 9.0 11.7 11.9 13.6

Source: Gibson Consulting Group based on data received from FBISD Instructional Technology

Fort Bend ISD iAchieve Program Review

17

Figure 3 shows the average number of devices used daily. Again, only two schools showed more than 50 percent daily usage of available iPads. Average usage at Baines Middle School over 100 percent indicates that devices are being checked-out multiple times daily. Figure 3. Average Percentage of iPads Used

Average Percentage of iPads Used

200.0% 160.9% 150.0% 100.0% 64.3% 50.0% 0.0% Avgerage Percentage of iPads Used
McAuliffe MS Oyster Creek ES Austin Parkway ES Cornerstone ES Colony Meadows ES Seguin ES Ridgegate ES Blue Ridge ES Lake Olympia MS Fleming ES Hodges Bend MS Baines MS Ridgemont ES Barrington Place ES

20.1% 10.1% 13.5% 7.3% 12.6% 12.3%

24.3% 24.5% 30.0%

38.9% 37.1% 45.3%

Source: Gibson Consulting Group based on data received from FBISD Instructional Technology

At the time of installation of wireless networks and iPad carts, each device was configured with standard applications (apps) for use in science classes. As the devices were rolled-out, teachers were encouraged to identify additional apps for use in their classes. Some apps are free; some are not. Free apps can be installed on any device by the teacher; however, this requires that the teacher install the app on each device in the cart. Apps can be purchased and installed on multiple devices using the Districts Volume Purchase Code (VPC) number. This allows the teacher to request the app and using the VPC and a spreadsheet listing the device code for each applicable iPad, the app is automatically loaded to the device. The use of the code is controlled by the Technology Department. Free apps can also be loaded to multiple devices using the VPC; however, the use of the code is controlled by the Technology Department. This purchase method has the advantage of keeping each device current when periodic refreshes occur. Free apps not installed through the VPC are erased when refreshes occur. Many teachers have complained about this feature and the process associated with requesting and loading free apps using the VPC.

Fort Bend ISD iAchieve Program Review

18

Platform Development
The development of the advanced custom interactive science learning experience product (the platform) occurred in three phases: 1. The Pilot model 2. The initial release (Version 1.0) 3. The current release (Version 2.0) The purpose of the platform is to host science lesson content and to enable the science teacher to interact with District databases (student information, curriculum, etc.), student iPad devices, and other classroom devices (smartboards, etc.) in real time.

Pilot Model Contract

The District contracted with Louque & Associates on September 15, 2011 for $135,000 based on approval from the Board on September 12, 2011, for development of an interactive science instruction system (the pilot model). Louque & Associates subcontracted two software development engineers as the designers and programmers for the pilot model. The term of the Louque & Associates contract was September 12, 2011 through September 30, 2011, and ARRA (American Recovery and Reinvestment Act of 2009) funds were earmarked to pay the contract. Item no. 2 in the contract specified the services:

The vendor will deliver the infrastructure and framework of an instructional delivery system to Fort Bend ISD by September 30, 2012. Services include: a. Development of a self-contained method of delivery for interactive learning experiences with on demand instruction capability uniquely designed to focus on science instruction. The self-contained method of delivery will be designed with following: a. A simple and efficient user-friendly interface for teachers with single touch access to curriculum, assessments, and instructional resources b. An engaging interface for students designed for the use of custom-built real-time interactive experiences aligned with the curriculum, and formative assessments to assess student progress c. The capability of projecting on demand appropriate teaching resources or student instructions

Fort Bend ISD iAchieve Program Review

19

b. Four customized interactive learning experiences, as described in Attachment A, for use in 4th and 8th grade and focused on identified areas of weakness in student understanding of science concepts. Each interactive learning experience will include a pre-assessment, interactive learning simulation, a post-assessment, remediation capability and a progress reporting interface. c. Six (6) containers (the box) for 6 classrooms

The contract specifically prohibited payment to the contractor before services were rendered. However, FBISD made advance payment to Louque and Associates on September 29, 2011, because the deadline for the use of ARRA funds was September 30, 2011. The Louque & Associates invoice, dated September 20, 2011, listed the deliverables verbatim from Item no. 2 of the contract, including the use of the future tense (Each interactive learning experience will include a pre-assessment . . .) from sub-part b. The four simulations and the platform to run the pilot lessons were not completed until the start of the 4th and 8th grade pilots in March and April 2012.

Platform Development Contract

In internal discussions, the funding for the development of the platform was to be substantially underwritten by a grant from the George Foundation (the Foundation). Originally, the Foundation had been approached by FBISD and approved a grant in 2010 of $1 million for use in the development of science experiences as part of the originally-conceived global center for science and technology. This grant was never funded. FBISD officials consulted the Foundation to change the terms or submit a new application for the development of the interactive science learning platform. A new application was submitted and the Foundation agreed to fund this grant in the same amount to create a Teaching and Learning Science Platform for 2nd through 8th grade level students. The grant requires the District to provide a report to the Foundation by November 30, 2013, discussing the impact of the grant activities, as well as documentation of grant expenditures. Although CV represents that the current version of the platform has been delivered to FBISD, to date, the interactive science learning platform has not been deployed. In September 2011, FBISD issued an RFQ for the development of the interactive learning platform. The RFQ was sent to 217 potential vendors. The deadline for submission of qualifications was October 14, 2011. In response to the RFQ, the District received four vendor proposals. These vendors were evaluated based on three criteria: (1) demonstrated competence and qualification, (2) overall pricing, and (3) delivery timeline. A four-person team conducted the evaluation of vendor proposals and scored each response. The tabulation of the results is shown in Table 5.

Fort Bend ISD iAchieve Program Review

20

Table 5. Evaluation of RFQ Responses. Vendor Name Curriculum Ventures (CV) Engaging Solutions Resource Data One Hundred Twenty Years
Source: FBISD Purchasing

Criteria 1 Demonstrated Competence and Qualification 4.5 3.5 2 1.25

Criteria 2 Overall Pricing 4.25 2.75 2.25 2.25

Criteria 3 Delivery Timeline 2.75 3.75 2.5 2.75

Total

Ranking

12.75 10 6.75 6.25

1 2 3 4

The winning firm, CV, submitted its proposal on October 14, 2011. According to documentation received from the Louisiana Department of State, CV registered in that state on the same date, and had no operations prior to that date. According to the RFQ response (resumes and staff experience) neither the company, nor its principals, had any experience developing or deploying an advanced custom interactive science learning experience similar to that envisioned by FBISD. The proposal listed only two employees of the company who would participate in the project. These individuals are the only officers and employees of the company. All other staff that provided assistance in the development of the platform pursuant to the contract with FBISD was engaged by CV as subcontractors. According to CV, FBISD is the only customer the company has had since its inception. The CV proposal listed three options and associated pricing for development of the interactive learning system, or platform, for science education in seven levels (Grades 2-8). These options are shown in Table 6 below. Table 6. Curriculum Ventures Total Cost Projection System Design and Deployment Deployment by Phase Content Integration Simulations Total System
Source: Curriculum Ventures RFQ response

Option 1 Six Months $171,000 $138,600 $497,700 $807,300

Option 2 Ten Months $285,000 $138,600 $497,700 $921,300

Option 3 Eighteen Months $513,000 $138,600 $497,700 $1,149,300

Fort Bend ISD iAchieve Program Review

21

Key aspects of the proposal included: Each option presented represented varying levels of functionality and integration with other FBISD databases and information systems. Deployment was anticipated by phases: Discovery, Implementation, Full Deployment and Ready for Use. Content integration was estimated at $19,800 per grade level Simulation cost estimates included 1,260 simulations (180 simulations per grade level with 2 activities per simulation) at an individual cost of $395.

Exhibits 1-5 of the CV proposal showed various templates or mock-ups of the system design proposed by the company. These exhibits were part of the presentation that was presented to the Board at the August 22, 2011 meeting. Neither CV (which was formed in October 2011) nor Louque & Associates were paid for work performed prior to September 2011. As noted above, Louque & Associates received a $135,000 contract for system design and pilot lesson development in September 2011, but prior to that, all services to FBISD by that company related to character training. As a result of the RFQ process, FBISD contracted with CV on February 14, 2012, for planning, design, development, testing and delivery of a self-contained, on-demand instructional delivery system (the platform). The specific elements of the system based on the contract were to include:

a. Lesson template with sections for curriculum alignment, concept development, interdisciplinary connections, media library, etc. b. One hundred customized simulations. Each simulation to include pre-assessment, interactive learning simulation, post-assessment, remediation capability and reporting interface c. Content integration tools for loading all aspects of the curriculum framework, lesson, activities, assessments, notes and other resources provided by FBISD d. Intuitive and simple interface for teachers with single touch access to curriculum, assessments and instructional resources e. An engaging interface for students designed for the use of custom-built, real-time interactive experiences aligned with the curriculum, and formative assessments to assess student progress f. The capability of projecting on-demand appropriate teaching resources or student instructions

g. Print capabilities h. Performance reporting dashboards for teacher, student and administrator levels

Fort Bend ISD iAchieve Program Review

22

i. j.

Lesson template with sections for curriculum alignment, concept development, interdisciplinary connections, media library, etc. One hundred customized simulations. Each simulation to include pre-assessment, interactive learning simulation, post-assessment, remediation capability and reporting interface

k. Content integration tools for loading all aspects of the curriculum framework, lesson, activities, assessments, notes and other resources provided by FBISD l. Intuitive and simple interface for teachers with single touch access to curriculum, assessments and instructional resources

m. An engaging interface for students designed for the use of custom-built, real-time interactive experiences aligned with the curriculum, and formative assessments to assess student progress n. The capability of projecting on-demand appropriate teaching resources or student instructions o. Print capabilities p. Performance reporting dashboards for teacher, student and administrator levels

The contract included further specifications regarding operability, web-browser compatibility and interfaces with the FBISD active directory, student information databases, and curriculum management system (Data Driven Software Corporation for Curriculum and Assessment system, known as D2SC). The contract compensation provisions differed substantially from the cost options proposed by CV in its response to the RFQ. The compensation section in CVs proposal included the items shown in Table 7.

Fort Bend ISD iAchieve Program Review

23

Table 7. Compensation for Platform Development. Contract Component Activity Planning & Discovery Design & Development Consulting Coding, Testing & Configuration Maintenance & Improvement Content Integration Simulations Contingency Total Contract
Source: FBISD Business and Finance Department

Per Unit Cost $40,000 $80,000 $10,000 $10,000 $14,000

Units 3 4 8 5 7

Total Pricing $120,000 $320,000 $80,000 $50,000 $98,000

Development, Testing, Production & Release of Auto Load Tool Simulations & Associated Activities Development Contingency

$2,000 $132,000

100 1

$200,000 $132,000 $1,000,000

The contract was primarily drafted by FBISD without significant input from CV. According to CV representatives, they agreed to the compensation terms (without fully understanding the pricing structure) in order to be awarded the contract. We identified three main areas of concern with the contract: Neither CV representatives nor FBISD could explain how the units used in pricing the activities that comprise the Consulting component of the contract correspond to deliverables. The billings submitted by CV (discussed below) suggest that these units were intended to be months, but the amount of work to be performed each month by CV is not specified. The number of simulations and the price per simulation in the contract differs significantly from the CV proposal. CV originally proposed 1,260 simulations at a per simulation cost of $395. The contract includes only 100 simulations at an individual cost of $2,000. Nothing in the contract explains the difference in expectations of the simulations. Without considering the cost of simulations which were never delivered the option in the proposal closest to final contract (Option 3) estimated that the project would require 18 months to complete. When the final contract was negotiated, the timeframe for delivery of the platform was expected to take less than six months.

Beginning in March 2012, CV invoiced the District for work completed as shown in Table 8.

Fort Bend ISD iAchieve Program Review

24

Table 8. Platform Invoices from Curriculum Ventures.

Invoice Details Consulting
Planning & Discover $40,000@ (3) Design & Development $80,000@ (4) Coding, Testing & Configuration $10,000@ (8) Maintenance & Improvement 10,000@ (5)

Date

Amount

Approval

Content Integration
$14,000@ (7)

Simulations
$2,000@ (100)

Contingency

$132,000

March 18, 20121 April 13, 2012 May 15, 2012 June 15, 2012 October 3, 20122 February 15, 20133 March 15, 20133 April 15, 20134 Total Dollars Total Units

$156,000

Herron

$180,000

Herron

$128,000

Herron

$128,000

Calvert

$58,000

Calvert

$10,000

Calvert

$10,000

Calvert

$130,000

Outstanding

$130,000

$800,000

$120,000

$320,000

$80,000

$50,000

$98,000

$2,000

$130,000

3
Source: FBISD Business and Finance
1

N/A

The only simulation completed was the chemical reaction simulation which was part of the 8th grade pilot. FBISD paid for that simulation under the September 2011 contract to Louque & Associates. None of the 100 simulations under the CV contract were delivered. Therefore, this represents an overpayment of $2,000.

The October 3, 2012 invoice originally submitted by CV requested payment of $124,000 for one unit of planning ($80,000), one unit of coding/testing ($10,000), two units of maintenance ($20,000), and one unit of content integration ($14,000). Because the company had already billed for all planning units and only one unit remained unpaid for maintenance, Mr. Calvert changed the items to be paid on the invoice to those shown in the table above. No explanation was provided by either CV or Mr. Calvert as to the discrepancy between what was delivered and billed by CV and what was accepted and paid by FBISD. No line item in the contract corresponds to System Integration and Testing. At the time that the February and March invoices were submitted, $350,000 was left unpaid under the contract: one unit of coding/Testing/Configuration ($10,000), one unit of
3

Fort Bend ISD iAchieve Program Review

25

Maintenance and Improvement ($10,000), contingency ($132,000) and 99 simulations ($198,000). Units paid for system integration and testing were apparently substituted for units of Coding/Testing/Configuration and Maintenance and Improvement.
4

The invoice for payment of the contract contingency listed only those deliverables specified under the contract. The company did not submit any description of additional time and effort expended on the contract, or describe specific deliverables that necessitated the company to expend additional effort under the contract.

The contract required CV to submit detailed invoices to FBISD describing the Services rendered, the times when such Services were performed, compensable expenses and the amount due. According to District staff, CV did not submit the documentation required under the contract with each invoice to support the progress of the work performed to-date, the deliverables provided, or the remaining work to be completed. Project plans and status reports related to this contract were requested from both CV and FBISD but were not provided. As noted in Table 8 above, CV began billing FBISD for Maintenance and Improvement of the platform beginning with its first invoice in March 2012. By the time the initial version of the platform was delivered, CV had already billed all units related to maintenance and improvement. The original invoice and the only documentation from CV supporting the March 18, 2012, are shown in Appendix B. The invoice is similar to the other invoices submitted for payment. This recap, as well as recaps for the other invoices, was prepared upon request of Gibson, and we are unable to conclude as to the availability of concurrent documentation, such as time and effort reports, maintained by CV to support the project invoices. No documentation was available from FBISD that supported approval of the CV invoices. There was also no documentation to indicate that the status of the platform development was discussed with the iAchieve steering committee as a whole. According to District staff and CV representatives, CV delivered Version 1.0 of the platform to FBISD in June 2012 and loaded 791 science lessons. Curriculum writers developed storyboards for simulations; however, none of the simulations were coded and loaded into the platform. As noted in the Timeline of Events section in this report, FBISD reported what they perceived as deficiencies with the platform and instructional content during the summer 2012. FBISD continued to consult with CV regarding improvements over the subsequent months. A modified version of the platform was used for the second round of pilots in October and November 2012; however, a thorough testing of the platform in late November indicated that further modifications of the platforms functionality and features were needed to meet the Districts needs. In December 2012, a science helping teacher was assigned full-time to work with CV to improve the platform functionality. Another revised version of the platform was used for the demonstration to Curriculum Department leadership in January 2013, but the reception was negative. The science helping teacher has continued to work with CV to improve the platform, which is currently in Version 2.0.

Fort Bend ISD iAchieve Program Review

26

Gibson reviewed the platform to assess its functionality. Our comments are listed below. Based on our interviews, we are not aware that the District has performed a formal test and acceptance of the system to determine whether the terms of the contract with CV have been met satisfactorily. The software lacks certain functionalities specified in the contract: printing capabilities; performance reporting dashboards for teachers, students and administrators; and integration with other FBISD systems including active directory, Skyward (student information) and D2SC (curriculum management) There is no documentation indicating the level and rigor of software testing performed by CV or FBISD user groups There is no help documentation that explains the functionality of the system There is no technical documentation that explains the application architecture, data architecture or any technical documentation about all the components of the system There is no documentation regarding the support of the software. There is no service level agreement indicating whom the District will call in case of support issues or what the response time will be in varying priorities. The contract with CV indicates that the District will get free upgrades to the software, but it is unclear who will lead the upgrade effort, and what frequency the updates will be available. Also, the question of who will decide what functionality or technical upgrade will be in the next upgrade effort is not specified. With the exception of one Helping Science Teacher from a power-user point of view, the District has no staff knowledgeable about the platform software from a user or technical support point of view. It is unclear whether the software is scalable within the District network. The system was never tested beyond one or two pilot lessons or campuses; issues may arise when the system is scaled to the 14 campuses with 700+ lessons. The contract indicated that the District owns the software and the vendor is working for hire; however, the contract also stated that the vendor has the rights to all intellectual property owned or used by the contractor to construct and deliver the product, as well as the intellectual property rights to the instructional delivery system platform and associated coding. In interviews with the principals of CV and FBISD officials, there was confusion regarding the ownership of the intellectual property associated with iAchieve. The respective rights of FBISD and CV to the platform software are also unclear. The Curriculum and Technology Departments have not undertaken a comprehensive review of the platform and instructional content to determine the long-term viability of the system in

Fort Bend ISD iAchieve Program Review

27

science or other academic disciplines. This review would include IT technical and instructional technology staff, curriculum specialists, teachers, principals and other potential users of the system.

Fort Bend ISD iAchieve Program Review

28

Project Management
As noted above, the CIO served as iAchieve Project Manager. Given the demands of the project, it was a questionable decision for the CIO to assume the role of project manager for a major technology initiative, such as iAchieve. The scope of this program warranted a fully dedicated position to project management. The Board approval of the project in February 2012 included $60,000 for a technical application support programmer; however, this position, which would have provided assistance to the project manager in overseeing the development of the platform, was never filled. We believe that the project success was hindered by not having a dedicated FBISD position with the technical and project management skills assigned to oversee the program. The iAchieve project was launched in February 2012. Prior to that time, the Project Manager and the Curriculum Directors prepared schedules for each component of the project as shown in Figure 4 and Table 9 below. The Virtual Science and Technology Project Plan (the name used before iAchieve) was presented to the Board in December 2011, and was presented again unchanged to the Board on February 13, 2012 at the project launch. Figure 4. iAchieve Technology Plan

Source: Documentation from FBISD Board Meetings

Fort Bend ISD iAchieve Program Review

29

The Curriculum Development Plan was prepared under the assumption that the project would launch in late 2011. The Curriculum Directors revised the schedule again before the actual program launch and several times thereafter. Table 9. Curriculum Development Schedule. October 31 - December 18, 2011 Hire Special Project Coordinators (3) Assemble content writing teams Train teams on lesson format and expectations

January 9 - January 27, 2012 Checkpoint #1 Seven (7) lessons due per person Coordinators review content and provide feedback Coordinators work with digital architects on simulation development Train teachers of pilot schools/grade levels

January 30 - February 24, 2012 Checkpoint #2 Eight (8) additional lessons due per person Coordinators review content and provide feedback Coordinators work with digital architects on simulation development Content writers revise and fine tune first seven (7) lessons Monitor pilot progress and collect feedback - use to make adjustments

February 27 - March 9, 2012 Content writers turn in final version of first seven (7) lessons Content writers revise and fine tune remaining eight (8) lessons Coordinators submit first set of content to vendor for loading Testing of content in the system Development of professional development sessions

Fort Bend ISD iAchieve Program Review

30

March 30, 2012 Remaining eight (8) lessons due in final form

April 2 - April 20, 2012 Coordinators submit a second set of content to vendor for loading Coordinators work with digital architects on simulations development Testing of content in the system

April 23 - May 25, 2012 Final testing and delivery of platform, content, simulations Training of Trainers sessions for Professional Development Gold Standard

August 2012 Implementation

Source: Reproduced from original from FBISD Curriculum Department

The initial plans called for development of the interactive platform in three months (January April 2012) and completion of the curriculum in four months (December 2011 April 2012). As noted in Figure 4, the curriculum plan called for four check points, and the infrastructure and hardware plan included a Go/No Go Decision point. We requested, but did not receive, documentation to support the curriculum directors review of the project at each of the check points. According to the project manager, a formal analysis of the project was not conducted to support the no/no-go decision in June 2012.

Special Project Coordinators

As indicated in Table 10 above, the first activity in the Curriculum Development Plan was hiring three SPCs. The job description for the SPC position required a successful candidate to be a certified teacher with a minimum of three years teaching experience and demonstrate expertise in project development and implementation and staff development. Preferred qualifications included a masters degree and experience with curriculum writing. The job description did not require any experience in curriculum development, utilizing interactive technology tools, or any expertise in science curriculum. The actual responsibilities of the SPCs included coordinating simulation design and curriculum development, as well as liaising between the content writers, curriculum directors, the project manager, digital architects, and school-based instruction staff. These three positions were critical to the success of the curriculum portion of the project as shown by their responsibilities for the various activities in Table 10. A tool for the assessment of SPC candidates

Fort Bend ISD iAchieve Program Review

31

prepared prior to the launch of the program indicates that the SPCs were expected to identify resources, develop strategies, and implement methodologies to accomplish the following tasks: Pilot lessons in 4th and 8th grades Work with content writers to write 1,260 days of lessons that incorporate the iPad, apps, probeware, high-yield strategies, and assessments Work with digital architects to develop 100 novel, differentiated simulations Train curriculum writers Provide feedback on lesson content Test lessons and simulations Provide progress updates Design and deliver professional development to pilot teachers, as well as all teachers participating in year 1 implementation Evaluate pilot success

The three SPCs were hired shortly after the launch of the program in February 2012. Each met the requirements specified in the position description, and each was familiar with technology used in the classroom. However, none had a technology-related degree or significant expertise in developing cutting-edge, instructional technology. Two of the SPCs had experience teaching science and coaching other science teachers; the other had no experience in science curricula or instruction. Each was expected to work closely with digital architects at CV in designing simulations to run on the iPad, and to oversee and give feedback to science content writers. The SPCs as a group lacked the experience specified in the task requirements. The SPCs initially reported to the Curriculum Directors; however, friction between the SPCs and the directors resulted in the decision by the Project Sponsor to change the reporting structure in March 2012 to have them report directly to her. Their roles continued to evolve over the course of the project.

Changes to Project Timelines

The pilots were originally scheduled for February 2012, but evidence in the form of emails and meeting notes indicates that the pilot classes were delayed for a variety of reasons including: lack of instructional content in the system, simulations not being ready, functionality problems with the platform itself, and network or connectivity issues. FBISD paid Louque & Associates the full contract amount ($135,000) in September 2011 and delivery was required by September 30, 2011. However, based on documentation in the CV proposal, it appears that CV was continuing to work on the pilot model version of the platform needed to deliver the pilot lessons, as well as the related 8th grade simulations, well into April 2012. During that time, CV was submitting invoices under its $1 million contract.

Fort Bend ISD iAchieve Program Review

32

During the period from February through May 2012, timelines for the development of the curriculum for Wave 1 (Fall 2012 roll-out) were revised several times as the project deadlines passed. As indicated above, the delay in Board approval of the program due to the delay in the awarding of the George Foundation grant pushed the iAchieve launch date back one month. However, the impact on schedules for curriculum and platform development slipped much further. According to internal timelines developed in October 2011, the iAchieve program was scheduled to start in late October (2011) with the hiring of SPCs and assembling the content writing teams. May 25, 2012 was set as the date for final testing of the system and completion of all content and simulations. Wave 1 would begin in August 2012. In November 2011 the timeline was revised to push the start date back one month to November 30, 2011, with final testing and delivery of the platform, content and simulations now projected three months later (August 23, 2012). Over the next few months, timelines for the completion of content and simulations continued to be pushed back; however, the start of Wave 1 in August 2012 was never revised. Similarly, timelines presented to the Board showed significant delays in platform development and delivery. When Board approval of iAchieve was expected in January 2012, FBISD represented that that the production release of the platform would occur in April 2012. In a presentation to the Board on June 11, 2012, the board approval date was adjusted by one month (to February from January), but the scheduled delivery of the platform was pushed back six months to October 2012, and the start of the Wave 1 roll-out was delayed to October 22, 2012. The pilot for the 4th grade was held in March 2012, and the 8th grade pilot was conducted in April 2012. Eight teachers and approximately 270 students were involved in both grade levels. FBISD distributed questionnaires following the pilots and six teachers and 256 students responded. Of the six teachers, five indicated a generally positive experience; however, one teacher was consistently negative in his/her feedback. The same five teachers noted above felt that the iPad devices kept the students engaged, although they cited frequent technical glitches and mistakes in the materials. Students feedback was positive; however, the questions were subjective: How do you feel about the iPad experience? and What would you change about the iPad experience? Eighty to ninety percent of the students were able to use the iPad functions without help. The novel use of the technology and the capability of apps appealed to most students. Results of the pilots were presented to the Board at a workshop meeting in June 2012. In the absence of a vote to terminate the program, the Board essentially confirmed the decision to proceed with the iAchieve implementation.

Curriculum Management
The CIO, as project manager, assumed responsibilities of certain aspects of the curriculum development process.

Fort Bend ISD iAchieve Program Review

33

For example, the project manager prescribed a framework of control documents for the Curriculum Writers to complete when creating their lessons. This framework included the following key components (and related documents) for each grade (levels 2-8):

1) Storyboard Outline: A day-by-day detail of the lessons by grade level. The storyboard outline, in excel format, includes the following components: a) Unit Description: for example, Solar System and Space Exploration (grade 6) or Forms of Energy (grade 4) b) Unit overview: description of the unit objectives. c) First deliverable (for content writers). For each lesson within the unit: i) Lesson descriptor and schedule (1) Lesson overview (2) Number of days to cover the lesson ii) Strategies and activities (PSET strategies and related activities) iii) Resources used iv) Assessment d) Second deliverable: Teacher script documentation - draft e) Third deliverable: Teacher script documentation final 2) Unit Plan. A brief synopsis of the unit including: unit scheduler, TEKS standards, strategies emphasized, interdisciplinary connections, objectives, vocabulary words, other topics 3) Simulation Documents: a) Simulation definition sheet: brief synopsis of the simulation: TEKS addressed (key nouns and verbs), student experience, activities b) Simulation drawing called Unwrapping the Simulation c) Simulation checklist 4) Lesson Plan Outline. A brief synopsis of the lesson plan to guide the teacher including objectives, interdisciplinary connections, devices, teaching strategies, pre-lesson planning tips, simulations to be used, pre-assessment and post-assessment guides 5) Lesson Script. Actual instruction script for use by teacher in the class 6) Assessment. Activities to measure the comprehension of the lesson topics by the students as a group and individually 7) Evaluation or Review Tools. Storyboard outline rubric, unwrapping the simulation rubric, and lesson script rubric 8) Storyboard Outline: A day-by-day detail of the lessons by grade level. The storyboard outline, in excel format, includes the following components:

Fort Bend ISD iAchieve Program Review

34

a) Unit Description: for example, Solar System and Space Exploration (grade 6) or Forms of Energy (grade 4) b) Unit overview: description of the unit objectives. c) First deliverable (for content writers). For each lesson within the unit: i) Lesson descriptor and schedule (1) Lesson overview (2) Number of days to cover the lesson ii) Strategies and activities (PSET strategies and related activities) iii) Resources used iv) Assessment d) Second deliverable: Teacher script documentation - draft e) Third deliverable: Teacher script documentation final 9) Unit Plan. A brief synopsis of the unit including: unit scheduler, TEKS standards, strategies emphasized, interdisciplinary connections, objectives, vocabulary words, other topics 10) Simulation Documents: a) Simulation definition sheet: brief synopsis of the simulation: TEKS addressed (key nouns and verbs), student experience, activities b) Simulation drawing called Unwrapping the Simulation c) Simulation checklist 11) Lesson Plan Outline. A brief synopsis of the lesson plan to guide the teacher including objectives, interdisciplinary connections, devices, teaching strategies, pre-lesson planning tips, simulations to be used, pre-assessment and post-assessment guides 12) Lesson Script. Actual instruction script for use by teacher in the class 13) Assessment. Activities to measure the comprehension of the lesson topics by the students as a group and individually 14) Evaluation or Review Tools. Storyboard outline rubric, unwrapping the simulation rubric, and lesson script rubric

The Curriculum Directors included many components of the framework, including unit and lesson plans and the lesson scripts in their initial training of teachers. The storyboard outlines were considered necessary to define the scope of the curriculum development phase, to ensure compliance with state guidelines, and to promote consistency among the work of over 70 content writers. However, the Curriculum Directors considered the storyboard outlines to be particularly time-consuming to complete and set the writing of the lesson scripts and simulations back.

Fort Bend ISD iAchieve Program Review

35

We were unable to obtain information about where the complete versions (either electronic or hardcopy) of all the above components of documents for every grade level were stored. Evidence was not found to determine whether the lessons currently loaded in the platform (version 1.0 only) have been subjected to rigorous quality assurance inherent in the curriculum framework. No lessons have been loaded to the version 2.0 platform. Simulations, which were to be the key interactive component of the iAchieve system, were drawn by content writers to accompany each lesson. We were unable to obtain an accurate count of the simulations drawn for all grade levels and the extent of review/approval of each simulation. However, the simulations were developed late in the process (June 2012) and required significant interaction with the CV software engineers to code. To date, no time has been spent to code the hand-written simulation storyboards.

Project Transition
Curriculum writing and platform development took longer to complete than planned. Lesson writing, which was originally scheduled to begin February 2012, did not occur until May and June 2012. The initial schedule had the platform release in April 2012, but punch list items for version 1.0 were not completed until late August 2012. During the summer months, leadership in the District changed significantly. In June 2012, the Project Sponsor resigned and was replaced by acting CAO. In July, Dr. Jenney resigned and was replaced by an Interim Superintendent. Several meetings were held during July and August regarding the iAchieve program status and direction. An iAchieve program executive leadership team was formed and regular meetings were held. This group included senior management and staff of Technology and Curriculum Departments. Due to the district management changes and friction among the remaining original members of the iAchieve leadership team, progress of the iAchieve program stalled. Over 700 science lessons were completed in June and loaded into the platform. While several FBISD staff members interviewed for this project expressed the opinion that the lesson quality was poor, none of those staff members were directly involved in the development of the lessons. The direction of the program was also changed in September 2012 when the curriculum directors responsible for the content were re-assigned to other roles in the District. Of the original five key members of the project leadership, now only the Project Manager remained. Despite the fact that none of the planned science simulations were developed which was one of the primary purposes for the iAchieve program the initial version of the platform was delivered, the content entry system was finalized (over 700 lessons were loaded), and a punch list of changes to the system was developed and completed by August 2012. While there is evidence of several meetings and discussions regarding the system and content during the summer and fall of 2012, there were few documents to indicate that a formal plan was in place

Fort Bend ISD iAchieve Program Review

36

similar to those used during the development of the platform, the writing of instructional content, or the implementation of the wireless infrastructure that occurred in the five months following the launch of the program. Two other key changes occurred during the July September 2012 period: 1. A shift of the program from science simulations to other areas and widgets 2. A modification of scripted lessons to a 5E model From the inception of the program, iAchieve was primarily designed to provide an enhanced experience for science students. In July, principals expressed their concern that the program needed to include more than just science. Since none of the 100 planned simulations were developed or implemented, future system development focused on the expansion of the platform to accommodate any academic area and to enhance the usefulness of widgets. A widget is the name given to any tool that enables the teacher to perform lesson building or instructional activities directly on the iPad. An example of a widget is a button to push content of the teachers iPad to the students iPads, or to push content to a smartboard. Widgets were developed for drawing/painting, assessment or for students to individually ask the teacher a question during class. The change in curriculum leadership after the re-assignment of the Curriculum Directors also resulted in a re-assessment by the Curriculum Department of the scripted lesson plans. As noted above, many reviewers questioned the quality of the lessons. Many interviewees including SPCs, teachers, principals, and others indicated that the process for developing curriculum was made overly complicated and, as a result, the lesson quality was inconsistent and teachers spent more time preparing curriculum development-control documents than the lessons themselves. This may be one reason why the simulations were never completed as envisioned.

Current Status of the Program

During the months of February through April 2013, CV continued to make modifications and improvements to the platform at the direction of project manager and the science helping teacher. We found no evidence of any plans to re-introduce the curriculum or the platform in other schools or classrooms. During this same period (February-April 2013), the emphasis of the iAchieve program shifted from science instruction using a dedicated system/platform to a broader use of the iPads. Each of the fourteen iAchieve schools was encouraged to utilize the iPad carts for any subject matter and to identify apps that could enrich existing curricula and improve student engagement. According to interview sources, many schools took advantage of the iPads for other uses, while some schools kept the iPad carts within the science classes. The initiative to expand the use of iPads beyond the science area was named Team 21. The Team 21 leadership group included the Apple contractor and members of both the Curriculum and Technology

Fort Bend ISD iAchieve Program Review

37

Departments. Rather than the top-down approach used in the iAchieve program, Team 21 provided training to teachers at certain schools wanting to incorporate the iPads in the classroom. The value of the training was in the collaborative approach by Team 21 members and the ability to answer teachers questions regarding technology, curriculum and apps at one time. To date, training sessions have been held at Hodges Bend and Lake Olympia middle schools.

Fort Bend ISD iAchieve Program Review

38

Appendix A Project Interview List

Name Arthur Allen Lisa Baum Robert Calvert Ginger Carabine Carmen Colligan Tracy Ginsberg Mary Jackson Cindy Juarez Alex Jones Mike McKie Lynette Meyer Jaime Marshall Jan Moore Michele Riggs Melissa Stadtfeld Susan Voradakis Rachele Wooten Dennis Harrison Bonnie Louque Karen Percak
1

Position Director, Technical Services Special Projects Coordinator Chief Information Officer Executive Director, Curriculum Special Projects Coordinator Chief Financial Officer Chief Academic Officer Science Helping Teacher Special Projects Coordinator Interim Superintendent Director, Educational Technology Science Curriculum Coordinator Director, Accountability Principal of Meadows ES Director, Curriculum Team 21 Liaison Principal Principal Apple Technology Consultant
1

Organization FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD FBISD Curriculum Ventures Curriculum Ventures Apple Corporation (contracted)

Assistant Principal of Lake Olympia MS1

Melissa Stadtfeld and Michele Riggs were Curriculum Directors when assigned to the iAchieve team. They were re-assigned to their current positions in September 2012.

The FBISD former Superintendent Dr. Tim Jenney and former Chief Academic Officer Dr. Olwen Herron, both of whom played roles in the creation and development of the iAchieve program, were not available for interview.

Fort Bend ISD iAchieve Program Review

Appendix B Curriculum Ventures March 18, 2012 Invoice and Documentation

Fort Bend ISD iAchieve Program Review

Title Planning & Discovery Design & Development Code/Testing/Configuration

Associated sprint (Subgroup) Initial discovery of stakeholder requirements (D&D) Initial stack stubs and user stories (*) Initial unit tests and workflow configuration (*) Initial configuration of Staging & CI (Devops)

# Engineers Notes 4 6 2 We immediately noticed performance incosistencies inside of the FBISD network. This is turned into "Content Entry" right away, as FBISD wanted to be able to have different stakeholders edit the content at will. This is also 4 activities and a formative assessment relating to the chemistry simulation @ rigor for 8th grade

Maintenance & Continuous improvement

Development/Test/Production release of an auto load tool

Ontology Discovery & Content Toolchain (KEEP_OPEN)(*)

Simulations and associated activities as specified in "Definitions" Simulations (KEEP_OPEN)(*) (Chemistry)