December 15, 2009
IN THE MATTER OF STEVEN WINTERS, FIRE OFFICER 2 (PM3602G), NORTH HUDSON REGIONAL FIRE AND RESCUE.
On appeal from the Civil Service Commission, Department of Personnel, Docket No. 2008-3944.
NOT FOR PUBLICATION WITHOUT THE APPROVAL OF THE APPELLATE DIVISION
Submitted October 27, 2009
Before Judges Skillman and Gilroy.
Steven Winters appeals from a final decision of the Civil Service Commission, issued on July 31, 2008, which denied his appeal from the rescoring of one answer on the multiple choice portion of a promotional examination for battalion fire chief and fire officer 2. This rescoring resulted in the reduction of Winters's score from 90 to 89.7, as a result of which his rank on the promotional list for Fire Officer 2 in North Hudson Regional Fire and Rescue was reduced from third to fifth.*fn1
The multiple choice portion of the promotional examination consisted of eighty-one questions relating to twelve different fire emergency "scenarios." The general instructions for this portion of the examination stated:
The Written Multiple-Choice Examination requires you to respond to a series of multiple-choice items based on each of 12 scenarios that comprise the test. All scenarios consist of a written description and at least one diagram that provides the information regarding each scenario. Immediately following the written description for each scenario, instructions are provided to inform you about the diagram(s) to which you should refer, and how many test items pertain to that scenario. Based on the information contained in the written description and diagram, you will select the best response from the four choices provided for each test item.
Winters's appeal is directed solely at the correct answer to question 40 under scenario 6. Scenario 6 stated as follows:
You are a recently promoted Battalion Chief. You have been dispatched to a report of a fire in a vacant warehouse. Police have reported smoke coming from the building.
The structure is 100' X 100' and of mill construction. It is 6:00 a.m. with a temperature of 67 degrees Fahrenheit and a wind speed of 10 - 15 mph accompanied by gusts of up to 25 mph blowing from the east to the west. Weather channel reports an imminent storm approaching.
From pre-planning, you know that homeless persons sometimes occupy the building.
Scenario 6 also included a diagram.
Question 40 asked:
The general collapse characteristic of a building of this type is:
(a) an early roof collapse.
(b) a parapet wall collapse near roof.
(c) a bearing wall collapse with instant floor collapse.
(d) a steel support beam separating from the main girder at the connection.
The subject matter experts who prepared this promotional examination for the Department of Personnel determined that the correct answer to question 40 was (c), "a bearing wall collapse with instant floor collapse." Winters's answer to question 40 was (c). Consequently, his answer was initially scored a correct answer.
Thirty of the persons who had taken the promotional examination appealed to the Merit System Board (subsequently superseded by the Civil Service Commission), challenging the correct answers to thirty-eight different questions. Winters was not one of the examination takers who questioned the correct answers. One of the challenged answers was the answer to question 40. The Board consolidated the appeals. In a lengthy written decision issued on August 17, 2007, the Board rejected most of the challenges to the correct answers on the promotional examination. However, the Board upheld the challenges to the answers to two questions, one of which was the answer to question 40, which the Board concluded should have been (a), "an early roof collapse." In reaching this conclusion, the Board stated:
Question 40 asked candidates for the general collapse characteristic of a building of this type. The keyed response was option c, a bearing wall collapse with instant floor collapse. The appellants argue for option a, an early roof collapse. They state that Norman indicates that a general collapse occurs with an early roof collapse, and that early collapses can occur with a large open area and peaked truss roof, such as this warehouse. In reply, the scenario described this structure as being 100'X 100' and of mill construction. The roof is not described, but appears to be a truss roof. A basement is not shown and it is not likely that a building this size has a basement. The diagram does not support mill construction and side a, where the fire is, is not a bearing wall. The correct response to this question should be rekeyed to option a, an early roof collapse.
Two of the original appellants petitioned the Board for reconsideration of this decision. One of them, Michael Curtin, challenged the rescoring of the correct answer to question 40 as (a) rather than (c).
In support of his petition, Curtin submitted a letter from John W. Norman, who is the author of Fire Officer's Handbook of Tactics and the expert referred to in the Board's decision rescoring the correct answer to question 40. The Board's decision on Curtin's petition for reconsideration contains the following description of Norman's letter:
Mr. Norman states that option c, a bearing wall collapse with instant floor collapse, is the recognized collapse hazard in mill buildings. He argues that he does not state in his book that mill buildings will suffer early roof collapses. He explains that mill construction uses heavy timber structural elements with brick bearing walls and large interior wood columns spaced twenty feet apart in all directions. He contends that a 100 ft by 100 ft mill building would have all four walls serving as bearing walls, with girders supporting the roof and resting on the columns and the bearing walls, and would not have a peaked roof.
Mr. Norman argues that the appellants who argued for option a incorrectly state that "general collapse occurs with early roof collapse, and that early collapses can occur with a large open area and peak truss roof." He states that the question did not ask for the causes of a general collapse, that peaked roofs would only be on narrower (40 feet wide) mill buildings, and that mill buildings are not meant to provide large open spaces but are built to support heavy loads. Since roof supporting elements for heavy timber structures must be a minimum of 4" by 6", the structural members take longer to burn. Hence, early structural collapse is not a characteristic of mill construction. He argues that mill buildings were built in the 1800s and almost every mill building in the Northeast has a basement. He states that incident commanders should assume a basement is present until proven otherwise by physical examination. He states that on page 423 (of the second edition), regarding heavy timber buildings he wrote "collapses in these buildings are often large-scale ones, with both walls and floors being affected," and he adds that if a collapse should occur, it would not happen early as "these buildings are generally quite stable due to the size of their load bearing members, which are generally 12 x 12 wooden columns and brick walls."
In a written decision issued on December 7, 2007, the Board rejected Curtin's challenge to the rescoring of the answer to question 40, stating:
The diagram shows a large, one-story, warehouse-type structure with a peaked roof. Along side a of the building, from left to right, is a double garage door, a two-door entrance, and seven windows. Along side d are four single garage doors. The decision below indicated that "the diagram does not support mill construction." However, based on the diagram, the building could be of mill construction, which is determined by the class of lumber, and the peaked roof could be heavy timber because of the scenario description, although the building is not a typical mill building. As such, there is continuing ambiguity between the description and the diagram. Mr. Norman's arguments that peaked roofs would only be on narrower (40 feet wide) mill buildings, and that mill buildings are not meant to provide large open spaces, simply are not germane to the information given by the scenario. The diagram clearly shows a peaked roof on a one-story building that is 100' by 100'. Mr. Norman's argument when applied to the diagram essentially supports that this is not a mill construction building.
Question 40 asked candidates for the general collapse characteristic of a building of this type, and the keyed response of option c, a bearing wall collapse with instant floor collapse, was changed to option a, an early roof collapse. A review of the additional information presented indicates that the best response to the question is option a, an early roof collapse. In the text, Building Construction for the Fire Service, Francis Brannigan discusses mill construction. On page 204, Brannigan states that "the ends of girders are fire cut to release in the event of a collapse without bringing the wall down. Sometimes a cast iron box is built into the wall to receive the end of the girder. Sometimes the beams are set on a corbelled brick shelf." In his letter, Norman states that on page 423 (of the second edition of Fire Officer's Handbook of Tactics), he wrote "collapses in heavy timber buildings are often large scale ones. Both walls and floors being affected." The next sentence in the text is "The impact of upper floors collapsing on lower floors can cause each successive floor to collapse." A truss roof could be considered to behave like an upper floor, and, due to the fire cut, the roof could collapse without the walls collapsing. In the third edition of Norman's text, page 17 displays a picture of a heavy timber building that has no floors or roof, but the walls are still standing. Nonetheless, option c did not read "a bearing wall collapse with instant roof collapse," but indicated an instant floor collapse.
To that end, while there is no mention of a basement or repair pits, generally, a heavy timber building would indicate a concrete or heavy timber floor. In this building, the presence of many garage doors suggests that you can drive at least six trucks, or large loading equipment pieces, inside. It is unlikely that this building has a basement given this fact pattern, although an [Incident Commander] might be concerned about repair pits and alterations. There was no mention of a basement in the scenario. A bearing wall collapse "with instant floor collapse" is improbable since a building that could hold six vehicles is unlikely to have a basement, and therefore a floor that could collapse. Further, there is little likelihood of a basement wall collapse, which would be a bearing wall, even if there was a basement. Given the facts of the scenario, option a, an early roof collapse, is the best response of the four options given.
Thereafter, Winters appealed from the rescoring of the answer to question 40.*fn2 In a decision issued on July 31, 2008, the Civil Service Commission, which had superseded the Merit System Board after its August 17, 2007 and December 7, 2007 decisions regarding the correct answer to question 40 on the fire officers' promotional examination,*fn3 rejected Winters's appeal, stating:
In his appeal, [Winters] argues that an authoritative text supports option c as a better response, the Board "misapplied" Mr. Curtin's arguments, the conclusion that "a basement is not likely to be in a mill constructed building of the size described" is erroneous in the northeastern part of the United States, and there is significant vagueness and inconsistency between the diagram in the test booklet and the scenario description.
The crux of Mr. Winters' arguments is his assertion that this is a building of mill type construction. In Curtin..., the Board decided that the diagram could support mill construction, but if so, it would not be a typical mill building, and the arguments given by Curtin's subject matter expert (SME), John Norman, when applied to the diagram, indicate that it is not a mill constructed building. Mr. Winters states that the question asked for the general collapse characteristic of a mill constructed building, and he proceeds with arguments that support that a bearing wall collapse with instant floor collapse best answers the question. While it is true that this type of collapse is consistent with mill construction, the layout of the diagram and description is such that this building could also be construed as a large warehouse. The information given simply does not confirm the type of construction of this building, nor does the question ask for the general collapse characteristic of a mill constructed building, and this ambiguity cannot be ignored when determining the best response to the question.
The Board noted in Curtin that option c involved an instant floor collapse, not an instant roof collapse, and the only floor that could collapse would be into a basement. Mr. Winters argues that mill constructed buildings have basements, but again, his citations are not germane to the building described in the scenario. This is not a church, school, or town hall, and it is not established that it is mill construction. If it were, as he argues, a heavy timber building usually indicates a concrete or heavy timber floor. Again, the presence of many garage doors for large vehicles suggests that there is no basement, and basements are not mandatory in warehouses. The construction of the building was ambiguous, the building could be either a large warehouse of an atypical type of mill construction, and the best response to the question must be based on the information given in the examination. It is possible that repair pits or other alterations could have been made to the building and an Incident Commander would be remiss if he ignored that possibility. The building clearly has a large roof, probably a truss roof, over a large open area, and there is no mention of a basement in the description or indication of one on the diagram which would substantiate a floor collapse after a bearing wall in the basement collapsed. The best response to the question remains option a, an early roof collapse. A bearing wall collapse with instant floor collapse, option c, is not the best response, and the appellant's arguments are insufficient to warrant a change in the key.
It is firmly established that "[a]n appellate court will reverse the decision of the administrative agency only if it is arbitrary, capricious or unreasonable or it is not supported by substantial credible evidence in the record as a whole." In re Taylor, 158 N.J. 644, 657 (1999) (quoting Henry v. Rahway State Prison, 81 N.J. 571, 581 (1980)). "[A]lthough the scope of review of an agency's decision is circumscribed, an appellate court's review of an agency decision is 'not simply a pro forma exercise in which [the court] rubber stamp[s] findings that are not reasonably supported by the evidence.'" Ibid. (quoting Chou v. Rutgers, 283 N.J. Super. 524, 539 (App. Div. 1995), certif. denied, 145 N.J. 374 (1996)). "Appellate courts must engage in a 'careful and principled consideration of the agency record and findings.'" Id. at 657-58 (quoting Mayflower Sec. Co. v. Bureau of Sec., 64 N.J. 85, 93 (1973)).
We conclude that the Civil Service Commission's conclusion that answer (c) to question 40 was a wrong answer, and that (a) was the only correct answer to this question, was arbitrary, capricious, and unreasonable. The Department of Personnel's own subject matter experts who devised the firefighter's promotional examination designated (c) as the correct answer to the question. Moreover, the Commission acknowledged that (c) was the correct answer to this question based on the verbal description of scenario 6, without consideration of the accompanying diagram.
However, the Commission concluded that the diagram created ambiguity concerning the correct answer and that the diagram was controlling even if it seemed to conflict with the verbal description of scenario 6. Thus, in its original decision in which it changed the correct answer from (c) to (a), the Board concluded that "[t]he diagram does not support mill construction" even though scenario 6 expressly stated that "[t]he structure is... of mill construction." In its decision on Curtin's petition for reconsideration, the Board stated that "based on the diagram, the building could be of mill construction, which is determined by the class of lumber," but then stated, again apparently based on the diagram, that "the building is not a typical mill building." Based on these observations, the Board concluded: "As such, there is continuing ambiguity between the description and the diagram." Nevertheless, the Board decided that answer (a) was the one and only correct answer to question 40. Similarly, in its decision denying Winters's appeal relating to the answer to question 40, the Commission stated that "the arguments given by Curtin's subject matter expert..., John Norman, when applied to the diagram, indicate that it is not a mill constructed building," and that "this ambiguity," apparently referring to the difference between the verbal description of the building in scenario 6 and the diagram, "cannot be ignored when determining the best response to the question."
There was nothing in the general instructions for completion of the multiple choice questions that gave notice to an examination taker that even though the verbal description of the fire emergency situation in one of the scenarios indicated that a building was of a particular type -- in the case of scenario 6 "mill construction" -- that the accompanying diagram could negate or modify that description. Moreover, the very expert cited by the Board in its initial decision changing the correct answer to question 40 from (c) to (a) concluded that (c) was the correct answer. Under these circumstances, it was arbitrary, capricious, and unreasonable for the Board and now the Commission to conclude that (c) was an incorrect answer to the question. Rather, in view of the ambiguity in the correct answer to question 40 that the agencies considered to have been created by the accompanying diagram, they should have either disregarded the answer to question 40 entirely in scoring the promotional examination or else "double-keyed" the answer, that is, treated either (c) or (a) as the correct answer.
Accordingly, we reverse the Commission's denial of Winters's appeal and remand the matter to the Commission to determine whether to disregard or double-key the answer to question 40 and to take other appropriate action to implement this decision.