Pre-Simulation Survey (approximately 5 minutes)
You will answer brief questions about your professional background (rank, years of service, department type, prior AI experience) and your general attitudes toward AI technology.
Introduction
Welcome to the AI-Assisted Incident Command Decision-Support Study
Thank you for your interest in this research study conducted through the University of Maryland Global Campus (UMGC) Doctor of Business Administration program.
The purpose of this study is to evaluate how artificial intelligence (AI) decision-support tools present threat assessments and tactical recommendations to fire service incident commanders during emergency scenarios. Specifically, this research examines how different formats of AI-generated information influence command decision-making under time-pressured, high-consequence conditions.
Your participation will involve completing a brief pre-simulation survey, interacting with a simulated structure fire scenario that includes an AI decision-support system, and completing a short post-simulation survey. The entire session takes approximately 15–20 minutes.
Your professional expertise as an incident commander is essential to this research. The findings will contribute to evidence-based guidance for designing AI decision-support systems and training programs for emergency management professionals.
Please review the informed consent information on the next page before proceeding.
Informed Consent
Informed Consent for Participation in Research
Study Title
Explainability and Framing Effects on AI-Assisted Decision-Making: An Experimental Study of Incident Managers Under Simulated Emergency Conditions
Principal Investigator
David Povlitz, Doctoral Candidate, DBA, UMGC
Faculty Advisors
Dr. Tacy Holliday, UMGC
Dr. Jason Stroman, UMGC
Dr. Jason Stroman, UMGC
IRB Contact
You are invited to participate in a research study. Please read the following information carefully before deciding whether to participate.
Purpose of the Study
This study examines how AI decision-support systems communicate threat assessments and recommendations to incident commanders during emergency scenarios. The goal is to generate evidence-based guidance for designing AI tools that support effective emergency management decision-making.
Procedures
If you agree to participate, you will: (1) Complete a brief pre-simulation survey about your professional background and attitudes toward AI technology, taking approximately 5 minutes; (2) Receive an orientation to the simulation interface and controls; (3) Interact with a web-based simulation of a residential structure fire scenario in the role of incident commander, during which an AI system will provide threat assessments and tactical recommendations at several decision points—the simulation lasts approximately 4–5 minutes; and (4) Complete a short post-simulation survey about your experience with the AI system, taking approximately 2–3 minutes. The total time commitment is approximately 15–20 minutes.
Voluntary Participation and Right to Withdraw
Your participation is entirely voluntary. You may decline to participate or withdraw from the study at any time without penalty or loss of any benefits to which you are entitled. You may skip any survey question you do not wish to answer. If you withdraw, any data collected up to that point will be discarded and not included in the analysis.
Risks
The risks associated with this study are minimal and do not exceed those ordinarily encountered in routine incident command training exercises. You may experience mild cognitive fatigue or brief stress due to the time-pressured decision-making scenario. The simulation does not depict graphic imagery. You may pause or stop the session at any time if you experience discomfort.
Benefits
There are no direct benefits to you from participating. Your participation will contribute to research that may improve the design of AI decision-support tools and training programs for emergency management professionals.
Confidentiality and Anonymity
All data collected in this study are anonymous. No personally identifiable information, including your name, email address, department name, I.D. number, or IP address, is collected or stored at any time. You will be assigned a randomly generated participant code. Your responses cannot be linked to your identity. All data will be stored on encrypted, password-protected media accessible only to the principal investigator and retained for at least three years before secure deletion. Demographic questions use broad response categories to prevent the identification of individuals in small departments or specialized roles.
Contact Information
If you have questions about this study, please contact the principal investigator, David Povlitz, at dpovlitz@student.umgc.edu. If you have questions about your rights as a research participant, please contact the UMGC Institutional Review Board at irb@umgc.edu.
By clicking "I Consent – Begin Study" below, you confirm that you have read and understood this information, that you are 18 years of age or older, that you hold the rank of battalion chief or higher, that you have completed ICS-100 and ICS-200 certifications, and that you voluntarily agree to participate in this study.
Session Overview
Simulation Orientation: What to Expect
Thank you for consenting to participate. Before you begin, here is an overview of what your session will involve.
Your session consists of four parts and will take approximately 15–20 minutes total:
-
1
-
2
Simulation Orientation (approximately 2 minutes)
You will familiarize yourself with the simulation interface, including how to read the AI system's threat assessments, how to view the scenario information displays, and how to submit your decisions using the action buttons.
-
3
Simulated Scenario (approximately 4–5 minutes)
You will assume the role of the incident commander arriving on scene at a residential structure fire. The scenario will evolve through several critical decision points. At each point, an AI decision-support system will present a threat assessment and a tactical recommendation. You will review the available information and select your command decision. There are no trick questions; choose the decision you believe is most appropriate given the information presented.
-
4
Post-Simulation Survey (approximately 2–3 minutes)
After the scenario concludes, you will answer a few short questions about your experience with the AI system.
At Each Decision Point, You Will See:
(1) A situation briefing describing current fireground conditions; (2) the AI system's threat appraisal of the evolving situation; (3) information about potential outcomes; and (4) the AI system's recommended tactical action. You will then select one of the available options: ADOPT the AI's recommendation or OVERRIDE it with alternative tactical decisions.
Important Reminders
There are no right or wrong answers—the study is examining how AI information presentation affects command decision-making, not evaluating your performance as an incident commander. Respond as you would in an actual incident command situation based on the information available to you. The scenario is time-pressured; make decisions at a pace that feels realistic to you. You may stop the session at any time.
Part 1 of 4 — Pre-Simulation Survey
Anonymous Demographic and Professional Information Questionnaire
The following questions collect anonymous background information about your professional experience and demographics. No personally identifiable information is collected. Your name, department name, badge number, and specific geographic location are not requested. Please select the response that best describes you for each item. You may skip any question you prefer not to answer.
Section 1 — Demographic Information
1What is your age?
2What is your gender?
3What is your race or ethnicity? (Select all that apply)
4What is the highest level of education you have completed?
Section 2 — Professional Background
5What is your current rank or position? (Select the highest that applies)
6How many total years have you served in the fire service?
7How many years have you served in a command or chief officer role (battalion chief or higher)?
8What type of department or agency do you currently serve?
9What is the approximate size of the community your department or agency primarily serves?
10Approximately how many personnel (career and/or volunteer) are in your department or agency?
Section 3 — Incident Command Experience
11What is the highest NIMS/ICS certification you have completed? (Select the highest)
12In the past 12 months, approximately how many incidents have you commanded or co-commanded?
13How often do you participate in simulation-based training exercises?
Section 4 — Artificial Intelligence and Technology Experience
14Prior to this study, have you used any AI-powered tools or systems in your professional work?
15If you have used AI-powered tools professionally, in which context(s)? (Select all that apply)
16How would you rate your overall comfort level with using computer-based technology in your professional role?
Part 1 of 4 — AI Attitudes Scale
General Attitudes towards Artificial Intelligence Scale (GAAIS)
Source: Schepman & Rodway (2023). International Journal of Human–Computer Interaction, 39(13), 2724–2741. doi:10.1080/10447318.2022.2085400
Please indicate how much you agree or disagree with each of the following statements about artificial intelligence (AI). There are no right or wrong answers. Please respond based on your general attitudes, not your experience in today's simulation.
Positive Subscale — 12 Items
| Statement | 1 Strongly Disagree |
2 Disagree |
3 Neutral |
4 Agree |
5 Strongly Agree |
|---|
Negative Subscale — 8 Items
Note: One item in this section is an attention check. Please read each statement carefully before responding.
| Statement | 1 Strongly Disagree |
2 Disagree |
3 Neutral |
4 Agree |
5 Strongly Agree |
|---|
Part 3 — Simulation Background
Pre-Arrival Incident Briefing
LIFE HAZARD: CRITICAL — Multiple Non-Ambulatory Occupants Reported. Strategy: OFFENSIVE — Rescue, Confinement, Extinguishment.
Incident Information
Incident
Structure Fire — Single-Family Dwelling (1-Story)
Date / Time
10:00 AM
Address
Generic Residential Address, Sector Alpha
Weather
Sunny, 50°F, Wind: Light 5–8 mph from West
Occupancy
Single-Family Dwelling, 1-Story, Slab Foundation, Wood Frame, Comp Shingle Roof
Life Hazard
CRITICAL — Multiple Non-Ambulatory Occupants Reported
Strategy
OFFENSIVE — Rescue, Confinement, Extinguishment
Command Post
Alpha Side — Front of Structure (Street)
Scene Narrative and Background
At 10:00 AM on a clear, sunny morning with a temperature of 50°F and light westerly winds, dispatch receives multiple 911 calls reporting fire and heavy smoke from a one-story, wood-frame, single-family dwelling in a residential neighborhood. Callers report visible flames from the front (Side Alpha) of the structure and heavy black smoke venting from windows on the Bravo (left) side. One caller specifically reports seeing elderly occupants prior to the fire. Two additional callers independently state they believe occupants are still inside and cannot get out.
The structure is an approximately 1,500 square foot, single-story residence with a composition shingle roof and wood-frame construction on a concrete slab. A hydrant is located 150 feet from the front of the structure on the Alpha side.
Life safety is the immediate priority. Three occupants have been reported as non-ambulatory and are confirmed trapped. The fire appears to have originated in the Alpha/Bravo (front-left) quadrant and is spreading rapidly toward the rear or Charlie side. Standard operating procedures recommend an offensive interior attack with concurrent search and rescue, ventilation, and fire confinement operations.
Structure Layout and Fire Conditions
| Charlie Side (Rear) | Bravo Side (Left) | Alpha Side (Front) | Delta Side (Right) |
|---|---|---|---|
| Main entry door Front windows Driveway approach Command Post location |
Bedroom windows Heavy smoke showing |
FIRE ORIGIN AREA Active flames visible Kitchen/utility room Command Post location |
Light smoke showing from attic vent |
Dispatch Resource Assignments
| Unit | Personnel | Assignment | Arrival | Initial Task |
|---|---|---|---|---|
| Engine 1 | 4 | Initial Attack — Lead Engine | Minute 1 (10:01) | Size-up, establish water supply (hydrant), deploy attack line — Charlie side |
| Engine 2 | 4 | Back-up Attack / Search | Minute 1 (10:01) | Backup attack line, assist search operations Charlie side |
| Engine 3 | 4 | Water Supply / RIC | Minute 3 (10:03) | Supplement water supply, establish Rapid Intervention Crew (RIC) |
| Engine 4 | 4 | Staging / Rehabilitation | Minute 5 (10:05) | Stage in designated area; support rehab and relief operations |
| Truck 1 | 4 | Primary Search and Rescue | Minute 1 (10:01) | Forcible entry, primary search Alpha/Bravo quadrant, victim removal |
| Truck 2 | 4 | Ventilation / Secondary Search | Minute 4 (10:04) | Horizontal ventilation, roof assessment, secondary search Charlie/Delta quadrant |
| Rescue 1 | 4 | Rescue — Victim Extrication | Minute 4 (10:04) | Technical rescue support, victim packaging/extrication, EMS support |
| Medic 1 | 2 | EMS / Triage | Minute 3 (10:03) | Establish treatment area, triage and treat rescued victims |
| Chief 1 | 1 | Incident Commander | Minute 2 (10:02) | Assume command, direct all operations, establish command post Alpha side |
Radio Communications — Pre-Arrival through Minute 2
Pre-Arrival — Dispatch Notifications (09:58 – 10:00)
09:58:30 DISPATCH → ALL UNITS Structure fire assignment — residential occupancy, one-story single-family dwelling. Multiple callers report fire and heavy smoke. Reports of non-ambulatory occupants unable to evacuate. Engine 1, Engine 2, Engine 3, Engine 4, Truck 1, Truck 2, Rescue 1, Medic 1, and Command 1 — respond on Channel Fire Bravo.
09:59:10 E1 → DISPATCH Engine 1 is responding to the structure fire at 207 South Street.
09:59:15 E2 → DISPATCH Engine 2 is responding.
09:59:18 T1 → DISPATCH Truck 1 is responding.
09:59:20 CMD → DISPATCH Command 1 is responding. Confirming reports of non-ambulatory occupants and multiple fire/smoke reports?
09:59:25 DISPATCH → CMD Command 1 — affirmative. Three separate callers report occupants unable to self-evacuate. Flames observed from the front of the structure. Hydrant confirmed 150 feet north on the Alpha approach.
09:59:40 E3 → DISPATCH Engine 3 is responding.
09:59:45 M1 → DISPATCH Medic 1 responding.
Minute 1 — First Units On Scene (10:01:00) — Heavy black smoke from Bravo windows. Visible flames from Alpha/Bravo corner. Fire spreading. Life hazard confirmed.
10:01:05 E1 → DISPATCH Engine 1 is on location. We have a working structure fire — one-story, single-family dwelling, approximately 1,500 square feet. Heavy black smoke from the Bravo side windows, visible flame from the Alpha/Bravo corner. I am laying a supply line from the hydrant on Alpha approach. Engine 1 is establishing water supply and advancing an attack line to the Charlie side entry. Life hazard is confirmed. This is Engine 1 establishing the SECOND STREET COMMAND.
10:01:20 DISPATCH → ALL All units — Engine 1 is establishing COMMAND at a working structure fire.
10:01:25 E2 → E1 CMD Engine 2 is on location. Requesting assignment.
10:01:28 CMD (E1) → E2 Engine 2 — advance a backup attack line to the Charlie entry. Assist with initial search operations on the Bravo side. Stand by for Command 1 arrival.
10:01:32 T1 → CMD Truck 1 is on location. Requesting assignment.
10:01:35 CMD (E1) → T1 Truck 1 — perform forcible entry at the Bravo side main door. Initiate primary search of the Alpha and Bravo quadrants. Confirmed non-ambulatory occupants — priority is victim location and removal. Keep me advised.
10:01:40 T1 → CMD Truck 1 copy. Forcing Alpha entry now. T1 Officer and crew going in for primary search. T1 Driver will remain outside for aerial and forcible entry support.
10:01:50 E1 → CMD Engine 1 attack crew — water is on. Advancing a 1¾-inch line to the Charlie entry. Smoke is heavy, visibility is near zero inside. We have fire showing in the front hallway toward Alpha side.
Minute 2 — Command 1 On Scene, Assumes Command (10:02:00) — Fire spreading to interior hallway. Smoke now pushing from Alpha side windows. Conditions deteriorating.
10:02:05 CMD → ALL UNITS Command 1 is on location. I am assuming command of this incident. This is COMMAND — establishing the command post at the Alpha side, front of structure in the driveway. ALPHA COMMAND is in effect.
10:02:08 DISPATCH → ALL All units — Chief 1 has assumed SECOND STREET COMMAND.
10:02:12 CMD → ALL UNITS 360-degree size-up is complete. Structure is a single-story wood-frame dwelling. Fire is in the Alpha-Bravo quadrant and spreading through the hallway toward Charlie. Heavy smoke is banking down throughout the structure. We have three confirmed non-ambulatory occupants. Rescue is the number-one priority. Strategy is OFFENSIVE.
Tactical Worksheet — Minute 2 (10:02:00)
Incident Commander: Command 1 — Chief Officer (Alpha Command — Command Post: Alpha Side Driveway)
Command 1 assumes command. Full size-up complete. Strategy confirmed OFFENSIVE. Three non-ambulatory victims confirmed. Rescue is Priority 1.
Strategy: OFFENSIVE | Mode: INTERIOR ATTACK | Status: OFFENSIVE — RESCUE PRIORITY | CONFINEMENT AND ATTACK
Command 1 assumes command. Full size-up complete. Strategy confirmed OFFENSIVE. Three non-ambulatory victims confirmed. Rescue is Priority 1.
Strategy: OFFENSIVE | Mode: INTERIOR ATTACK | Status: OFFENSIVE — RESCUE PRIORITY | CONFINEMENT AND ATTACK
Fire Attack Group
Engine 1 (Attack Crew)Interior — Charlie hallway
Fire confinement
1¾" line — water on
Backup Attack
Engine 2 (4 Personnel)Backup line — Bravo entry
Search and Rescue Group
Truck 1 Officer (Group Supervisor)T1 Crew — Alpha/Bravo search
Ventilation
Assigned, not yet on scene —Truck 2 (ETA min 4)
Horizontal ventilation planned
Bravo/Delta windows
EMS / Treatment
Medic 1 (ETA min 3)Treatment area planned Alpha
RIC — Rapid Intervention
Not yet established(E3 ETA min 3)
Staging
Not yet establishedPart 3 — Simulation Instructions
AI Command Assistant — Operator Instructions
Research Focus Areas
- How incident commanders integrate AI-generated situation assessments with their own training and experience
- The quality, timing, and rationale behind command decisions made with AI support
- How participants evaluate, accept, question, or override AI recommendations
- Cognitive workload and confidence levels during the simulation
- The overall effectiveness of the AI Command Assistant Agent as a decision support tool
The AI Command Assistant Agent is a tool — not a replacement for command authority. The research team is specifically interested in how experienced fire service professionals apply their training and judgment in combination with AI-generated information. There are no wrong answers. You are the Incident Commander. You make the decisions.
How the AI Agent Operates
The AI Command Assistant Agent operates by continuously analyzing three data streams simultaneously:
Video analysis: Live and recorded video feeds from scene cameras, aerial assets, and body-worn cameras on operating crews. The agent identifies fire behavior, structural conditions, victim locations, crew positions, and exposure risks.
Radio traffic: Real-time monitoring and transcription of all fireground radio communications. The agent tracks unit assignments, PAR status, conditions reported by interior crews, and deviations from assigned tasks.
Environmental sensors: Data from environmental monitoring sensors including atmospheric readings, thermal imaging overlays, wind speed and direction, structural heat sensors, and SCBA telemetry where available.
AI Agent Output Types
| Output Type | Description |
|---|---|
| Situation Assessment | A concise, current picture of incident conditions — fire location and spread, victim status, crew positions, hazards, and resource status. Updated continuously as conditions change. |
| Threat Assessment | Identification and prioritization of imminent and developing threats — to life safety, structural integrity, crew safety, and exposure structures. Includes confidence level and supporting data. |
| Strategic Recommendations | High-level command guidance — recommended strategy (offensive/defensive), priority objectives, resource allocation, and staging recommendations based on current conditions. |
| Tactical Recommendations | Specific operational suggestions — unit assignments, positioning, ventilation coordination, water supply decisions, search priorities, and egress routes. |
| Decision Prompts | At key moments during the simulation, the AI will prompt you with a decision point. You will be asked to make a command decision based on your training, experience, and the AI's analysis. |
Your Role vs. the AI's Role
Command authority remains with you.
The AI Command Assistant Agent is an advisory tool. It does not issue orders, take independent action, or have authority over any unit on the fireground. All command decisions — including whether to accept, modify, or reject any AI recommendation — are yours alone.
The AI Command Assistant Agent is an advisory tool. It does not issue orders, take independent action, or have authority over any unit on the fireground. All command decisions — including whether to accept, modify, or reject any AI recommendation — are yours alone.
- You may accept an AI recommendation in whole or in part
- You may reject any AI recommendation
- You may ask the AI to elaborate or justify a recommendation
- You may override the AI based on your own assessment at any time
The AI does not feel offended. It does not second-guess you. It continues to provide support regardless of whether you agree with it.
Part 3 of 4 — Command Simulation
Incident Command Simulation
Command AI Assistant: Standby
AI Command Assistant — Fire Detection Analysis
Fire Involvement (AIFireInvolvement)
--
Hazard Assessment (AIHazardAssessment)
--
Risk Classification (AIRiskAssessment)
None
Command AI Assistant initializing. Start video to begin real-time fire detection analysis.
Incident Status
Simulation Time00:00
Video StatusStopped
StrategyOFFENSIVE
Life HazardCRITICAL
Command Log
--:-- Simulation loaded. Awaiting start.
Unit Assignments
Engine 1Interior Attack
Engine 2Backup / Search
Truck 1Search and Rescue
Engine 3ETA Min 3
Medic 1ETA Min 3
Truck 2 / Rescue 1ETA Min 4
Incident Command Decisions and Assessment
AI Command Assistant Recommendations
AI recommendations will appear here once the simulation begins...
Decision Point 1 — Active at 60 seconds
Decision Point 2 — Active at 120 seconds
Decision Point 3 — Active at 180 seconds
Decision Point 4 — Active at 240 seconds
Part 4 of 4 — Post-Simulation Survey
Post-Simulation Trust Assessment
You have just completed the simulated incident command scenario. The following questions ask about your experience with the AI decision-support system you used during the simulation. Please rate each statement based on your interaction with the AI system in the scenario you just completed.
Source: McGrath, R., Lamba, R., Bhatt, S., & Maloney, A. (2025). A short trust in automation scale. International Journal of Human–Computer Interaction, 41(3), 1–13. doi:10.1080/10447318.2024.2366182
Trust in AI System — 3-Item Scale
| Statement | 1 Not at all |
2 Slightly |
3 Somewhat |
4 Moderately |
5 Quite |
6 Very |
7 Extremely |
|---|---|---|---|---|---|---|---|
| I am confident in the system. | |||||||
| The system is reliable. | |||||||
| I can trust the system. |
Debriefing
Study Explanation and Thank You
Participant ID: ---
Thank you for completing this research study. Your participation is greatly valued and contributes to important research on AI-assisted decision-making in emergency management.
Full Explanation of the Study
This study investigates how two specific features of AI decision-support systems influence incident commanders' decisions during emergencies: (1) the level of explainability provided by the AI system, and (2) how decision outcomes are framed.
AI Explainability Manipulation
Participants were randomly assigned to either a high-explainability condition, in which the AI system provided detailed reasoning for its threat assessments (including perception data inputs, reasoning chains, and confidence levels), or a low-explainability condition, in which the AI provided only its threat classification and recommended action without disclosing its underlying reasoning. This manipulation tests whether AI transparency enables the cognitive appraisal processes that protection motivation theory (Maddux & Rogers, 1983) identifies as prerequisites for appropriate protective action.
Gain–Loss Framing Manipulation
Participants were also randomly assigned to either a gain-frame condition, in which AI recommendations emphasized the positive outcomes of compliance (e.g., lives saved, structures preserved), or a loss-frame condition, in which the AI emphasized the negative consequences of noncompliance (e.g., lives lost, structures destroyed). The numerical information was equivalent across conditions; only the framing language differed. This manipulation tests prospect theory's (Tversky & Kahneman, 1992) prediction that loss framing induces different risk preferences than gain framing.
What We Are Studying
The central research question is: Under simulated emergency conditions, how do the AI explainability of threat appraisals and the situation's gain–loss framing of outcomes influence incident commanders' appropriate adoption of AI recommendations with time-bound, high-consequence decisions? Your decisions during the simulation help us understand how AI system design features influence command decision-making, which will inform the development of more effective AI tools for incident management and command decision support.
Important Note
The AI system in the simulation was designed for research purposes. Its recommendations were trained and validated by subject-matter experts and found to be objectively appropriate for the simulated conditions. The study did not evaluate your competence or decision-making ability as an incident commander. Instead, it examined how different AI presentation formats influence decision-making behavior among groups of participants.
Your Participant ID
Your randomly generated participant ID is displayed at the top of this page. If you wish to withdraw your data from the study at any time, please provide this ID to the researcher. Because no personally identifiable information was collected, this code is the only way to locate and remove your data.
Contact Information
- Principal Investigator:David Povlitz — dpovlitz@student.umgc.edu
- Faculty Advisor:Dr. Tacy Holliday — tacy.holliday@faculty.umgc.edu
- UMGC IRB:irb@umgc.edu
Support Resources
If you experienced any distress during the simulation scenario or have concerns about stress reactions, the following resources are available:
- SAMHSA Disaster Distress Helpline: 1-800-985-5990 (24/7 crisis counseling)
- National Alliance on Mental Illness (NAMI) Helpline: 1-800-950-6264
- Fire/EMS Helpline (National Volunteer Fire Council): 1-888-731-3473
We respectfully ask that you not discuss the specific details of the experimental conditions (the AI explanation levels and outcome framing) with colleagues who may also participate, as doing so could influence their responses and compromise the validity of the research.
Session Data
Study Session Data
All data collected during this session. Participant ID:
Session Metadata
Pre-Simulation Survey — Demographics and Professional Background
General Attitudes towards AI Scale (GAAIS)
Simulation Variables
Post-Simulation Trust Scale
Full Event Log (all recorded changes with timestamps)