Client
RedLab (Prev. CTIL)
Role
My Responsibilities
Timeline
Tools
Team
Optimizing firefighters' time to focus on critical tasks like emergency response and mental health recovery.
Inaccuracies can lead to legal consequences and reputational risks.
Writing incident reports for firefighters is not only time-consuming but also error-prone. It considerably increases their work stress, which has an adverse effect on their psychological well-being and job performance.
Project Scope
User interviews with 8 firefighters revealed this could accelerate their incident report generation by at least
Individual Impact
Led to addition of the data science team, to further research and develop the concept MVP
I conceptualized and designed an advanced incident reporting system for the firefighters. It leverages deep learning and Natural Language Processing (NLP) to analyze raw conversation data, automatically extracting key details and summarizing the incident. This generates a high-quality draft report, saving firefighters time and ensuring consistent reporting. After reviewing and making any necessary edits, firefighters can submit the final report with ease.
Teleport to the solution
Ready for the details? Let’s take a closer look
IU RedLab (formerly Crisis Technologies Innovation Lab) leads the charge in developing and deploying cutting-edge technologies that empower emergency responders on the front lines to better manage crisis and disaster situations.
This resource-intensive data entry task diverts valuable time and energy that can be better utilized for other critical tasks. Furthermore, the potential legal consequences and reputational risks associated with inaccurate incident reports underscore the urgent need to address this issue.
Here is a quick walkthrough of my proposed concept
(2 mins watch)
What prompted my exploration into AI/ML solutions?
Primary research finding
Primary research revealed that in the current operational flow, incident radio conversations are recorded but remain largely unused, representing an underexplored data resource. This prompted me to ask: How can we effectively leverage this data? Subsequently, I began exploring opportunities within the AI/ML domain.
Literature review
Similar research and implementations exist across various domains: healthcare (clinical data summarization), supply chain (inventory and warehouse report generation), education (lecture note generation), business (GPT implementations) etc.
While the concept may appear straightforward, we made some initial assumptions while ideating the concept. These assumptions necessitated further exploration into technical feasibility to assess the concept's true potential.
One of the major assumption was:
Assumption :
Based on successful implementations in other domains, we hypothesize that this concept could be transferable to the emergency response domain.
To validate the core assumptions of our concept, I needed a deeper understanding of core AI principles, generative AI, and Natural Language Processing (NLP). Through interviews and collaboration with subject matter experts (data scientists and researchers), I obtained valuable feedback on the proposed concept. This comprehensive research allowed for the identification of critical technical prerequisites to bring the vision to life.
Real data from incidents is required to train and fine-tune the model for generating the required output
A trained LLM model that can perform the Abstract Dialogue Summarization and data extraction using the input (real time incident data)
Servers, powerful computers that can handle substantial memory demands, stable network.
While previous research and implementations in other domains, have explored similar report generation concepts utilizing abstract data summarization techniques, our domain presented distinct challenges. To address these, I collaborated with the data science team to perform some pilot testing with existing SOTA base LLMs without pretraining to thoroughly analysis and identify and the current constraints and limitations.
Unique Nature of First Responder Conversations
Extensive Use of Code Words
Increased Background Noise and Interference
Estimating incident duration is complex
Requires substantial and continuously accessible data storage infrastructure
Now lets take a deeper dive in order to understand some of the additional domain challenges:
Therefore, the data science team performed some pilot testing to test the hypothesis. They used two SOTA text summarization models: Open AI's GPT 2 and SpaCy.
Initially the models were fed list of commonly used incident terminologies and firefighters codes to build some basic understanding (pretraining). Following that we asked the model to generate an incident summary for a transcript that we prepared from a incident training video (without data scrubbing or data cleansing).
Result:
Unclean raw data packed with firefighters' jargon hampered model accuracy. Concept implementation requires the model to have a solid understanding of firefighter communication in order to handle potential situational challenges such as network issues and noise affecting incident data.
Their final findings matched the initial hypothesis.
Zero Dataset
Zero Research
Zero Benchmarks
Added Domain Challenges
Intensive Need for Funds
No publicly accessible datasets that include actual incident conversation data (either recordings or transcripts).
No prior research on abstractive dialog summarization for Fire & Emergency service domain
Due to the absence of prior research, there is No established benchmarks.
Unique domain challenges makes state-of-the-art (SOTA) models ineffective against incident dialogue summarization
Exploring this uncharted domain requires significant financial investment, due to its novelty and complexity.
Significantly speeding up the report
generation process. Allocating time for critical stuff and reducing stress.
Testing: Model Algorithm
Tasked to: Data Science Team
Precision is crucial in fire and safety domains as inaccuracies may cause disruption and fatal consequences.
Testing: Model Output
Tasked to: Data Science Team
Concept validation and adoption; user trust impacts adoption, but over-reliance risks errors.
Testing: Prototype (wt. Firefighters)
Tasked to: UX Team
With a deep understanding of backend constraints, our data science team delved into tackling additional domain challenges, while our design team pivoted towards designing the frontend for the AI integration.
Goal: Brainstorming ideas to integrate the AI tool seamlessly while ensuring consistency, ease of use and familiarity.
After exploring various ideas, the team selected one that offers practical innovation, prioritizing ease of integration and direct benefits without compromising robust reporting standards.
Through rigorous design sprints with the team, we iteratively refined our initial designs, gathering valuable feedback from both stakeholders and firefighters. Over three iterative rounds, we refined each detail to create the high fidelity prototype.
01 Advanced incident management dashboard
Integrating all-in-one cohesive enterprise solution
Personalize your workspace with widgets that deliver real-time analytics, putting crucial information at your fingertips. Further embrace AI integration for limitless potential in enhancing operational efficiency and safety.
02 New incident reporting flow
Simplified workflow and amplified efficiency
Optimized NFIRS user flows, ensuring intuitive navigation and minimal disruption. Your team can effortlessly leverage advanced technology without missing a beat in their critical operations.
.gif)
03 Firegen: AI integration
AI automation for precise & swift incident documentation
Analyzing raw radio data, the system automatically extracts essential information, auto populates and creates a draft report. Firefighters can then swiftly validate and submit the draft, streamlining the reporting process with accuracy and efficiency.
04 Firefighter-centric design
Enhanced experience with easy editing
Designed with firefighters' mental models in mind, the platform simplifies data entry for firefighters by offering a dedicated section for editing all form field values. Select the field, move to the edit response section, make changes, and save (auto).
Assumption :
User might get confused regrading which values might have been auto-filled by the AI and could lead to error.
05 Keyword-based data categorization
Introducing novel keyword categorization
To optimize AI-driven report generation, we've introduced an innovative keyword-based categorization system.
We classify incident transcript data into five key categories:
Identifiers (e.g., "Street; Intersection; Incident"); Extraction Data (e.g., "West Michigan St."); Sectional Keywords (e.g., "Arrived at location"); Metadata (e.g., fire department details); and Filler Data (anything that cannot be classified or situational content).
This structured approach enhances data processing efficiency, ensuring vital information is accurately extracted for streamlined reporting.
I presented the prototype (v3.0) designs to the stakeholders and the project team to gather initial feedback before testing with firefighters. The stakeholders raised some insightful questions about the current designs that we will be validating during user testing with firefighters. They also shared some ideas that could improve platform usability.
#1 Source representation
Do firefighters want to see the source?
Would they prefer a link to the actual transcript?
Rationale behind the existing design approach:
provide enough detail to the firefighter to validate the data extracted by the AI without overwhelming them or adding cognitive load
#2 Edit response
Is it easier for firefighters to have a separate section to edit response (validate assumption) ?
Rationale behind the existing design approach:
simpler workflow and prevent possibilities of unintended errors (act as two-factor verification)
#3 Response change history (idea)
What if someone wants to verify who made specific changes (validate use case)?
Rationale behind the existing design approach:
use case was not identified initially
This project marks a pivotal moment in leveraging the latest advancements in AI/ML in the fire and safety domain. By addressing critical domain challenges and collaborating closely with fire departments, we aim to pioneer new frontiers. Training AI models like LLM promises enhanced accuracy and customization, setting benchmarks for future advancements.
Here are some potential future directions for AI integration into the Incident Management platform:
01 Diverse report generation
Automatically generate a variety of comprehensive reports tailored to different stakeholders and needs.
02 Anomaly Detection
Quickly detect anomalies in incident data, enabling proactive management and response to unexpected situations.
03 Individual performance analysis:
Utilize AI to analyze and provide insights into the performance of individual firefighters based on incident data and historical trends.
04 Team performance enhancement
Provide actionable insights and recommendations to improve overall operational performance of the team based on AI-driven analysis of incident handling and response strategies.
Next we need to rigorously test the prototype (v3.0) with firefighters. This testing will determine if this approach aligns with their mental models and effectively balances ease of use with the required control.
Engaging with Subject Matter Experts (SMEs) allowed me to identify key long-term milestones that would be crucial for shaping the project.
Navigating Leadership
Spearheading the project from start taught me the vital role of effective leadership. Successful project completion hinges on skills such as effective communication, task delegation, informed decision-making, and most importantly, the willingness to take responsibility, including accepting failure when necessary.
Data Driven Approach
One of the key learnings from this project was the importance of having a data-driven approach. A major accomplishment I had in this project was effectively demonstrating the project's potential to clients through my research-backed project proposal and impactful visuals, which actually led to team expansion and additional resource allocation.
Collaborative Synergy
Cultivated essential interpersonal skills through collaboration with diverse professionals (firefighters, researchers, entrepreneurs, and data scientists). Emphasized active listening, empathy, and personal connection to foster collaboration, cultivating positive team dynamics.
.webp)
#halamadrid
