You can upload your fire department's report exports into an AI chat, but you may not get the same answer twice. We tested it on Norfolk Fire Department's real inspection data in October 2026. With the report files uploaded, the AI's answers to four inspection questions came back like this: one headline matched the records, two were misleading, and one was wrong. Asked the same question four times, it gave three different numbers. Connected to the same records through FlorianAI, the same AI matched the records on all four questions and gave the same answer every run. We tested Claude only. We did not test ChatGPT.
FlorianAI, an AI operations assistant built for fire departments, connects the assistant a department already uses to its own records. This post walks through exactly what we asked, what came back, and why the two approaches split.
Why Not Just Upload Your Data Into ChatGPT?
It is a question chiefs ask us: "Why use FlorianAI when we could upload our data into ChatGPT?"
It is a fair question. Uploading costs nothing extra if you already have an AI account. You export a few reports from your RMS, drop them into a chat window, and ask. For drafting a memo or summarizing a standard, that works fine.
The real question is narrower: can you trust the numbers? A chief who takes a count of open violations to a select board, or an overdue-inspection figure to a budget meeting, needs that number to match the records, and needs it to be the same number if somebody asks again next week. So instead of arguing the point, we tested it on a real department's data.
How We Tested Norfolk Fire Department's Inspection Data
Norfolk Fire Department in Massachusetts runs FlorianAI and gave us permission to publish the results. The test used the department's own inspection, violation, and building exports from 23 October 2026:
- 674 inspections
- 35 violations
- 4,543 buildings
We asked the same four questions, word for word, in a fresh Claude chat on each side:
- How many inspections were completed?
- How many violations are still open, and which is the oldest?
- How many inspections are overdue?
- How many buildings have never been inspected?
On one side, the AI got the department's report files uploaded directly into the chat. On the other, the same AI was connected to FlorianAI. Every answer was graded against the department's own export files, not against the other side's answer. Two of the questions were then asked four times each to check consistency, and two were timed with a stopwatch across three runs per side.
The same assistant ran on both sides. The only thing that changed was how it reached the data.
What Happened When the Inspection Reports Were Uploaded?
Inspections completed. The uploaded-files answer got the headline count right. But of the six supporting claims it made along the way, three were wrong or unsupported by the records. Through FlorianAI, every figure we checked was right.
Open violations and the oldest. This is where the uploaded files went wrong. The AI said 12 violations were open. The records show 8. It had decided the files contained no field recording whether a violation was open, then invented its own rule for what "open" meant. Through FlorianAI, the answer was 8, every run.
Overdue inspections. The uploaded-files answer reached the right count, then led with a guessed "real backlog" figure that the records do not support. A chief skimming the response would likely have walked away with the guess. Through FlorianAI, the answer stuck to what the records show.
Buildings never inspected. The uploaded-files answer overstated the number, and the number kept moving. Through FlorianAI, the answer was the same every run. One honest caveat: a supporting count in FlorianAI's answer was off by one. The headline was right every time.
Final scorecard: uploaded files produced one answer whose headline matched the records, two that were misleading (one led with a guessed backlog figure, one overstated the count), and one that was wrong. FlorianAI matched the records on four of four.
Does AI Give the Same Answer If You Ask the Same Question Twice?
With uploaded files, not reliably. A trustworthy answer should not change when nothing in the records has changed. So we asked two of the questions four times each.
How many violations are still open? The records say 8.
Uploaded files answered 12 on the first run, then 8 three times. Three right out of four is not bad, until you notice that the wrong answer read just as confidently as the right ones.
How many buildings were never inspected?
Three different numbers in four runs, while FlorianAI returned 4,023 every time. The uploaded-files runs kept redefining what "inspected" meant, so each run was answering a slightly different question without saying so.
That is the failure that should worry a chief most. Not a number that is obviously wrong, but a number that looks right, sounds sure of itself, and would have come out differently if you had asked five minutes later.
Why Does Uploading Files Give Different Answers Than Connecting Your Data?
Uploading asks the AI to be your records clerk on the spot, every single time. Four differences explain the results.
What the words mean. With uploaded files, the AI guesses what "open," "completed," and "inspected" mean from column names, fresh on every run. In the open-violations question, it decided "open" was not recorded at all and made up its own rule. With FlorianAI, those definitions are written once in code: an inspection counts as completed only if it has a completion date, and "open" is the department's own violation status.
Definitions are not a technicality in fire prevention. Whether an inspection is overdue depends on how often that occupancy is supposed to be inspected, and NFPA 1730 ties inspection frequency to occupancy risk. If the AI invents a different rule each time it reads the file, the overdue number moves with it.
Who does the math. With uploaded files, the AI does the arithmetic itself. Some figures are computed, some are estimated, and the answer does not say which is which. With FlorianAI, FlorianAI computes every number, and the AI quotes it and explains it.
The same question tomorrow. An uploaded-files answer may come back with a different number or a different definition. FlorianAI applies the same rules and returns the same answer until the records themselves change.
Data and upkeep. To keep uploaded answers current, someone has to re-export and re-upload the reports every day. For Norfolk that meant 22 files, about 15 MB, and a copy of the department's data going into each chat. With FlorianAI, the data is imported once a day, it stays in the department's system, and only signed-in, authorized users can query it.
The chief asks the question. FlorianAI keeps the rulebook.
Is a Connected AI Faster Than Uploading Files?
In this test, yes, on both the simple question and the complex one. We timed three runs per side with a stopwatch and averaged them. FlorianAI averaged 9.7 seconds on the simple question (one file, 35 rows) and 11.7 seconds on the complex one (two files joined, about 5,200 rows), against 25.5 seconds for uploaded files on both: about 2.6 and 2.2 times faster.
The spread mattered as much as the average. FlorianAI's runs varied by about a second. The uploaded-files runs varied by up to 11 seconds. FlorianAI's slowest run was still about twice as fast as the fastest uploaded-files run.
Seconds are not the point on their own. Waiting for a slower answer that might also be wrong is the point.
What Should Chiefs Expect From AI for Fire Department Data?
Be clear about what this test is and is not. It was four questions on one department's inspection data, asked in Claude. It is not a study of every AI tool or every kind of fire department data, and we did not test ChatGPT. One of FlorianAI's supporting counts was off by one, and we are telling you so because you should expect that kind of candor from anyone showing you results.
What the test does show is the shape of the risk. Uploading your exports into a chat is fine for drafting a memo or summarizing a report. It is not a safe way to produce numbers you plan to stand behind, because the AI has to rebuild your department's definitions from scratch every time, and it will not always rebuild them the same way.
If you are going to put a number in front of a select board, a city manager, or your own officers, it needs two properties: it matches the records, and it comes back the same when someone asks again. That second property is the one most chiefs never think to test. It is also why we built FlorianAI to cite where every answer came from, a practice we cover in how to verify AI-generated answers in a fire department.
FAQ
Q: Can I upload my fire department's data into ChatGPT?
A: You can, but we have not tested ChatGPT. In our test with Claude, uploaded inspection files produced one wrong answer and two misleading ones out of four, and the same question returned three different numbers across four runs.
Q: Why did the uploaded files give different answers each time?
A: The AI had to guess what fields like "open" and "inspected" meant from column names on every run, and it sometimes guessed differently. It also did its own math without saying which figures were computed and which were estimated.
Q: What does connecting through FlorianAI change?
A: Definitions are set once in code and FlorianAI computes every number, so the AI only quotes and explains the result. In the test, that produced the same answer on every run.
Q: Does my department's data leave our system?
A: With FlorianAI, data is imported once a day, stays in the department's system, and only signed-in, authorized users can query it. Uploading files puts a copy of the data into each chat.
Bring Your Hardest Question
We will run it on your data. Send us the question that currently takes four exports and a morning, and we will show you your department's numbers answered the same way, every time. Schedule a demo or see how FlorianAI connects to the assistant you already use.
Want the results to share with your officers or city manager? Get the two-page field test summary (PDF).