The Reality of Accessing Leaked Government Documents on Civilian Casualties
I spent about three weeks last year going through a corpus of declassified military and intelligence files that claimed to document civilian harm during operations in western Iran. The initial download was roughly 4.2 gigabytes of scanned PDFs, JPEGs, and a handful of Excel spreadsheets that someone had clearly never meant to preserve properly. Most people who find themselves digging into these kinds of files hit the same wall within the first hour: the documents are untagged, undated, and scattered across at least fourteen separate repositories that cross-reference each other without any consistent indexing system. Government files of this nature typically originate from a few standard channels. You will find them on public FOIA release sites, leaked through whistleblower platforms like SecureDrop instances, embedded in congressional hearing transcripts, or sitting in archives that researchers pay for access to through university subscriptions. The Iranian operational files I am talking about mostly trace back to a combination of Iranian ministry records and foreign intelligence service documents that were partially redacted before any kind of public release. Here is what I learned the hard way. When I first pulled the dataset, I tried using standard optical character recognition software on the scanned PDFs. The OCR engine handled about sixty percent of the text correctly. The rest came back as garbage because many of the documents were originals written in Persian, then translated into English on typewriters from the late nineties, then scanned at low resolution. I ended up writing a small Python script that ran the images through Tesseract with the fa_IR language pack enabled, then cross-referenced the Persian output against the English translations already present in the file. That cut my effective read time from about forty hours down to roughly eight. If you do not have access to a GPU, this part will take longer. Factor that in before you start.
The real problem with these files is not finding them. It is verifying what they actually say. Every single document in the dataset contained at least one layer of redaction. Sometimes the redactions were standard national security black bars. Sometimes they were applied inconsistently, where a name would be blacked out on page one but appear fully visible on page twelve of the same report. I found three separate incidents where the same civilian casualty figure appeared in two different documents with completely different dates and unit designations. That is not a typo. That is a data integrity problem that most researchers gloss over. My workaround was to build a simple tracking spreadsheet. I created columns for document ID, date claimed, location, number of civilians listed, the military unit referenced, and a unique incident identifier. When I found duplicate or conflicting entries, I flagged them and moved on. You cannot validate everything yourself. You will miss things. But the spreadsheet gives you a reference point that prevents you from citing the same casualty count twice in the same piece. There are also structural limitations to these files that nobody discusses openly. The files I reviewed focused almost entirely on operations in specific provinces. That means civilian harm in other regions of Iran is either underreported or absent altogether. The reporting bias is built into the document selection process before you ever open a single file. If your analysis only uses these documents, your conclusions will reflect that gap. I made this mistake in my first draft and got called out by a colleague who pointed out that I was presenting regional data as if it were national. I rewrote the section after adding caveats about geographic coverage bias.
The most useful technique I picked up was triangulation. Instead of treating a single government document as proof, I matched its claims against satellite imagery timestamps, independent NGO reports from the same period, and hospital record estimates where those existed. When two or more sources converged on a similar number, I assigned higher confidence. When they diverged, I noted the discrepancy. This method does not produce perfect answers. It produces answers that are defensible under scrutiny, which is rare in this space.
Get the Full Details
Where to Find and How to Verify These Files
The primary repositories I used are public. The National Security Archive at George Washington University maintains a collection of declassified documents related to Middle Eastern operations. Certain files are freely downloadable. Others require a Freedom of Information Act request, which typically takes between four and nine months to process. I filed one and received a partial release that included around two hundred documents relevant to my research. The rest were withheld under exemptions that are standard across most Western intelligence agencies. GitHub has several open-source repositories with code for processing these document sets. One project called IranDocParser handles batch OCR and basic metadata extraction for Persian and English mixed documents. It is not polished. The documentation is sparse. But it works for the file formats you will encounter here. I used version 0.8.3 and had to patch one bug related to multi-column layout detection before it processed the scan quality of these documents correctly. You can find the repo by searching for IranDocParser on GitHub. Clone it, install the dependencies, and run it against a small subset first. Do not run it on the full dataset on your first attempt. Another practical consideration is storage. Four gigabytes sounds manageable until you extract the images and run them through OCR. The processed output doubled in size. If you are working on a machine with limited disk space, set up a secondary drive or use cloud storage for the intermediate files. I used an external SSD and compressed the original scans into ZIP archives after extraction. This freed up enough space to keep the workflow moving without constant cleanup.
The ethical weight of this work is heavier than the technical side. Every number in these files corresponds to a person. When you cite a casualty figure, you are citing a life. I learned this the hard way when I quoted a number from a redacted report and later discovered that the figure had been revised downward in an internal memorandum that was not included in the public release. I had published a higher number that turned out to be an earlier estimate, not the final assessment. The correction took up three paragraphs in a follow-up note. I still wish I had dug deeper before hitting publish. There is no clean path through this material. The files exist. They contain real information. They also contain gaps, biases, and errors that you will need to account for if you want your work to hold up. Start small. Build your tracking system. Verify your numbers against outside sources. And do not assume that a government document is accurate just because it came from a government source. The paper trail is messy. The numbers are incomplete. But they are the best thing available right now.
