The newest messages are missing from my export. Why?

An export that stops before the conversation does, with no error and every message in it genuine, has one cause worth checking once the backup’s date is ruled out: the message database’s write-ahead log. A maker of forensic software reports that from iOS 17.4 a backup can carry that log as a separate file, and says itself that nobody knows whether Apple will keep it that way. When the log is there, a reader that does not merge it loses whatever was still waiting in it.

Why would the newest messages be missing and nothing else?

Because the newest messages are the ones most likely to still be in the log. The message database, sms.db, is a SQLite database, and in write-ahead log mode SQLite appends new writes to a companion file and copies them into the database itself later, at a checkpoint. Until then, the most recent messages exist only in that companion file. A backup that carries both looks like this:

Library/SMS/sms.db
Library/SMS/sms.db-wal
Library/SMS/sms.db-shm

The third file, sms.db-shm, is the shared-memory file in which SQLite keeps an index that helps readers locate pages in the log quickly.

Then the backup’s naming rule separates them. A file in a backup is stored under the SHA-1 hash of its domain and path, so the database and its log arrive as two unrelated forty character names:

sms.db      3d0d7e5fb2ce288813306e4d4636395e047a3d28
sms.db-wal  cd47480f213dba9bc38ee792775d17e3f5a73a59

SQLite finds a log by its name, which is usually the database’s own name with -wal added. In a backup the two never carry matching names. A tool that locates the message database, opens it where it lies and reads it gets a database with no log to apply, so every transaction still in that log is invisible. Nothing is corrupted and nothing is reported.

Which iOS versions does this affect?

Backups made from iOS 17.4 on, going by the one outside comparison found for this page, and that comparison covers only iOS 17.3.1 and 17.4. MSAB compared backups of one phone made before and after it was updated from iOS 17.3.1 to 17.4, and wrote that on 17.4 “we now have access to” the .db-shm and .db-wal files that belong to the databases, sms.db among them. That the 17.3.1 backup lacked them is our reading of that comparison, not a sentence of theirs. MSAB describes the change as Apple appearing to have stopped compacting the databases it puts into a backup; the step this page is about is the one SQLite calls a checkpoint, folding the log into the database. Before that, Elcomsoft had written that the moment a backup starts, the logs are merged into their databases, which fits backups from before 17.4 receiving the messages already folded in. Both articles are written for forensic work on deleted data, which this page does not cover: it is about messages nobody deleted.

Neither statement comes from Apple, and MSAB says plainly that nobody knows whether the change will last. This article cannot tell you how every later version behaves. The file check further down answers the question for your own backup, whatever version made it.

How much of the conversation can this hide?

Whatever had not reached the database yet. SQLite’s default is to fold the log in once it grows to 1000 pages, and an application can change that threshold, so the amount is a matter of timing rather than a fixed figure. The finished export gives no sign that it is short.

Why does the obvious fix make things worse?

The obvious fix is to give sms.db and sms.db-wal back their paired names inside the backup folder and open the database there. It brings the missing messages back, and it changes the backup it is reading.

SQLite writes too. A write-ahead database opened read only can be read when its companion files already exist, when they can be created in its folder, or when it is opened as immutable, so a reader that does not use that last option may create files inside the backup. And when the last connection to a database closes, it runs one last checkpoint and deletes the log. For a reader that opened the database with write access, that means sms.db inside the backup is rewritten and the log it came with is gone. The backup is then no longer, byte for byte, what the phone made.

What does a careful tool do instead?

It never opens sms.db inside the backup folder. The sequence that avoids both writes is: find both files by their hashes, copy them into a scratch folder of its own under real, paired names, fold the log into the copy, and read the copy. Folding the log is a write, so it happens only on the copy and the original is never written to.

That is what this program does: the backup is opened read only, the work happens on a working copy in a temporary folder, and the log is merged on the copy and never in the backup. The same merge runs on encrypted backups, and if it fails for any reason the program falls back to the unmerged database rather than giving up.

How do you check whether your own export is short?

1. Check the backup’s own date first

A backup made on Tuesday cannot contain Wednesday’s messages, so if the backup is older than the messages you are missing, the backup is the answer and no reader can do better. Where Windows stores iPhone backups covers the folders and how to read the dates on them.

2. Find the last message in the export

Not the last page, the last message with a date on it.

3. Compare it against the phone

Open the conversation on the phone and look at the same stretch. If the phone has messages after the date the export stops, note the date of the newest one the export lacks and set it against the backup’s date from step 1.

4. Look for the log in the backup

Search the backup folder for a file named cd47480f213dba9bc38ee792775d17e3f5a73a59. Since iOS 10 a backup keeps each file in a subfolder named after the first two characters of its name, so this one sits in a folder called cd. If there is no such file, the log is not why your export is short. If it is there with a size of 0 bytes, it holds nothing to merge. Any other size means it may hold messages the database does not, though not that it must, because a checkpoint does not normally shrink the log; only a reader that merges it will show them.

If the backup is newer than the missing messages and the log is in it, the cause is on the reading side, and re-exporting with the same tool will produce the same short document. For the wider question, the complete guide to exporting text messages from an iPhone to a computer covers the routes and what each one carries.