write-hole

Flag: isfcr{p4r1ty_n3v3r_l13s_wr1t3_h0l3_r3m3mb3rs}

Summary

A 3-drive RAID5 NAS lost power while isfc-stage.py was rewriting a sealed report. Nobody saved the md settings, and the sealing key was burn-after-read. Recovery needed four steps:

1. Work out the RAID geometry from the raw images.
2. Recover the sealing key from the ext4 journal.
3. Rebuild the torn records from parity.
4. Repair one record that was still slightly wrong after the parity rebuild.

1. Recon

Three 17,825,792-byte images: bay0.img, bay1.img, bay2.img.

Entropy per 64 KiB showed this layout on every drive:

At 0x100400 on bay0 and bay1 there is an ext4 superblock (magic 53 EF at +0x38). It describes a 4 KiB-block filesystem with label ev-nas-3. So the array data starts 1 MiB into each member.

XORing the three drives together gave zero for every 4 KiB block up to 0x338000. From there to the end of the data region the XOR was non-zero. That is the write hole: stripes where data and parity disagree.

2. Recovering the RAID geometry

A test of the ext4 magic at filesystem offset 0x438 is useless here, because it only inspects stripe 0 and passes for every layout. I scored candidates with debugfs and e2fsck instead:

• Chunk size: 4 KiB to 128 KiB
• Algorithm: left/right, symmetric/asymmetric
• All 6 member orders

Only one family listed /stage with real files and passed e2fsck cleanly (the only complaint was the truncated device size):

• Chunk: 32 KiB
• Layout: left-symmetric (parity disk = (n-1-stripe) % n)
• Member order: bay1, bay2, bay0
• Data offset: 0x100000

la (left-asymmetric) listed the files but e2fsck reported inode bitmap and i_blocks errors, so ls is the right one.

Reassembled directory tree:

/stage   README.txt  isfc-stage.py  stage.log  job-0007.intent  payload.bin
/backup  nightly-0311.tar
/etc     nas.conf
/docs    intake-notes.txt  tail-note.txt      (filler)

3. Reading the design docs

• isfc-stage.py seals payload.bin as six independent 32 KiB records: nonce(12) | AES-GCM ciphertext(32740) | tag(16).
  • nonce = job(8) || idx(4)
  • AAD = payload.bin|<job>|<idx>
  • The 32 KiB record size equals the md chunk size, so one record is one chunk.
• job-0007.intent stores key_id and a SHA-256 for each sealed record. These act as checksum oracles.
• stage.log shows that all six job-7 records were submitted at once (qd=6) and the box died mid-write. It also says the job-7 keyslot was regenerated by the operator earlier that night.
• nas.conf shows data=journal and write-intent bitmap: none, so there is nothing to tell md which stripes were in flight.
• README.txt says keys are never kept on the array and the keyslot is burn-after-read.

4. Recovering the key from the journal

The key is deliberately absent from the filesystem, but the volume is data=journal, so the keyslot file's data block was journalled. Searching the reassembled array for the keyslot magic KSLT found four copies in the journal area:

The intent file's key_id is b5125b19a47d9d61, so the fourth slot is the key that was used:

f1d921fa87176e694601c89950e4bae1f38bae1e19b0e5ae5c4aa00b31fcfa3d

All four slots pass the keyslot CRC32, so the earlier job-7 slot is a plausible decoy rather than a corrupt one.

5. Finding the torn records

payload.bin is at fs blocks 1160-1207, which is array offset 0x488000. Checking each 32 KiB record against the intent hash gave:

The old-job nonces show these three records were never written to the data disk. Each sits in its own stripe (73, 74, 75), where the parity chunk had been updated before the power cut.

6. Rebuilding from parity

For each bad record, XOR the stripe's other two chunks (parity and the neighbouring data chunk) to recover what the missing data chunk should hold.

7. The last record

Record 3's rebuild had the right nonce (...07 00000003) and decrypted, via AES-CTR with the counter starting at 2, to fully printable text. So almost all of it was right. The intent hash still failed, and the custody shard read isfar{... instead of the expected isfcr{....

GHASH is linear over GF(2^128), so any ciphertext error shifts the tag by a predictable amount. I assumed the stored tag was the real one and computed the tag for the rebuilt ciphertext. XOR of the two gave a difference X in the GHASH accumulator. For a candidate error in ciphertext block i, multiply X by H^-(2048-i) to get that block's error pattern. Scanning all 2047 ciphertext blocks, exactly one block gave a sparse pattern:

• Block 554, byte 0, XOR delta 0x02
• That is ciphertext offset 8864, or record offset 12 + 8864
• 0x61 ('a') ^ 0x02 = 0x63 ('c'), which matches isfar becoming isfcr

After flipping that one bit:

• the SHA-256 of record 3 matches the intent file
• AES-GCM decryption authenticates for all six records

8. The report and the flag

The decrypted report is intake log lines plus three custody-token lines, spread over the parts:

part 2/6  shard 3/3 = h0l3_r3m3mb3rs}
part 4/6  shard 1/3 = isfcr{p4r1ty_n3
part 6/6  shard 2/3 = v3r_l13s_wr1t3_

The instructions say to concatenate shards 1, 2, 3 with no separators:

isfcr{p4r1ty_n3v3r_l13s_wr1t3_h0l3_r3m3mb3rs}