Enterprise Backup Architecture: Block-Level Engineering Deep Dive
ShadowCradle operates where most backup products don't dare go - every 64 KB block is individually hashed, deduplicated, optionally compressed, encrypted, and verified. A kernel-mode CBT driver tracks writes in real time. CoW layers let iSCSI snapshots mount without a full restore. The result: changed-block incrementals, hash-verifiable integrity, and bare-metal recovery over the network.
Every Component Stands Alone
The iSCSI server, backup engine, block store, and recovery layer are fully independent subsystems. Deploy what your environment needs - skip what it doesn't. No monolithic dependencies, no forced lock-in.
iSCSI Server
Expose any snapshot as an iSCSI target - on the same machine or on a remote host across the network. Runs without touching the backup pipeline. Mount on a different machine to recover a downed server without moving a single byte first.
Backup Engine
Smart, Compressed, Local Raw, or Appliance mode - each is a self-contained pipeline. Switch modes per-volume, per-schedule, or per-policy. The engine never assumes what the consumer will do with the data it produces.
Recovery Layer
Restore from local chain, appliance, S3, or a combination of all three simultaneously. Multi-source ingestion means the best available source serves each block - local cache first, then LAN appliance, then WAN.
Bitmap-indexed, 64 KB blocks. Changed-block incrementals.
Smart + LZ4 or Zstd. Smaller on disk.
Sector-level copy. No transformation, maximum auditability.
LZ4-compressed POST to appliance. Agent is stateless.
Combine Compressed + Appliance. A local copy keeps your most recent data on-machine for fast local recovery. The appliance receives the same data simultaneously - two independent copies, one backup run.
When the local cache lives on an SSD USB drive or internal NVMe, recent snapshots restore at local drive speed - no network needed for cached blocks. For blocks not in the local cache (older snapshots or evicted data), the appliance fills them over the network in parallel. The result: cached blocks are read locally, with only the missing delta fetched from the appliance.
The 7-Phase Backup Pipeline
Every backup - full or incremental - passes through a deterministic seven-phase pipeline with timing at each stage. No shortcuts, no silent failures.
SCAN
Enumerate all partitions, detect filesystem types, compute stable partition UIDs - GPT GUIDs for GPT tables, blake3 fingerprints for MBR disks. Reads partition tables and metadata only, not file data.
MAP
Build compact block change bitmaps - per-block blake3 hashes for every partition. Multi-threaded across CPU cores. Produces the foundation for incremental change detection.
DETECT
Select the most efficient available change-detection method: the CBT driver with a VSS copy-on-write diff on Windows, the block driver on Linux, or per-block hash comparison as the universal fallback. Unchanged blocks are skipped.
COPY
Read only changed blocks. Check the content-addressable store - identical blocks are referenced, not re-written. Only changed data is transferred.
VERIFY
Sample-verify written blocks by re-reading and comparing blake3 hashes. CRC32 validates every block change bitmap. Configurable sample rate - defaults to 5% for low overhead with strong assurance.
COMMIT
Commit the new snapshot atomically. Write delta manifests, updated block change bitmaps, and rich snapshot metadata. The appliance manages all snapshot sequencing and block relationships server-side - no client-side chain management required.
CLEANUP
Release VSS shadow copies and volume locks. Remove the atomic crash-recovery state file. The backup is now permanent, verifiable, and available for restore or iSCSI mount.
CBT Kernel Driver - Real-Time Block Tracking
ShadowCradle ships a kernel-mode CBT (Changed Block Tracking) driver for Windows. It operates as a volume filter that intercepts every write to disk at the IRP_MJ_WRITE level - the lowest possible OS hook - and marks the affected 64 KB blocks in a real-time bitmap stored in non-paged kernel pool.
When backup starts, the driver hands back only the dirty block list - no scanning, no hashing, no guessing. A 1 TB volume with 2% daily change only needs to read the changed blocks instead of scanning the whole volume.
| Method | How It Works | Speed | Time / 1 TB |
|---|---|---|---|
| Windows - CBT Driver + VSS | Change-block tracking driver feeds a VSS copy-on-write diff and NTFS $LogFile sequence numbers, then changed blocks are hashed | Preferred | Depends on change rate |
| Linux - scradle block driver | Block-layer driver tracks changed blocks on a live disk, with native Btrfs and ZFS snapshot paths for those filesystems | Beta | Depends on change rate |
| All - Block Hash Comparison | Universal: compare per-block blake3 hashes against prior snapshot bitmap | Fallback | Full read of the volume |
Fully Chain Free Based Backup Streaming
The agent streams each backup directly to the ShadowCradle appliance. We independently store each section of each snapshot and map it to deliver minimum space usage, maximum performance, and optimal reliability. No chains. No rebuilds. No waiting.
The agent streams each backup directly to the ShadowCradle appliance. No local storage, no local block store - the appliance independently stores each section of each snapshot with full deduplication, snapshot management, and metadata. Hash blocks, send new ones, restore done.
- Zero local storage required on the protected machine
- Appliance serves snapshots and blocks directly
- Appliance deduplicates across all clients - shared block store
- Interrupted backups resume automatically from last block sent
- Ideal for desktops, laptops, and servers without large local storage
Beyond Backup Storage - The Appliance Computes
Cloud-native backup stops at storing data. The ShadowCradle appliance is a full compute node that actively verifies, scans, and estimates your backups. It boots Windows VMs from every snapshot to confirm they'll actually start when you need them. It runs antivirus on mounted volumes before any restore touches your network. It measures restore throughput and computes an estimated RTO from it.
On-Appliance Antivirus Scanning
Every backup can be mounted read-only and scanned by ClamAV or integrated AV engines before any restore touches your production network. Stops re-infection at the gate - even if ransomware bypassed the sentinel check at backup time.
Windows Boot VM Verification
The appliance spins up each Windows snapshot as a virtual machine guest over the CoW iSCSI layer. If it reaches the login screen, the backup is marked "boot-verified." Runs automatically after every full backup - no manual testing required.
RTO Estimation
The appliance measures actual local restore throughput (block size × link speed × dedup ratio) and computes an estimated RTO for each snapshot. MSPs can export RTO reports per client.
Local Restore Speed
Restores run over your LAN, not the internet. A local appliance can restore at the speed of its disks and your network link.
Full Air-Gap Operation
The appliance needs zero internet connectivity to protect, restore, or verify machines. Ransomware often targets cloud credentials first - an air-gapped appliance on a dedicated VLAN has no credentials to steal and no cloud surface to attack.
Cross-Client Block Deduplication
The appliance deduplicates across every protected machine in a single shared block store. A Windows Server OS block stored from machine A is never re-uploaded from machine B. Cloud-native solutions typically deduplicate per-tenant - not per-block across the fleet.
| Capability | ShadowCradle Appliance | Cloud-Only |
|---|---|---|
| Antivirus scan before restore | ✓ On-device ClamAV / AV engine scan | - Not available |
| Boot verification (Windows VM) | ✓ Auto-tested after every full backup | - Requires separate hypervisor infra |
| RTO estimation | ✓ Calculated + SLA-reportable per client | ~ Estimated from cloud benchmarks |
| Restore bandwidth | ✓ LAN speed | ~ Limited by internet link |
| Air-gap / offline operation | ✓ Zero internet required | ✗ Requires continuous connectivity |
| Cross-client deduplication | ✓ Shared block store across all clients | ~ Per-tenant only (no cross-org dedup) |
| iSCSI expose + PXE BMR | ✓ Built-in CoW iSCSI target on agent | - Requires separate BMR workflow |
| Pre-restore ransomware isolation | ✓ Scan on air-gapped VLAN before restore | - Network path exposes risk |
CoW Layer - iSCSI Snapshots Mounted Lazily
When ShadowCradle exposes a backup snapshot as an iSCSI target (for live-mount or BMR), a Copy-on-Write (CoW) layer sits between the consumer and the read-only backup blocks. Writes go to the CoW file; reads hit the CoW first, falling back to the backup store.
Traditional pre-allocation (ftruncate 1 TB) blocks mount for 30+ seconds. ShadowCradle's lazy CoW uses a compact bitmap (1 bit per 512-byte sector - just 128 KB for a 1 TB volume) so the mount does not wait on a full-size allocation.
Compact Block Tracking - Tiny Overhead, Maximum Speed
Every partition is tracked with a compact bit array - one bit per 64 KB backup block. A 1 TB volume's entire change map fits in about 2 MB of memory. Checking whether any single block has changed is an O(1) bit read regardless of volume size.
When comparing the current state to the prior backup, a single bitwise XOR across the two bitmaps produces the changed block list - no full-volume scanning, no re-hashing everything. Just bit operations over a compact data structure.
Compress as You Go - Zero Extra Passes
Every block is compressed inline during the COPY phase - immediately after reading and before writing to the block store. No second pass, no temporary uncompressed files on disk. The block is transformed once, in a single streaming pipeline:
On a 1 TB volume with 1% daily change, only ~160,000 of 16,000,000 blocks have bit=1 - the other 99%+ are skipped entirely.
The 256×256 directory tree gives 65,536 top-level buckets - no single directory grows unwieldy as the store grows. Any block is located with a constant-time O(1) path lookup using its hash as the address - no index table, no scanning.
iSCSI Expose + PXE Boot BMR
ShadowCradle doesn't just protect machines - it can become a boot target. From the Windows agent, any backup snapshot can be exposed as a local iSCSI target. A replacement machine boots from it over PXE - no USB, no imaging, no downtime waiting for a restore to complete.
The replacement machine can run the original OS from the CoW mount. Full local restore happens in the background while the user works.
Pure network-based BMR. No imaging USB drives, no shipping media. Any PXE-capable machine can become the recovery target.
The CoW layer absorbs all writes. The backup snapshot remains byte-for-byte intact - multiple simultaneous recoveries can boot from the same snapshot.
Expose any snapshot - from a local chain, S3, or the ShadowCradle appliance - as an iSCSI target. BMR source is wherever the backup lives.
Machine B can run the failed server's OS from the CoW iSCSI mount while the full restore happens in the background.
Local cache serves the most recent hot data first. The appliance fills remaining blocks over LAN. If both are available, they run in parallel - every block is fetched from the fastest reachable source.
Machine B doesn't need to be on the same LAN. If the appliance is reachable over WAN, recovery works remotely - a data center machine can recover a branch office server without physical presence.
The CoW layer absorbs all writes. The backup snapshot on the appliance stays byte-for-byte intact - multiple machines can recover from the same snapshot simultaneously.
Content-Addressable Block Store
Every block is identified by its blake3 cryptographic hash. Identical blocks across snapshots, machines, and time are stored exactly once. Incremental backups store only delta manifests - tiny reference files pointing at blocks already in the store. A 0.1% incremental produces a ~288 KB manifest instead of re-writing gigabytes.
Direct-to-Cloud Streaming for SAN & VM Volumes
Back up volumes exposed via iSCSI protocol - SAN LUNs, VM disk images, remote NAS - without requiring a full mount. Blocks stream from the iSCSI target through a local SSD/NVMe cache, hashed, compressed, and uploaded to S3 or the appliance in parallel.
Six Output Modes - One Engine
The same backup pipeline drives four standalone modes. Add the optional Local Cache hybrid for local recovery without network dependency.
- Blocks stored once by blake3 hash - zero redundancy
- Delta manifests reference blocks, never duplicate them
- Point-in-time recovery: Full + ordered deltas
- Storage savings depend on your data and change rate
- LZ4 (speed-oriented) or Zstandard (ratio-oriented)
- Configurable compression level 1–9
- Combined dedup + compression reduces storage
- Codec stored per-block - mixed archives fully supported
- Snapshots stored independently in S3-compatible storage
- Local SSD cache (1–500 GB) for hot-path restore speed
- Parallel upload streams + exponential-backoff retry
- Deduplication + compression on every upload - pay only for unique data
- Agent streams each backup directly - appliance manages snapshots
- Appliance owns the block store, snapshots, and all metadata
- Resume-capable: only missing blocks re-uploaded on interruption
- No chain assembly required on the client at restore
- Unprocessed sector copy: no compression, no deduplication, no chain
- Full-fidelity output for forensic imaging, compliance, or audits
- NTFS sparse file optimization - only sectors with data written
- Maximum simplicity and auditability - what you see is what you get
Defense in Depth - At Every Layer
AES-256-GCM Per-Block Encryption
Every 64 KB block encrypted individually. 12-byte nonce + 16-byte auth tag per block. Block offset as AAD prevents block-swap attacks. Just 28 bytes overhead per block (0.04%).
TLS 1.3 in Transit
All appliance and cloud communications use TLS 1.3. Credentials and keys never travel unencrypted.
Blake3 Integrity Hashing
Every block hashed with blake3 - cryptographically secure, and natively parallelizable. Any hash mismatch triggers automatic rejection.
Ransomware Sentinel Detection
15 sentinel files placed in 3 strategic OS locations. Pre-backup integrity check detects ransomware modification across 5 file types before any backup proceeds.
Scrypt Key Derivation
Password-derived encryption keys via Scrypt (N=2¹⁴). Secrets never stored in plaintext. Config files enforce OS-level read permissions.
Atomic Write + Crash Safety
All checkpoint files written atomically via temp-file + os.replace(). Power loss mid-backup = clean resume from last checkpoint - never a corrupt or partial snapshot.
Overhead: just 28 bytes per 64 KB block (0.04%). Keys are derived via Scrypt (N=2¹⁴) and never stored in plaintext. Block offset as AAD prevents block-swap attacks without any extra computation.
Stop Infected Backups Before They Happen
ShadowCradle places 15 sentinel files across three strategic OS locations. Before every backup, all 15 are re-hashed and compared to their baseline. Ransomware that modifies any sentinel file triggers an immediate halt.
Performance That Scales
One Agent - Every Major OS
- CBT kernel driver (IRP_MJ_WRITE intercept, real-time bitmap)
- VSS snapshot creation + NTFS $LogFile change tracking
- Sparse file optimization (NTFS hole-punching)
- iSCSI target exposure - expose any backup for PXE/BMR
- Physical drive + partition backup (GPT and MBR)
- EXT4 / btrfs filesystem support
- inotify change event monitoring
- CoW snapshot comparison for iSCSI volumes
- Direct-to-cloud streaming (S3-compatible)
- iSCSI target backup from SAN and NAS volumes
- APFS filesystem support
- FSEvents change tracking
- Full partition and drive imaging
- S3 and appliance streaming modes
Per-Backup Metrics Out of the Box
Every backup emits rich per-job telemetry - blocks written, compression ratios, deduplication savings, throughput, and duration. Metrics are shown in the agent and on the appliance.
Why This Engineering Translates to Real Business Value
Smaller Storage Bills
Content-addressable dedup + Zstd compression reduce storage compared with full-image backups. You pay for unique data, not duplication.
Smaller Backup Windows
Change tracking reads only changed blocks, so incremental backups run while production workloads keep running.
Verifiable Integrity
Every block carries a blake3 hash. You can independently re-hash any backup and confirm against stored metadata. No vendor trust required.
Crash-Safe by Design
Atomic checkpoints mean power failure mid-backup = clean resume - not a corrupt snapshot that silently fails on restore day.
Pre-Backup Ransomware Check
15 sentinel files catch encryption activity before any backup commits. You never inadvertently seal an encrypted snapshot into your retention chain.
Remote BMR Without Physical Media
iSCSI expose + PXE boot turns any replacement machine into a network recovery target. Replacement machine boots from the snapshot while restore to disk runs in the background.