Blog Post
Data Normalization: Handling Swift MT940 and ISO 8583 Formats
The modern bank operates on a "Tower of Babel." The Core Banking System speaks SQL, the Correspondent Bank speaks SWIFT MT (text), the Payment Switch speaks ISO 8583 (bitmap), and the new Fintech partner speaks JSON. For a reconciliation engine to find matches across these systems, it must first translate everything into a common language. This process is called Data Normalization. Without it, you are comparing apples to binary code. This guide dives into the technical challenges of parsing two of the most difficult formats: MT940 and ISO 8583.
The Canonical Data Model
Before matching can begin, every transaction must be mapped to a standard internal structure—often called a "Canonical Model." This model typically contains fields like:
{ TransactionID, Amount, Currency, Date, Narrative, SourceSystem }.
The challenge lies in extracting these simple fields from complex legacy formats.
Challenge 1: The SWIFT MT940 (The Narrative Beast)
MT940 is the global standard for electronic bank statements. It dates back to the 1970s and was designed for telex machines, not databases.
- The "Tag 86" Problem: Crucial details like Invoice Numbers or Reference IDs are often buried in "Tag 86" (Information to Account Owner). This is a free-text field that spans multiple lines. A standard CSV parser cannot handle it.
- The Solution: Modern tools like Reconwizz use Regex-based parsers specifically tuned for SWIFT. They can "stitch" multi-line descriptions back together and extract specific keywords (e.g., "REF: 12345") into a dedicated column.
Challenge 2: ISO 8583 (The Bitmap Puzzle)
ISO 8583 is the language of card payments (VISA, Mastercard, ATMs). It is not human-readable text; it is a stream of data optimized for bandwidth.
How it works: The message starts with a "Bitmap"—a sequence of bits (0s and 1s). If the 4th bit is '1', it means "Field 4 (Amount) is present." If it's '0', Field 4 is skipped.
The Headache: You cannot just open this in Excel. You need a parser that reads the Bitmap first, then calculates the byte offset for every subsequent field. Furthermore, different switches (Postilion vs. Base24) implement slightly different "flavors" of ISO 8583. A robust reconciliation tool must support dynamic "Packager" configurations to decode these dialects.
Why Normalization Matters for API-First Banking
As banks move to API-First architectures, the need for real-time normalization increases. You cannot wait for a nightly batch job to parse an ISO 8583 message; it must happen in milliseconds to update the agent float or block a fraudulent card transaction.
Conclusion: The Foundation of Accuracy
You cannot reconcile what you cannot read. Investing in a platform with a powerful, pre-built library of parsers (for MT940, MT950, ISO 20022, ISO 8583, BAI2) is the only way to ensure data integrity. It frees your team from writing custom scripts and ensures that when the "Tower of Babel" speaks, your ledger understands every word.