In today’s climate of content exchanges and web APIs, it’s often necessary for someone with both business and technical knowledge to participate in data modeling activities or building data specifications. Dumping data into your organization’s data base often requires the application of sophisticated business rules.
Organizations invest a significant amount of time and money in establishing data feeds. While the technical implementation is typically relatively simple, the coordination between partners, mapping of data elements, and decisions around what functionality is required can create complexity.
The Data Feed Specification Template
If you are going to be setting up new feeds on an ongoing basis, a best practice is to develop a standard requirements specification template for new feeds and a package of support materials for partners. The package should minimally include a description of your preferred standard format for sending or receiving data, a sample, a list of fields and business rules for populating them, and a sample data file. If it’s possible to include developer tools, such as a validator or converter, that’s ideal.
The template should be specific to the type of content exchanged but should minimally include the following elements:
- Frequency with which the file will be delivered (and day/time)
- File format
- Filename
- File transfer instructions (FTP server address, API format, etc)
- Data mapping instructions
(By the way, while you can certainly recreate this yourself, I’ve included my Data Feed Specification template as part of the Business Analyst Template Toolkit. Why not save yourself a little time?)
This is the high-level view of what to include, now let’s drill into what you should be thinking about for the data mapping instructions. Then we’ll look at the types of questions you want to ask and answer to fill in the rest of the template.
Details About the Data Fields to Be Included in the Feed
The logistics of the file is one thing. The actual data to be included in the file is another. It’s in structure and content of the data file that most technical issues surface.
You’ll want to answer these types of question:
- What fields are required?
- What fields are optional?
- Do any fields have default values? Can these be specified by feed?
- For fields that must be matched to a specific set of values, will the partner be asked to provide the field IDs or a set of terms?
- What are the business rules for inserting a new record into your database? For example, if it’s an order, do you need to have a customer record set-up? Are there any limits to how many records a partner can post? Are the records loaded directly to a live system or do they go through a review process (manual or automated) first?
Many of these questions could be answered by reviewing or creating a data dictionary for your system and then thinking through how you’d want new records added to your system.
Don’t Overlook Data-Related Functional Requirements
In addition to creating a template and instructions to send to your partners and specifying field-specific rules for data feed, you’ll need make some business decisions as to accepting the feed and making it live in your system.
Consider the following types of questions:
- Will you ask your partners to push files to you or will you pull the files from your partner?
- On what schedule will the file exchange happen?
- Will each file have a full set of all active data, or will you need to trigger adds, updates, and deletes?
- What will happen to invalid files?
- Will you be able to isolate invalid records and load the valid records?
- Who will receive notification of invalid files and what can they do to rectify the errors?
- Will you check for duplicates? If so, what rules are used to flag a duplicate and what happens to a duplicate record?
Specifying data exchanges is an important part of getting the technical requirements right for the business. One wrong field or missing rule, and your business users will be in a world of hurt!
This is a lot to think about! You can learn more about the process of data mapping in this video:
>>Get the Data Feed Specification Template
You can grab my Data Feed Specification Template along with a corresponding work sample as part of the Business Analyst Template Toolkit. The Toolkit also includes templates for 11 other common business analyst documents.
Click here to learn more about the BA Template Toolkit
>>Learn More About Data Modeling (Free Training)
Learn the essential Data Modeling Techniques (even if you don’t know how to code) with this free training.
