V.DIG.1902-Development of a single processor data feed
This project seeks to investigate and develop a consistent process that all MLA programs could leverage to receive and process carcase data.
| Project start date: | 17 December 2018 |
| Project end date: | 31 March 2020 |
| Publication date: | 19 January 2022 |
| Project status: | Completed |
| Livestock species: | Grain-fed Cattle, Grass-fed Cattle, Sheep, Goat, Lamb |
| Relevant regions: | Southern Australia, Northern Australia, National, NSW, Western Australia, Victoria, South Australia, Queensland, Northern Territory, Tasmania, Eastern Australia |
|
Download Report
(1 MB)
|
|
Summary
Meat & Livestock Australia (MLA) and Integrity Systems Company (ISC) receive carcase feedback data feeds from processors to support industry programs and producer tools. Processors deliver data to MLA systems through a variety of means. These are often non-standardised and may need manual intervention and duplication of effort. The different feeds may also result in inconsistency in data quality and interpretation. The purpose of this project is to investigate and develop a consistent process that all MLA programs could leverage to receive and process carcase data.
Objectives
1. Establish the data requirements and pipeline complexity for each existing MLA system receiving data from processors
2. Document the data currently being sent from processors and the sources of that data confirming, augmenting and superseding the documentation already held by MLA
3. Recommend a common data model amalgamating all MLA system requirements into a single schema for implementation
4. Identify gaps and opportunities for additional data collection and build into data model
5. Identify changes required to MLA systems to adopt the single data schema and work with stakeholders to develop and execute a transition plan – initially providing the data in the same formats but via data platform infrastructure
6. Provide a transition plan for processors and engage with their system providers to create an implementation plan.
Key findings
• Delivered a data schema for a single carcase data feed to enable processors and software vendors to upload carcase data into key MLA products in a single file. The project delivered the schema with the data requirements for NLIS, LDL, MSA and animal disease reporting. The schema has been delivered through:
o A data dictionary, developed in both Microsoft Word and as GitHub Wiki documentation;
o Messages in JSON Schema format, delivered into the Integrity Systems GitHub repository.
• This project interviewed a number of internal and external stakeholders to capture their feedback and requirements of a single carcase data schema. This included MLA and ISC programme teams providing documentation and materials to support the development of a draft data dictionary and schema. Processors and software providers were supportive of the move to a single schema and data transfer mechanism. However, this project identified a number of opportunities and challenges that needed to be taken into consideration before implementing a single carcase data schema. These are outlined below (please refer to the report for a more detailed explanation):
Opportunities:
• Potential to improve data quality and integrity;
• Opportunity to manage long-term maintenance and upgrade costs;
• Enabling increased speed of feedback for producers and efficiency and utility for processors; and
• Improving the ability to innovate and support adoption of initiatives.
Challenges:
• Necessity to standardise identity and permission models;
• Requirement to support a variety of processor implementations and levels of use;
• Issues with deployment to, and connectivity from, processor sites.
Benefits to industry
• Significant time savings in support and maintenance of data provision
• Simplified execution reducing implementation costs of new consumers
• Increased adoption due to standardised data models and documentation
• Increased trust from processing sector
• Significant cost savings for industry and MLA.
MLA action
A second phase of this work has been taken forward to refine and build on the single carcase feedback data dictionary and schema to ensure MSA Next Generation, Genetics R&D, animal disease and Lean Meat Yield (DEXA) requirements were captured, as well as any changes required from other MLA and ISC products.
Future research
This project provided four key recommendations that MLA and ISC should consider when planning a transition to a single carcase schema and standardised methods for ingesting data into MLA and ISC program systems. These were:
1. Create a roadmap for the process and effectively communicate this to relevant stakeholders
2. Ensure that the schema, replacement data transfer mechanism, and process are well documented, and that feedback from the participants is used to improve the documentation
3. Use an iterative approach for the transition and later evolution of the data model. Proof of concept and phased implementation allows feedback, learning, and confidence building before the new schema becomes mandatory
4. Actively support adoption by processors and their software providers and consider contribution to costs.
|
For more information: Contact Project Manager: Verity Suttor Email: reports@mla.com.au |
