Traceability in the southern rock lobster (SRL) export supply chain: investigating lobster grading and identification using low-cost image-based biometrics

This research examines traceability in the SRL export supply chain to investigate the use of low-cost image-based biometrics in lobster grading and identification. The SRL industry is a strategic export fishery contributing $250 million annually to the Australian economy with over 90% of its turnover coming from the Chinese market (prior to the COVID19 crisis). While the industry continues to grow, challenges have been identified that pose risks to the fishery's economic and environmental sustainability including: (1) Outbreaks of marine biotoxin and fish mortality/damage during transportation have raised food safety and quality concerns, (2) Product substitution via fraudulent business in export markets impacts the SRL premium brand, (3) The China Free Trade Agreement (ChAFTA) is changing requirements for food compliance regulations and, (4) consumer expectations for food quality, authenticity and provenance information on products is increasing. The SRL industry has identified product traceability as an important part of its response to these challenges. Traceability is broadly defined as the ability to track and trace food products at either item or batch level along part or all of the supply chain. Currently, some parts of the SRL supply chain do utilise product tracking such as barcodes and radio-frequency identification (RFID) tags in batch level traceability. However, item level traceability remains limited, with the granularity, form and availability of information on individual products, varying considerably along the chain. This is partly due to the high proportion of small SRL businesses continuing to rely on manual handling practices and paper-based traceability techniques. But it is also because of the continued relatively high cost of tools and techniques available to automate and digitise traceability data at the item level. Image-based biometrics deploying computer vision techniques have previously been trialled in individual animal tracking studies. These studies highlight the importance of understanding identifiable animal characteristics and the processes for image capture and analyses. In this research the key attributes of SRL captured and analysed relate to the lobster's size, weight, colour and spiny shell carapace patterns. Using these data, the feasibility of combining image processing and machine learning technologies to support automatic grading and individual identification of lobsters for processors, distributors and consumers in the SRL export supply chain were investigated. The research methodology involved a two-phase research strategy. Phase one identified key lobster characteristics and developed and tested image capture and analysis methods for both automatic grading and individual lobster identification. A key consideration for these methods is how to utilise low-cost equipment that would be readily accessible to SRL small businesses and consumers. Using the output of the automatic lobster grading, a physical prototype system was developed and tested to simulate an automatic grading operation in a fish processor. Based on the grading attributes and individual biometric characteristics selected and tested, a multimodal biometric identification model was developed. Building on these results, phase two focused on approaches to implementing traceability along the SRL supply chain through the design, testing and laboratory simulation of product tracking using a mobile application and verification server. The research design involved data collection at a Tasmanian fish processor where 8Mps Raspberry Pi cameras using a fixed distance and angle compiled a dataset of 4000 images of 200 lobsters. The lobster carapace was the \region of interest\" (ROI) during the development of automatic grading and individual biometric identification. This ROI was extracted from lobster images using: (1) traditional image processing techniques and (2) convolution neural networks (CNNs). The preliminary results highlighted a pre-trained Mask-RCNN outperformed the traditional image processing techniques and the Mask-RCNN was utilised for individually analysing the grading attributes of size weight (converted from size) and colour. These results were used to build and test a prototype grading system to simulate an automatic grading operation for use in a fish processor integrating software a conveyor belt relay and cameras. In the analyses of individual techniques for lobster identification colour histograms texture and Siamese networks were tested and evaluated. Based on the outputs of the automatic grading and individual identification a multimodal biometric identification model that combined lobster size colour histogram texture and deep learning results was developed and tested. However the false-positive rate of this model remained high (64.5%) and this meant that the results achieved were insufficiently reliable for individual lobster identification when used as a standalone solution. For this reason two hybrid biometric verification models for product tracking along the SRL supply chain were designed and tested. In both cases lobster image identification was used as part of a layered hybrid authentication approach along with product tags attached on individual lobsters and/or boxes of lobsters. These approaches aimed to create '1-to-1' and/or '1-to-some' matching mechanisms. In both scenarios a species-specific identification module was also developed and deployed to handle unmatched cases. This module was designed to enhance consumer confidence because it ensured that only genuine SRL products could be verified even if there were problems with individual verification based on tag identity number (ID) or lobster image. Using the main image library collected in Phase one a biometric verification server and mobile application were used to demonstrate the operation of these two tracking models. The analyses on lobster grading yielded positive results in which size accuracy is 90% with a range of +/- 5% error compared with the actual sizes and weight accuracy is 85% with a range of +/- 10% error compared with the actual weights. Colour classification also achieved a good result based on Euclidean distances between the average HSV colours of the ROIs and a base colour (black in this research) that helped identify a possible threshold to separate between the pale red and dark red groups. The testing with an automatic grading system also produced an important proof of concept where each lobster could be profiled with ID images size weight colour and other historical information. For lobster identification the testing results with individual techniques including colour histogram texture and Siamese network showed an ability to filter from one hundred objects down to a group of between 10-15 best candidates during the matching processes. The multimodal biometric model highlighted a further improvement of the matching process with the average of 5 candidates returned from the testing cases. However the challenge of the false-positive rate in lobster identification remained high with 64.5% within this model. In the simulations for the two hybrid models the false-positive rate showed a decrease to 0.35% for the '1-to-1' matching and 45% for '1-to-some' matching. For the species identification the use of Support Vector Machine (SVM) for classifying between SRL and non-SRL products based on colour histogram of the carapace regions also achieved a positive result with 92% accuracy. The simulation of the product traceability also demonstrated a range of important functions for end-users supported by the central verification server and mobile application including image capture lobster grading barcode scanning and individual product verification. The results obtained across two phases demonstrated how low-cost computer vision technology can be utilised to analyse grading attributes of lobsters (RQ1) and highlighted the current capability of this technology in identifying individual lobsters (RQ2). The research also demonstrated approaches to using low-cost biometric grading and identification outcomes to improve the traceability performance along the SRL supply chain (RQ3). Based on the image-based biometric models developed and tested in this research several contributions can be identified: ‚ÄövѬ¢ From the technical viewpoint the research strengthens the case for utilising lowcost computer vision in food traceability. Although the false-positive rate for the individual identification led to the deployment of hybrid traceability models the multimodal biometric model developed does provide a platform for ongoing improvements to the accuracy of low-cost image-based biometric identification. ‚ÄövѬ¢ From the traceability viewpoint the research offers an enhancement to conventional product tracking relying on physical tags. Importantly the low-cost image-based biometric verification solutions developed support small SRL businesses to improve item tracking level based on the existing box-tagging practices. Additionally the ability to integrate both the grading and identification functions into a mobile application offers direct traceability communication between overseas consumers and the Australian SRL exporters. ‚ÄövѬ¢ From the SRL industry viewpoint the research provides industry stakeholders with a response to the concerns of food safety quality and security. With the ability of verifying the authenticity and provenance of individual lobsters the confidence of end consumers in the safety and authenticity of SRL products can be consolidated and the Australian sellers can be better protected from fraudulent activities in export markets. The approach also illustrates how Australian SRL businesses may be better prepared for any compliance requirements imposed by ChAFTA. Future work will be able to further improve identification techniques and testing in live environments and it is anticipated that low-cost tag...