The use of Artificial Intelligence (AI) has long been recognized as a potential path to cost savings in EM programs, but has not yet been formally implemented as a review standard within a US fishery. In New England, a project implemented by Teem Fish and Lynker Analytics has the potential to change that.
After several years of successful experimentation in electronic monitoring (EM), the New England groundfish fishery went to a full implementation as an operational program in fishing year 2021. This transition to full implementation and the subsequent costs are currently being managed by the federal government, but there is a distinct understanding that this reimbursement process is temporary and the cost burden of the EM program will ultimately land on the fishers and sectors participating in the programs. This cost burden has the potential to be extensive, and therefore Teem Fish Monitoring (Teem Fish) is committed to investing in areas where advanced technology shows the greatest potential cost savings that would directly benefit the local fishing community.
In working to bring cost saving advancements to the forefront of implementation in the Northeast, Teem Fish first looked to other projects they are involved with internationally that are providing similar benefits to other fishers. Specifically, Lynker Analytics had recently been engaged by Fisheries Inshore New Zealand Limited (now, Seafood New Zealand) to develop an integrated tool that combined the use of cameras, machine learning and broader AI to automatically measure legally released fish at sea. That project was designed to establish a long-term change to how inshore fisheries record sub-MLS fish to benefit scientific assessments and enable more informed management decisions, a current management imperative as part of the eastern tarakihi stock rebuild plan and results were promising.
Utilizing a variation of this established tech and work plan from New Zealand fisheries, Teem Fish and Lynker have begun a more formal transition to incorporating AI into the standard review features for New England’s Northeast Multispecies Groundfish EM Audit Program. This work, funded by the National Fish and Wildlife (NFWF) Electronic Monitoring and Reporting Grant Program, will specifically focus on quantification of discards of the 13 groundfish annual catch entitlement (ACE) species of concern for the Audit program, a task currently managed solely by manual review by certified video analysts for the program. Long-term, there is hope that the knowledge gained and groundwork laid from this project may lay a path towards formally incorporating this tech into EM review platforms across the fleet, helping to realize cost reductions thanks to accelerated discard quantification resulting in lower review costs for the EM fleet.
Moving from utilizing AI for detection and identification of an individual species in a focused fishery, to a similar application on 13 unique species within a multispecies fishery is not without its challenges. The good news is that due to the several years of participation in the pilot and Experiment Fishing Permit (EFP) stages of the Audit Model program, and the generosity of the fishermen that worked within the EFP to allow data sharing access, Teem Fish has years of relevant video data for this fishery and program to pull from. Based on the review analysis protocols of the Audit Model program, video analysts are required to account for all discards of the 13 ACE species by providing a positive identification to species, a count of individuals discarded per species, and a length for each discarded individual. This can be time consuming for video analysts, so this project is specifically focused on accelerating the analysis of these 3 data points. In order to most effectively break down the goals of this project into manageable pieces, Teem and Lynker are focusing on three main objectives.
Objective 1: Verifying the ability for AI to identify New England catch composition to count and species
There have been many studies and projects within fisheries AI to date that have shown that AI can be used to identify and track discards. For objective 1, we are focusing on ensuring that the specific 13 species of interest to this program can be accurately identified and counted at time of measurement. For this, Lynker is running their model over previously annotated video data provided by Teem Fish, and video analysts are verifying the AI output against the previously annotated data points.
Objective 2: Refine the accuracy of AI discard recognition and length frequency collection
In the Audit Model program, on-vessel fish handling requires vessel personnel to pass each discard over a provided EM measuring board for reviewers to obtain a length per fish to the nearest centimeter. In order to train the AI on length frequency, Lynker has isolated imagery with a fish on the board and the physical measuring board is used for calibration to each vessel. Video analysts are utilizing key points for initial data passing of each fish on a board, and those key points and identified species are then run through the AI model to begin training on AI generated length estimates per fish. Similar to the count and species verifications, video analysts will then verify the AI output against the previously annotated data points for continual data training and refinement.
Objective 3: Begin to develop an inclusive database of AI verified imagery for catch accounting within New England groundfish fleets for future integration into programmatic review softwares.
Here, we are specifically looking at the path to implementation. Based on past experience, Teem Fish was aware that use of an AI model within the data flow for an operational program will be dependent on two primary factors: regulators having enough confidence in the data coming from AI, and the ability to maintain support for the model to continually learn as more vessels join the program. To allow for this, objective 3 is focused on building out a final data processing pipeline to merge the results of each model into a data report (.csv file) which will be completed per discard sort event and which will provide the following metrics:
- Absolute count of discards
- Count per species
- Total length of all discards
- Image archive and frame reference library to be maintained within Engagement (Teem Fish review software program)
The database will contain at least enough data points per species to allow for an achievable confidence interval per species (goal of 95% alignment) and/or an understanding of the amount of additional data needed per species to achieve the necessary confidence interval. In order to continue progress towards implementation, Teem and Lynker also intend to hold a workshop in September 2024 with pertinent federal stakeholders to present findings-to-date and gain insight into any additional work needed to provide the necessary level of confidence among federal partners. These conversations are expected to shed light on whether there is in fact a future where partial or complete automatic review and submission of selected trips are permitted into the federal database. If permitted, this would lead to decreased review times and costs, and increased speed of data delivery to sectors, regulators, and end users.
This project is on-going and projected to run until November 2024 in this current phase. Based on the results of our stated objectives and the input received from NOAA at the September workshop, additional funding may be sought to maintain the momentum gained from this initial phase.
Jillian Di Maio is Fisheries Director at Teem Fish Monitoring, Inc. She welcomes questions and comments about this project and can be reached here. Projects in the Field is a series of independently produced articles profiling work supported by NFWF’s Electronic Monitoring & Reporting Grant Program, and is meant to raise awareness and support for these important initiatives.
