Introduction
Alaska is home to over 6000 resident fishing vessels employing over 19,000 resident fishermen harvesting almost $2 billion USD in seafood every year. More than 10% of these earnings come from the Alaska Fixed Gear Halibut and Sablefish program. This program has become a model for the implementation of fishery management to increase profitability and sustainability in commercial fisheries. In 1995, the program was among the first catch share programs with explicit profitability and sustainability goals which have now been fully met.
In 2014, the program installed electronic monitoring (EM) systems on 140 vessels. EM represents a promising opportunity for fishery management of the future. EM systems, consisting of various sensors and cameras, can capture fishing and catch activity while at sea. Video footage is then reviewed on shore following the return of the vessel. Results can be used in concert with other monitoring tools such as dockside weights and electronic logbooks to create a complete picture of fishing activity for a given trip.
This approach allowed the Alaska Fixed Gear fishery to continue to operate many vessels too small to accommodate At-Sea-Observers while improving the oversight of catch activities in the vulnerable halibut and sablefish populations.
Despite its promise, EM has been adopted in less than 1% of the world’s fishing fleet. This is in part due to the massive amount of data generated by EM systems. For example, one 10-day trip per year for each vessel with EM in the Alaska program would generate over 30,000 hours of video footage for review. This problem leads to sampling of video for review which can reduce the ability of EM to improve fishery oversight. For example, shore review might only review 5% of hauls for a given trip, even though footage of 100% of hauls is available.
Artificial intelligence (AI), and in particular computer vision, represents an opportunity to solve the problems of scale in EM. Computer vision is an AI field that enables computers to derive information from still images, video and other inputs.
Ai.Fish is a Hawaii-based startup solely focused on the application of computer vision to ocean conservation. With funding from the National Fish and Wildlife Foundation, Ai.Fish partnered with the Alaska Fixed Gear Program, Archipelago Marine Research Ltd who supplies the EM equipment, and the Pacific States Marine Fisheries Commission to conduct a project assessing the impact of applying AI to improve the cost effectiveness of the EM program.
The Project
We set out to develop a set of customized computer vision algorithms that analyze video data to detect fish, humans and gear. To create these algorithms, the first step is to collect a large amount of representative data that can be used to train the algorithms so that the computer knows what it is looking for when analyzing video data. Training is accomplished by manually annotating video data with appropriate labels. With footage from 8 vessels our annotation team was able to create a large dataset of 562,282 annotations.
Video analysis first starts with detecting humans and fish. We detect humans as the presence of humans on deck typically indicates fishing activity is occurring. After detection algorithms run, a second set of algorithms runs that create tracks. During fishing activity, the same fish can typically be seen in multiple frames of video as the fish is handled and sometimes processed within view of the EM cameras. To obtain an accurate count, the computer must be smart enough to recognize that the fish is the same fish from frame to frame. This is accomplished through a complex approach of prediction possibilities that are slowly reduced to a reasonable level of confidence as the computer moves through the video frames. As confidence in tracks is reduced, they are eliminated from the counts until the algorithm arrives at an accurate count.
Trained algorithms were then made available to EM reviewers within an existing software product used for video review through an Application Programming Interface (API). The API was integrated with FishVue Interpret, a review product developed by Archipelago Marine Research that is used for EM review in the Alaska Fixed Gear Program, allowing for AI results to be displayed within existing software workflows.
The AI-assisted review involved trip video being uploaded to the Ai.Fish API, subsequently analyzed by our algorithms in the cloud, and then results were returned via the API to the review software product. An EM reviewer could then review each relevant haul by simply verifying the AI results against the video. The project’s key aim was to improve the efficiency of the EM review data analysis. Our hope was that improved efficiency would translate into reduced costs.
The Results
A/B testing was carried out on 6 trips of footage. Trained reviewers supplied by Archipelago Marine Research Ltd, reviewed each trip once using the standard approach and once using the AI-assisted approach. This gave us a total of 24 reviews. Reviewers recorded their typical review output including fish caught and discarded. They also kept track of the time required for review.
On average, AI-assisted review resulted in a 48% time savings. When factoring in the cost of cloud-based analysis, this led to a 46% cost savings.
For 5 of the 6 trips reviewed, AI-assisted review introduced a mean error rate of 2.7%, compared with a 1.3% difference in results between reviewers conducting standard reviews.
In the 6th trip, results were significantly poorer, however, after discussion with the reviewers it was determined that this was likely due to a poor camera view on the hauler camera which caused occlusion of fish most of the time they are in the camera view. Occlusion means only part of a fish may be visible and this can create challenges with algorithm performance. This could be fixed by altering the camera location for future trips.
Discussion
Our findings that AI-assisted review was comparably accurate to human review are important. Access to human reviewers remains a significant barrier to EM adoption globally. Many fishing communities are remote, and have limited access for travel. Attracting reviewers to live in these remote locations presents a significant challenge. Travel costs add to the already significant cost burden of these programs.
There are opportunities to further improve our cost efficiency as we optimize our algorithm performance in the cloud toward our goal of 100x speed.
Current Alaska EM program review costs are between $198 and $332 USD per day. This puts the review cost of a 10-day trip at $1,980 to $3,320 USD. AI-assisted review can reduce these costs potentially as low as $1,069 to $1,793. For the 140 vessels in the fleet this potentially represents program cost savings of $127,540 to $213,780 assuming only one trip is taken per year. We know that 75% of the Alaska fixed gear fleet fishes 1 – 3 trips per year.
A faster turnaround of results from AI-assisted EM review due to our time savings, enhances the benefit of the program with respect to fishery management as it can provide information supporting quota management closer to real-time. Better quota adherence in any given year improves the sustainability of the fishery and maximizes fisher revenues.
Overall our results are highly promising. They prove that AI can be used effectively to substitute a human reviewer with substantial time and cost savings to EM programs. This project demonstrates that AI-assisted EM review improves the scalability of EM which is a gateway to improving adoption rates globally and domestically.
Our API is commercially ready for use by any EM vendor. Our cloud-based analysis includes sophisticated algorithms for tuna longline fishing and the algorithms developed for Alaska fixed gear. We continue to work on enhancements to our algorithm performance and processing time. For fisheries not yet using EM systems, Ai.Fish offers a web-based review software product that can be used to perform AI-assisted review. Our team continues to tackle new opportunities in ocean conservation and commercial fishing that will benefit from computer vision assistance. The future of fisheries is here.
For more information about our work visit www.ai.fish or reach out to us at contact@ai.fish.
