How to teach a neural network to recognize jellyfish and fish at sea depth
Russian scientists have developed a vision system for observing the underwater world, which automatically detects natural and artificial objects in the depths of the sea. The neural network recognizes fish, animals, algae and even translucent jellyfish. The use of technology will allow obtaining a large amount of objective information about the processes taking place in the world ocean. Now there is not enough such data, since classical expeditions are difficult and expensive. According to experts, the use of optical observation methods under water is always associated with difficulties due to poor visibility. However, artificial intelligence is also able to see everything that a person can see, while AI can analyze a much larger amount of information.
A look into the abyss
Specialists of the Southern Federal University (SFU) have developed a system for monitoring underwater space based on a neural network. It automatically analyzes data from video cameras installed at sea depth and identifies fish, animals, plants, various natural and artificial objects. The development can also be used to monitor the condition of underwater pipelines. The accuracy of recognition of the invention is 90%. The authors are already conducting test tests of their complex at the university in Kagalnik on the shore of the Sea of Azov. For this purpose, video is used from a stationary underwater camera and another one installed on an unmanned bathyscaphe.
Photo: RIA Novosti/Anton Denisov
Image source: iz.ru
According to the university, today ocean researchers lack accurate data on the state of the sea depths. In order to get them, scientists are forced to either dive there themselves, or use expensive underwater vehicles for this. In addition, to calculate the volume of fish in the ocean, some researchers conduct a test fishing, on the basis of which they calculate the total number. Such work requires the equipment of an entire scientific expedition. At the same time, estimates based on these methods are always approximate and have a large margin of error.
Image source: Photo: SFU
— We are studying the underwater world. We are interested in artificial structures and biological objects. We need a surveillance system to analyze them. We suggest using a network with stationary buoys and mobile underwater vehicles and installing video cameras on them that would monitor what is happening at depth in real time. The image obtained in this way will be analyzed by a neural network," said Denis Krivoguz, a leading researcher at the Department of Oceanology of the Southern Federal University.
The use of such a system will give ocean researchers a large amount of objective information and help answer important scientific questions, the developers explained. At the first stage, scientists want to study the ecosystem of the Sea of Azov.
Find out the jellyfish
The development can detect both static and moving objects. For its operation, you can use ordinary portable cameras or even a smartphone camera hidden in a protective box. When shooting from a bathyscaphe, it is important to maintain the clarity of the picture, stabilization is used for this.
Image source: Photo: SFU
The neural network splits the image into frames, analyzes it and turns it back into a video. Processing of optical data obtained underwater is particularly difficult due to high turbidity and poor lighting. Scientists artificially degrade the quality of the video on which the program is taught with various interferences so that the AI is ready to work in difficult conditions. However, the system still needs at least minimal transparency for high-quality results.
To train the neural network, scientists manually mark up an image of the underwater world, which indicates the types of marine animals and plants that have come into view of the camera, different properties of the bottom and other data. The work of scientists is designed for a long period, since the Sea of Azov has a very diverse flora and fauna.
— For example, we have trained AI to recognize adult pilengas and their fry, needle fish, gobies and jellyfish. The most difficult thing is to recognize a jellyfish, since it is translucent under water and looks unusual. We are constantly adding new objects and retraining the neural network," said Denis Krivoguz.
Image source: Photo: SFU
Developers are experimenting with various states of the marine environment, along with changes in which the appearance of objects also changes.
— Any object in the aquatic environment with different amounts of silt and turbidity looks different. For example, its color changes. And the neural network is guided by these characteristics: color, shape, shine. Therefore, we are constantly increasing the variability, due to images with different water quality," the specialist added.
Spot the dolphin
Monitoring systems already exist in the ocean. For example, Americans are watching a volcano in the Pacific Ocean. This is a set of sensors that responds to earthquakes and eruptions. However, optical observation methods are not used in it, Alexander Mazarovich, chief researcher at the Geological Institute of the Russian Academy of Sciences, told Izvestia.
— The complexity of observations under water is explained by the almost complete absence of light. Very powerful lighting is needed to work there. However, even when using a spotlight, visibility is limited to 10-15 m. It is unclear how effective surveillance using video cameras will be in such conditions. Perhaps this is enough to control fish resources, but the survey area is too small for geological studies," Alexander Mazarovich said.
Visual observations are carried out from underwater vehicles. They examine the bedrock deposits at the bottom, move along them and look through them. At the same time, according to him, it is unclear how massively it can be used, since it will require huge financial investments. In addition, the pressure at the depth is very high. And the camera can simply be flattened, like the Titan bathyscaphe recently, the expert added.
Wreckage of the underwater vehicle "Titan" from the OceanGate expedition
Image source: Photo: REUTERS/David Hiscock
— The range of such a system is limited to several tens of meters. If, for example, a dolphin rushes by, it will fall into the camera's field of view for a moment, so this idea seems to me hopeless," said Alexander Agafonov, a researcher at the Marine Mammal laboratory of the Institute of Oceanology of the Russian Academy of Sciences.
Recognition of objects under water is quite a solvable task for a neural network, because AI can see any object that a person can see, says Mikhail Fromkin, head of the Educational and Scientific Center for Research in the field of Artificial Intelligence at URFU. Neural networks are perfect for automating routine, but at the same time difficult to formalize tasks, when it is difficult for a specialist to describe why he makes this or that decision, the expert believes.
Denis Gritsenko
