Computer Vision for Fish Diagnosis

[MorrisMustang]

New to keeping an aquarium and recently had my first fish die. Needless to say, this has me wondering a few things.

Did I do something wrong? Did I diagnosis the correct illness/ or disease or parasite or bacteria? Did I buy the right medication?

Members here on R2R were quick to jump in and provide some solutions and diagnosis and for that I am very thankful. What I did realize is that there is a lack of state of the art tools to help out us aquarists.

I have experience in building computer vision diagnostic systems. I am interested to see if it has a place within this hobby. The most important thing to make this work is volume of data, pictures of fish with their diagnosis.

I look to you members. If this is something that would be useful to you, help me build it with data! Send me or post pictures of your sick fish and their diagnosis. I will collect, sort and begin training the computer vision model!

If there are any developers in the community out there that would like to be a part of this, send me a message!

Looking forward to hearing the feedback.

 

[Brett S]

While I think this could definitely be very beneficial I think you may be shooting yourself in the foot asking yourself for the diagnosis from hobbyists. I suspect that many hobbyists misdiagnose diseases more often than they correctly diagnose them and if you put in all that bad data you’re going to wind up with a useless product.

If this is really something that you want to pursue (and I think there is merit in it), then I think crowdsourcing the pictures can be good as you will need a lot of images, but you probably should employ someone who can accurately provide the diagnosis for the pictures, rather than relying on what people think the problem is.

Just as an example, there are countless threads on here where someone posts “My fish have ich, help” and they get replies of “That’s not ich, that’s velvet” or something along those lines. In fact, frankly that problem is exactly why something like this is needed. But you need to build it on accurate data.

 

[Peace River]

New to keeping an aquarium and recently had my first fish die. Needless to say, this has me wondering a few things.

Did I do something wrong? Did I diagnosis the correct illness/ or disease or parasite or bacteria? Did I buy the right medication?

Members here on R2R were quick to jump in and provide some solutions and diagnosis and for that I am very thankful. What I did realize is that there is a lack of state of the art tools to help out us aquarists.

I have experience in building computer vision diagnostic systems. I am interested to see if it has a place within this hobby. The most important thing to make this work is volume of data, pictures of fish with their diagnosis.

I look to you members. If this is something that would be useful to you, help me build it with data! Send me or post pictures of your sick fish and their diagnosis. I will collect, sort and begin training the computer vision model!

If there are any developers in the community out there that would like to be a part of this, send me a message!

Looking forward to hearing the feedback.

I'm am sorry to hear about your fish loss. You pose an interesting idea! I am wondering if fish and/or coral ID would be an easier place to start when developing a vision system and then, if viable, progress more to anomalies such as visual disease indicators and so forth?

 

[MorrisMustang]

While I think this could definitely be very beneficial I think you may be shooting yourself in the foot asking yourself for the diagnosis from hobbyists. I suspect that many hobbyists misdiagnose diseases more often than they correctly diagnose them and if you put in all that bad data you’re going to wind up with a useless product.

If this is really something that you want to pursue (and I think there is merit in it), then I think crowdsourcing the pictures can be good as you will need a lot of images, but you probably should employ someone who can accurately provide the diagnosis for the pictures, rather than relying on what people think the problem is.

Just as an example, there are countless threads on here where someone posts “My fish have ich, help” and they get replies of “That’s not ich, that’s velvet” or something along those lines. In fact, frankly that problem is exactly why something like this is needed. But you need to build it on accurate data.

You’ve nailed the second task right on the head. While most people posting are hobbyists, they solve the initial problem of sick/not sick. Just like hot dog not hot dog. As with every AI or CV problem, higher levels of data cleansing require subject matter expertise. But it’s not worth talking to those people without an initial dataset, even if most are classified incorrectly. They get a quick initial win too in making easy corrections.

For the reason of misdiagnosis and mistreatment, this tool is needed even if it takes years to gather sufficient data.

 

[MorrisMustang]

I'm am sorry to hear about your fish loss. You pose an interesting idea! I am wondering if fish and/or coral ID would be an easier place to start when developing a vision system and then, if viable, progress more to anomalies such as visual disease indicators and so forth?

I thought about that. I’d be down to get those projects going as well as they rely on similar frameworks. Let me start new threads for those in the right places and we can see where it goes.

 

[Jay Hemdal]

New to keeping an aquarium and recently had my first fish die. Needless to say, this has me wondering a few things.

Did I do something wrong? Did I diagnosis the correct illness/ or disease or parasite or bacteria? Did I buy the right medication?

Members here on R2R were quick to jump in and provide some solutions and diagnosis and for that I am very thankful. What I did realize is that there is a lack of state of the art tools to help out us aquarists.

I have experience in building computer vision diagnostic systems. I am interested to see if it has a place within this hobby. The most important thing to make this work is volume of data, pictures of fish with their diagnosis.

I look to you members. If this is something that would be useful to you, help me build it with data! Send me or post pictures of your sick fish and their diagnosis. I will collect, sort and begin training the computer vision model!

If there are any developers in the community out there that would like to be a part of this, send me a message!

Looking forward to hearing the feedback.

Cool idea. The biggest obstacle is that many diseases have non-visual symptoms (rapid breathing, poor appetite, etc). Then, a number of visual symptoms overlap diseases - confusing ich and velvet is quite common. I find that using a microscope is the most important diagnostic tool, but one that people generally don’t have access to.
I’m finishing up a book on fish diseases, and while it has some sample disease images, I rely more on verbal descriptions to help try and diagnose issues.
What about a diagnostic key?
Jay

 

[Weasel1960]

New to keeping an aquarium and recently had my first fish die. Needless to say, this has me wondering a few things.

Did I do something wrong? Did I diagnosis the correct illness/ or disease or parasite or bacteria? Did I buy the right medication?

Members here on R2R were quick to jump in and provide some solutions and diagnosis and for that I am very thankful. What I did realize is that there is a lack of state of the art tools to help out us aquarists.

I have experience in building computer vision diagnostic systems. I am interested to see if it has a place within this hobby. The most important thing to make this work is volume of data, pictures of fish with their diagnosis.

I look to you members. If this is something that would be useful to you, help me build it with data! Send me or post pictures of your sick fish and their diagnosis. I will collect, sort and begin training the computer vision model!

If there are any developers in the community out there that would like to be a part of this, send me a message!

Looking forward to hearing the feedback.

I think it’s a great idea. I just sent @Jay Hemdal a thread asking if there were sticky notes regarding illness identifications and standard treatments. Maybe you guys could collaborate him as the expert in I’d and treatment you as the expert in the software. One thing to note pictures should be clear might have to specify lighting conditions and Jay (or other experts/long time Reefers) should determine illness, cure, etc.

just a thought.

 

[Juniorh2r]

Awesome idea! Would absolutely be useful the coral one too!

 

[Jay Hemdal]

Back in the 1980's I had written some computer software called Aquapro. I sold the company in the mid-1990's when I couldn't port the code over to Windows (it was an MS-DOS program). In it was a diagnostic key - that worked pretty well even with no pictures at all - people could run different scenarios and come up with different results/cures. I wish I had the time to rebuild that....

Jay

 

[Peace River]

Back in the 1980's I had written some computer software called Aquapro. I sold the company in the mid-1990's when I couldn't port the code over to Windows (it was an MS-DOS program). In it was a diagnostic key - that worked pretty well even with no pictures at all - people could run different scenarios and come up with different results/cures. I wish I had the time to rebuild that....

Jay

Jay - that sounds amazing and would definitely be useful!!!

 

[garbled]

My fear with the training data is not if people get the diagnosis wrong, but the commonality of the species. For example, if you end up with 1000 pictures of a powder blue tang with ich, and 10 of various clowns, then the alg is going to say "if it's a powder blue tang, then it has ich".

I almost feel that algorithm-wise, you need to start with "identify the fish", and then go into another training set for that fish for ich/not-ich.

I also wonder if you will need to somehow delete the backgrounds off the images before processing, so you don't get strange artifacts like "I see an acro in this photo so it has ich". Remember the amusing gaffe where the skin cancer algo trained itself to identify cancer by "this photo has a ruler for scale in it".

I think however, with enough training data, this could potentially be awesome. It's almost the perfect use for CV, but I think the datasets are going to be hard to get...

 

[MorrisMustang]

Back in the 1980's I had written some computer software called Aquapro. I sold the company in the mid-1990's when I couldn't port the code over to Windows (it was an MS-DOS program). In it was a diagnostic key - that worked pretty well even with no pictures at all - people could run different scenarios and come up with different results/cures. I wish I had the time to rebuild that....

Jay

Do you have the code base? By rebuild it you mean rewrite all of the case statements or just give it a facelift?
I’d love to learn more about what you Previously built. You’d be surprised at how some of these things can be spun up today.

 

[MorrisMustang]

My fear with the training data is not if people get the diagnosis wrong, but the commonality of the species. For example, if you end up with 1000 pictures of a powder blue tang with ich, and 10 of various clowns, then the alg is going to say "if it's a powder blue tang, then it has ich".

I almost feel that algorithm-wise, you need to start with "identify the fish", and then go into another training set for that fish for ich/not-ich.

I also wonder if you will need to somehow delete the backgrounds off the images before processing, so you don't get strange artifacts like "I see an acro in this photo so it has ich". Remember the amusing gaffe where the skin cancer algo trained itself to identify cancer by "this photo has a ruler for scale in it".

I think however, with enough training data, this could potentially be awesome. It's almost the perfect use for CV, but I think the datasets are going to be hard to get...

Datasets are always the hardest part. Cropping backgrounds is a function of dataset size as noise can be filtered out over time. The beautiful thing about these projects is they require the development of ancillary tech that itself is useful and valuable.

 

[Ben Pedersen]

I thought about that. I’d be down to get those projects going as well as they rely on similar frameworks. Let me start new threads for those in the right places and we can see where it goes.

I think both suggestions are good. This is a great idea! It will be hard to get enough data to adequately train AI for the sickness diagnosis. Are you going to create a GitHub repo for this?

 

[MorrisMustang]

I think both suggestions are good. This is a great idea! It will be hard to get enough data to adequately train AI for the sickness diagnosis. Are you going to create a GitHub repo for this?

The past few projects I did like this had the image repo in Dropbox to allow collaboration on larger files. The code will likely be on gitlab as it has better automated deployment than GitHub.

 

[Jay Hemdal]

Do you have the code base? By rebuild it you mean rewrite all of the case statements or just give it a facelift?
I’d love to learn more about what you Previously built. You’d be surprised at how some of these things can be spun up today.

I was looking for it the other day, I may have a copy in my safe deposit box. It would be on 3.5” floppies. It was written in xBase, almost entirely of nested if/then statements. My guess is that with my increased knowledge of fish diseases, none of the statements would work without a total rewrite. Easier to start over I think.
Jay

 

[MorrisMustang]

I was looking for it the other day, I may have a copy in my safe deposit box. It would be on 3.5” floppies. It was written in xBase, almost entirely of nested if/then statements. My guess is that with my increased knowledge of fish diseases, none of the statements would work without a total rewrite. Easier to start over I think.
Jay

Turn it into an Alexa skill!

 

[Ben Pedersen]

The past few projects I did like this had the image repo in Dropbox to allow collaboration on larger files. The code will likely be on gitlab as it has better automated deployment than GitHub.

Cool, let me know when it is out there.. I might just have to take a look.

 

[NeveroddoreveN]

New to keeping an aquarium and recently had my first fish die. Needless to say, this has me wondering a few things.

Did I do something wrong? Did I diagnosis the correct illness/ or disease or parasite or bacteria? Did I buy the right medication?

Members here on R2R were quick to jump in and provide some solutions and diagnosis and for that I am very thankful. What I did realize is that there is a lack of state of the art tools to help out us aquarists.

I have experience in building computer vision diagnostic systems. I am interested to see if it has a place within this hobby. The most important thing to make this work is volume of data, pictures of fish with their diagnosis.

I look to you members. If this is something that would be useful to you, help me build it with data! Send me or post pictures of your sick fish and their diagnosis. I will collect, sort and begin training the computer vision model!

If there are any developers in the community out there that would like to be a part of this, send me a message!

Looking forward to hearing the feedback.

This is a great direction! As a developer, I work with separately with Rust, Haskell and machine learning (almost entirely NLP).

I've thought about using a GAN to dream pictures of reefs, but I've never worked with image data or GANs ;Happy

 

[MorrisMustang]

This is a great direction! As a developer, I work with separately with Rust, Haskell and machine learning (almost entirely NLP).

I've thought about using a GAN to dream pictures of reefs, but I've never worked with image data or GANs ;Happy

I’m dreaming of that Nvidia paint to landscape GAN for aquascape layouts. Draw some lines and it coverts it into a realistic reef rock layout.