Bluechemist (Talk | contribs) |
Bluechemist (Talk | contribs) |
||
Line 297: | Line 297: | ||
</figure> | </figure> | ||
− | |||
− | |||
− | |||
− | |||
− | |||
<p> | <p> | ||
MCoINN was trained with around 200 pictures, of which every single one contained multiple colonies. The resulting colony prediction was very reflective of the training data we trained MCoINN with. This was very surprising because we expected it to work with much more training data (~1000 pictures) that we did not even need to do image augmentation for the training data. With much more time it will be interesting to retrain the algorithm with augmented images that will multiply the quantity of the data by at least a factor of 4. Nevertheless this is a very welcomed development, because this lowers the threshold for a custom AI algorithm even lower. The next time any user will only need around 50 colony pictures and image augmentation algorithm to train a functional AI. Moreover, other users who are interested in building their own AI will also benefit from our dataset, which can provide basic structures for the AI regardless of however their colonies look like. We got the pictures from our colony picking pictures collaboration and this is very beneficial for the training because the AI got exposed to various kinds of real lab conditions. More details on the implementation of the AI can be found in | MCoINN was trained with around 200 pictures, of which every single one contained multiple colonies. The resulting colony prediction was very reflective of the training data we trained MCoINN with. This was very surprising because we expected it to work with much more training data (~1000 pictures) that we did not even need to do image augmentation for the training data. With much more time it will be interesting to retrain the algorithm with augmented images that will multiply the quantity of the data by at least a factor of 4. Nevertheless this is a very welcomed development, because this lowers the threshold for a custom AI algorithm even lower. The next time any user will only need around 50 colony pictures and image augmentation algorithm to train a functional AI. Moreover, other users who are interested in building their own AI will also benefit from our dataset, which can provide basic structures for the AI regardless of however their colonies look like. We got the pictures from our colony picking pictures collaboration and this is very beneficial for the training because the AI got exposed to various kinds of real lab conditions. More details on the implementation of the AI can be found in | ||
Line 315: | Line 310: | ||
</figure> | </figure> | ||
− | + | ||
− | + | ||
− | + | ||
<h2 class="subtitle">Let the OT-2 pick them for you</h2> | <h2 class="subtitle">Let the OT-2 pick them for you</h2> |
Revision as of 01:21, 22 October 2019
Colony Picking
Modify your OT-2 to be a colony picker costing less than $300
Opentrons OT-2 is currently the cheapest liquid handling robot on the market and each year many iGEM teams get their hands on one through the Opentrons Robot Challenge. This pipetting robot is not only interesting for iGEM teams, but also for start-up companies (Link to Doulix IHP). To exploit the full potential of the robot and facilitate the workflow of cloning, we decided to turn the OT-2 into a fully functioning colony picker for a relatively low price and open-source based.
Material | Costs |
---|---|
Opentrons OT-2 with P10 Single-Channel Pipette | 5000 $ |
Raspberry Pi 4 B | 55 $ |
Spiral USB-C cable | 7.99 $ |
USB 3.0 Charger Cable - A-Male to Micro-B | 5.99 $ |
Arducam MT9F001 (14MP) | 64.99 $ |
ArduCAM USB3.0 Camera Shield | 99.99 $ |
Lens M27280M07S | 12.99 $ |
PETG required for mount and light table | 15.95 $ |
LEDs | 16.74 $ |
Arduino Uno R3 Rev3 Development Compatible Board | 12.95 $ |
Total | 5276 |
Upgrade your OT-2 with the CPU (Colony Picking Unit)
To enable colony picking with the OT-2 we developed a package of custom-made modular hardware, called the Colony Picking Unit (CPU), which fits perfectly into the slots of the OT-2. All devices are simple to install, easy to use and low cost solutions, while maintaining a high quality
The CPU consists of the light table, containing an Arduino and LEDs making it possible to automatically switch the light on and off while taking pictures of your plate, and the mount for the OT-2 arm, where the Raspberry Pi 4 Model B with an ArduCAM will be mounted, taking and subsequently analysing the corresponding picture.
Of course, all the files required for 3D printing and instructions on how to install the electronic devices are provided open-source (link to hardware and github).
Marburg Colony Identification Neural Network (MCoINN) powered colony detection
For analyzing the taken picture and identifying colonies we decided on using a neural network based artificial intelligence (AI) instead of a hard coded image processing approach This ensures a flexible solution and makes it possible to extend the functionalities in the future even further.
To train the MCoINN, many pictures of agar plates containing E. coli colonies were taken, not only by our team, but also by other iGEM teams in the context of a collaboration (Link to collaboration). Each individual colony which seemed suitable for picking was manually labelled, serving as training data for MCoINN to define the characteristics of a suitable colony.
MCoINN was trained with around 200 pictures, of which every single one contained multiple colonies. The resulting colony prediction was very reflective of the training data we trained MCoINN with. This was very surprising because we expected it to work with much more training data (~1000 pictures) that we did not even need to do image augmentation for the training data. With much more time it will be interesting to retrain the algorithm with augmented images that will multiply the quantity of the data by at least a factor of 4. Nevertheless this is a very welcomed development, because this lowers the threshold for a custom AI algorithm even lower. The next time any user will only need around 50 colony pictures and image augmentation algorithm to train a functional AI. Moreover, other users who are interested in building their own AI will also benefit from our dataset, which can provide basic structures for the AI regardless of however their colonies look like. We got the pictures from our colony picking pictures collaboration and this is very beneficial for the training because the AI got exposed to various kinds of real lab conditions. More details on the implementation of the AI can be found in github1 and github2
Let the OT-2 pick them for you
Combining the designed hardware with the trained AI will turn the OT-2 in a low-budget colony picker. The only thing we had to do, was putting the light table in the OT-2 and connecting it with the robot, mounting the Raspberry Pi 4 and the ArduCAM onto the arm, putting an agar plate onto the light table and then taking a picture of that specific plate. The trained AI would do the rest by identifying the colonies, translating the coordinates from the picture into coordinates of the OT-2 system, to eventually move to that certain position and pick the identified colony.
Via the GUI the OT-2 will take a picture of the colony laid on the light table, which will then be processed by the MCoINN. After analysis coordinates of the colonies from the picture is extracted and translated into coordinates that are intelligible for the OT-2. Via the move_to function from the robot library provided by Opentrons, the pipette will be moved to the target colony, which will be picked.
All done on the Graphical User Interface for Directed Engineering (GUIDE)
We at iGEM Marburg are serious about bringing AI and robotic revolution to biology labs. For this the system has to be user friendly and therefore we created a GUI. It will increase the accessibility of the software: with the GUI people with less technical experience can also operate the colony picker. Moreover the AI training is packaged into the GUI and will allow people to retrain their colony picker according to their own individual needs. Using AI as a backend allows our system to be flexible and using a GUI makes it more accessible.
The GUI is written on Python using the Kivy library. On the backend it connects to a server/ client on both the camera module and the Raspberry Pi inside the OT-2 and acts as a man in the middle communication between both. The GUI enables calibration of the OT-2, labware deck overview, and colony picking within a few clicks to guarantee accuracy and intuitive colony picking experience.
Build AI for Your own Need
Our AI is trained for detecting white E. coli colonies, but we understand that other people’s needs will differ. That is why we chose AI instead of hard-coded algorithm: so that other potential users can build one according to their needs. Some would like to pick bigger, or smaller, or green colonies; you name it. We designed our GUI so that other users can easily train their own AI that will suit their needs.
We contained all of our work on AI within a docker file, which enables training a new custom AI as easy as implementing a few commands. Users will just need to provide a folder containing training and test images according to their own needs. The AI can then be trained on a local computer and will output frozen inference graph at the end, which then can be utilized according to each user’s needs. More details on the implementation of the AI can be found
here
and
here
How does it all work?
Here is a graph that explains our project:
See the visual concept visual concept