Team:SMMU-China/Software
Quick Preview
FIRST PART: The design ideas
After we have designed the core component. When the receptor on core cells surfaces bind to the antigen of cancer cells surfaces. It can initiate a downstream signaling pathway, which could activate DNA, we transfected using plasmid or phage, transcription into mRNA. Then the mRNA through the nuclear pore into the cytoplasm translated into the signal molecules we needed (generally some interleukins, such as IL-15). Signal molecules finally secreting into the extracellular fluid into the blood. If we want to detect the released signal molecules, the only way is to puncture skin and take the peripheral blood for detection. Although this method can quickly determine the existence and concentration of the signal molecules, also has its shortage such as, must to collecting the peripheral blood, can not continuous monitoring the concentration and need to go to the professional medical institution to operate, etc.
In our extension part we have designed another engineering cell which could bind to the signal molecules mentioned above, and then express the fluorescent protein floating in cytoplasm. It could bind with the different type of signal molecules and the corresponding cell initiated in order to distinguish the dominant tumor cell population. So that we could detect the optical signal to determine the concentration of molecules.
Our method allows us to monitor the signal molecules expressed by our engineered cells in real time without collecting peripheral blood so that we can monitor cancer development simply and dynamically.
SECOND PART: Principle of the software and modeling
The principle flow chart is as follows:
[ About Our Process ]
The extensions in our overall project can be extended to a variety of fields.（After the core cells generated signal molecules, we could connect a variety of engineered cell for monitoring or treatment, such as importing the gene sequences into engineering cell to produce anticancer drug. Here we have access to a dynamic signal molecule monitoring device, which could initiate the downstream expression to generate fluorescent protein after the detection of signal molecules. At this time, the external device is used to take photos to analyze the corresponding RGB channel strength and forecast the corresponding relative concentration.）
[ About Our Software ]
We use Java as our programming language, which relies on two development packages. We used OpenCV engine for image recognition and jfreechart for final result output method.
Our camera model in laboratory is Canon EOS 6D, in order to guaranteed variable uniformity. Our parameter is: F number: f/14; Exposure time: 1/15; Focal length: 48mm; ISO: 1600.
Processing image by binarization: Using OpenCV engine for binarization, we used the parameter: Imgproc.THRESH_BINARY | Imgproc.THRESH_OTSU. The THRESH_OTSU parameter can be automatically binarized according to different picture scenes. How this algorithm works? First, we divide the image into two parts that are larger than the threshold and smaller than the threshold, which is the foreground and the background. Then first, we divide the image into two parts that are larger than the threshold and smaller than the threshold, that is, the foreground and the background. We can calculate the variance between the two parts of the class. The larger the variance between classes, the greater the difference in gray scale between the two parts. Try the 256 thresholds from 0 to 255 and find the value with the largest variance between classes, which is usually the best threshold we are looking for.
Using algorithm to find contours can be expanded in two parts. The first part is use FindContours parameter to find the edge of the image directly. However, this method is suitable for the single color of the image and it require the recognition area is conspicuous. Otherwise we use the Sobel Operator for edge detection. Technically, it is a discrete difference operator that is used to calculate the approximation of the gradient of the image brightness function. Using this operator at any point in the image will generate the corresponding gradient vector or its normal vector. The advantage is that the noise of the image can be filtered or minimized to enhance the accuracy of the search edge, and the Sobel Operator can be used to find the edge when the image color is more complicated.
After identifying the area, we can filter out the desired area according to the preset maximum and minimum values and the ratio of the area so as to improve the recognition rate of the area and reduce the errors. We also introduce HSV channels here to correct the average value of RGB we got to improve the recognition accuracy. The specific implementation of method is shown below.
Finally, we input the RGB average value of the recognition area into our model to get the relative concentration so as to make the chart to display the relative protein concentration we predicted visually.
[ About Our Model ]
We build a standard predictive curve suitable for our laboratory conditions in the laboratory. First, we took the values from the concentration 0% to 100%. Then we used the same parameters to take photograph, which we mentioned above, and get the relative fluorescence intensity and relative concentration using our algorithm. Then these values used to curve fitting and finally to determine the available model categories for predictions.
THIRD PART: Description and Demonstration
[ About Our Process ]
Image four is the main interface of our software. Select the photo that we have taken using “Choose File” Button on the right and click “Recognize” to get the standard model curve and the relative density of the identified position in the model.
[ Progress Indicator ]
Progress Indicator is a non-resizable and closed window that is only displayed when the image is processing. It is used to show the current recognition progress and observed which step the software stopped when some of the unrecognizable pictures are selected in order to adjust the corresponding parameter to solve the problems. It can also enhance the visualization of the software processing procedure.
[ Contours Screen ]
As the description in the second part. This part of the code can filter the image noise we do not need and identify the wrong contour, as the image below, the identified image noise will be removed.
[ Picture and Chart Container ]
When the processed result input into the model equation, we will pop up two windows. The first window will display the selected image and the second window will display our standard model and predicted concentration values. Right click the chart can pop up a menu to manipulate the chart.
[ Error Indicators ]
1) The left-up dialog box indicates that the selected image exceeds the maximum range that the model could handle.
2) The right-up dialog box indicates that the input box is empty. Most of it happened when click the “Recognize” button without using the “Choose File” button.
3) The left-middle dialog indicates that the internal image container error usually caused by input the wrong non-image file path without using the “Choose File” button.
4) The right-middle dialog indicates that some of the unknown errors were encountered during the processing and the final result container is empty.
5) The bottom dialog box indicates that the standard deviation of the extracted channel is greater than the preset error value. At this time, the software will continue to output the result chart. This dialog box is only used to indicate the value may not be accurate.
[ Filtered by Artificial ]
The code shown in the image saves the filtered legal area into a PNG format picture with an Alpha channel enabled. It is convenient for users to determine whether the parameters passed in the filter are accurate and manually select the target area further more. The meaning of this process is reduce errors. The adjustment method of the preset parameters required by the contour filter will be detailed in the part below.
[ Parameters need to be preset ]
1. Line 23 in [ MainActivity.java ] Double zero indicates the zero correction value of model.
2. File [ Process.java ] p_data method need to pass parameters below:
1) String filename – Selected File Name.
2) Double minarea – Minimum value of contour screen.
3) Double maxarea – Maximum value of contour screen.
4) Double whratioi – Contour aspect ratio threshold of contour screen.
5) Interger mode – Protein types need to be identified.
6) Double c_zero – incoming parameter of zero in MainActivity.
7) Double stdvalue – Allowable error standard deviation preset value.
FOURTH PART: Solved problems and highlight of software
We stimulate and get the standard curve in the laboratory according to the environment we preset. And then, we made another concentration fluorescent protein cell suspension in the laboratory and centrifuged it for detection. We found that the value is consistent with suspense cell solution we made.
[ About Our Process ]
The concentration predicted in the black box is 797.701 and the image we input is 800.000.
From the above results. We can see that under the laboratory conditions we built. We can get better reference values so we can now do the following things below:
1) We verified that the average value of RGB could reflect the concentration.
2) Visually validated our hypothesis with real programs and algorithm while variables fixed.
3) Changing the recognition mode according to different types of fluorescent proteins can improve the accuracy of recognition. And in case of a small range of concentration changes, we can also distinguish the concentration difference based on the nuances of the relative fluorescence intensity captured.
The highlights of our software is:
1) Our program is written in Java programming language and can be transposed into an Android application running on the mobile device.
2) Automatic regional identification and analysis according to preset environment and screening the regional we want by setting the maximum and minimum values to improve recognition accuracy.
3) Our analytical method is based on the analysis of the relative fluorescence intensity of engineering cells without the need for the corresponding experimental operation to analyze the concentration. It is only required to use camera photography to get the photograph meets the identification conditions for analysis. Then the results we want can be obtained quickly and conveniently.
4) Board application prospects in the future.
FIFTH PART: Currently known problems and solutions
[ First ] We can achieve now is to establish a standard curve and input the value in the curve after taking the sample value. The key to succeed is well-controlled the variable, but how to realize the unified control variable in different environments has the following solutions:
As the image shown above, we can put the sample in a closed environment and then fix the parameters of the camera sensor and the excitation light intensity. This basically achieves the uniformity of the variables basically eliminate the error caused by the camera.
[ Second ] How to recognize the types of proteins automatically
The above image shows the HSV color mode. The range of the H-Channels taken by the camera can be used to determine the type of our fluorescent protein. And we have another method of the different excitation light wavelength. Different types of fluorescent proteins can be determined according to different excitation sources. For example below:
Different wavelengths of excitation light can be used to illuminate the mixed cells and then the cells mixture with different kinds of fluorescent proteins can Ben distinguished. After that we can analyze the intensity to distinguish the dominant cell population.
[ Third ] Algorithm Optimization Problems
We have added the region filtering function in the software, as the image 6 and 12 shown. But after fixing the variable. We could find the highlighted area and expand the region in the image so that we could set a hotspot area according that area. Only find contours in the hotspot area further reducing the errors.
SIXTH PART: The future assumption
We could implant this kind of fluorescent protein cells into the surface of the skin and fix them in a microcapsule surrounded by a polymer membrane, which could selectively pass our signal molecules through the microcapsule to the engineering cells. Because of the cells are fixed in a microcapsule, the cell cannot spread throughout the body with blood circulation. The pattern is as follows:
Next, we can customize a small device, which is equipped with a small photosensitive element and an excitation parallel beam. It is worth noticing that a small filter lens can be added in front of the photosensitive element to filter the excitation light, which parts of wavelength can be captured by the camera element. When the device working, it should be placed above the polymer cell cluster and the optical signal captured. We also could place the signal transmitter in device in order to transmit the captured signal to the users’ devices, such as PC, tablet and smartphone. Then the data of users could be uploaded in to the server side and compared with the big data database. Comparison and analysis results can be back to the users, which could contain the following information:
1) Current progress in cancer and the predicted quantity and general distribution of dominant tumor cells populations.
2) Activities that can be carried out at this stage and drug recommendations.
3) Current stage contraindications.
4) Can cooperate with the entity hospitals and relevant with the users can obtain relevant advices from the hospital
5) Etc.
The pattern of this part is shown below:
With this device, we can monitor and collect relevant disease information in real time without puncture and blood sampling. Using the rear database to perform big data comparison and analysis or even associate with the third-party hospitals to obtain help information. Under this situation, the patient could self-test to understand their current condition even they are not aware of their disease at all.
In this case, we use the RGB color mode to analysis but if you want to further improve the accuracy of the results. You can use the spectrometer for sampling and measurement.
Release: https://github.com/igemsoftware2019/SMMU-China_FluorescentGUI/releases
Repository: https://github.com/igemsoftware2019/SMMU-China_FluorescentGUI