Are uBiome’s Tests Reproducible? In a Word: Yes!

In our previous blog post –  which was also posted on Medium – we wrote about the use of 16S rRNA gene (“16S”) amplification and sequencing for the analysis of microbial communities, such as those found in our gut. One of the topics we wrote about was the reproducibility of 16S analysis. Reproducibility is how similar the outcome of a test is if you perform the same test multiple times. In order to consider a result reproducible,  you should be able to process the same sample multiple times, while maintaining consistent results.

We also recently wrote a scientific manuscript about the reproducibility of our sample collection and lab processes, which was posted as a preprint on bioRxiv. In this paper, which is ready for scientific peer review, we investigated how reproducible microbiome profiles generated in our laboratory are.

Here at uBiome, we obviously want our analysis to be as reproducible as possible, and we take many precautions to ensure this is the case. All our customers get the same sampling instructions and tubes with the same buffer to make sure that all samples are treated in the same way. In our laboratory, we process thousands of samples with the same protocol. Our laboratory staff works according to Standard Operating Procedures (SOPs), strict protocols they have to follow, and we have robotic liquid handlers and software that ensure that every sample gets the same treatment. Our lab is also CLIA-licensed and CAP-accredited, so all of our laboratory processes have been reviewed by rigorous external audits.

Still, we wanted to make sure that all these measures had the intended effect of reproducibility. In the manuscript we posted on bioRxiv, we wanted to address some very specific questions that we (and our customers) might have about how our sampling and laboratory processes affect reproducibility. To answer these questions, we did four experiments. We looked at how similar the profiles are of samples taken on adjacent days, how similar the profiles are of samples taken from different pieces of the same toilet paper, and what happens if you analyze the same sample three – or even three hundred – times.

Let’s take a look at the four questions we tried to answer in this paper:

Question 1.

Does it matter on which day I take a stool sample? I did not make any big changes in my diet lately, but I eat something different every day. Does my microbiome vary a lot from one day to another? Is there a best day to sample myself?

Experiment: To address this question, a volunteer literally pooped for science! “Volunteer A”, a healthy man in his 30s, took stool samples over a period of 20 days. Although he did not obtain a sample every day, he sampled his stool on 11 days within this 3-week period. As he was doing this, he followed his regular diet, with normal variation from day to day. All samples from this volunteer were extracted and analyzed in our laboratory.

Conclusion: All samples from volunteer A looked similar to each other (Figure 1A). It did not matter on which day he sampled or that there were small variations in his daily diet; his microbiome shows little variation from day to day. Using a common measure of similarity, the Lin’s correlation, we found an average correlation of 0.68 between samples from different days. However, when we compared volunteer A’s stool samples to that of 8 other subjects, each person had their own unique microbiome pattern (Figure 1B).

On average, the Lin’s correlation between samples from different subjects is only 0.28. This means that samples from volunteer A always looked much more similar to other samples of the same volunteer, than to samples from another person, even if taken on different days. So the answer to the question above is no, your microbiome does not very a lot from day to day.

From this experiment we concluded that there is little variation in a person’s microbiome, as long as large dietary changes aren’t made. Other studies, however, have shown that your microbiome will change if you make large dietary changes, if there are changes in your health or during international travel. So if you are struggling with your health and are taking action to change that by taking supplements or changing your diet, it would be beneficial to do multiple tests to see how your microbiome will change.

Figure 1: To answer questions 1 and 2, subject “A” sampled his stool on 11 days within a 20-day period. On each of these 11 days, he took 2 pieces from the same toilet paper. The microbiome profiles of all 22 samples looked very similar to each other (A, left). Samples from 8 other subjects (2 replicates per subject) looked all very different than those from subject A (A, right). The plot on the right (B) shows in a different way that all 22 samples from subject A (shown in pink) were very similar to each other, but very different from those of the 8 other subjects.

Question 2.

The uBiome Gut Explorer and SmartGut  kits require me to sample from toilet paper. Does it matter which fecal piece I get from that toilet paper? If I would take 2 samples from the same piece of toilet paper and send them in for a uBiome test, would the microbiome profiles of these 2 samples look the same?

Experiment: To answer this question, we asked volunteer A from experiment 1 above to not only sample his stool for 20 days, but to also take duplicate samples. On each day he sampled his stool, he took 2 different pieces of fecal material from the same toilet paper.

Conclusion: The results showed that these 2 duplicate toilet paper samples looked very similar to each other (Figure 1A). Using that same Lin’s correlation that we used to answer the previous question, we found that duplicate toilet paper samples had an average Lin’s correlation of 0.95, which is very high. In other words, samples from the same piece of toilet paper are almost identical, and significantly more similar than samples that this volunteer took on other days. So the answer is yes, two samples taken from the same toilet paper look almost the same.

Question 3. How stable is my sample after I took it and swirled it in the tube with the lysis buffer? Can I really ship it at room temperature? Will some bacteria start to grow? Will my microbiome profile change during the shipping process?

Experiment: We tested if microbiome profiles would change after keeping our samples at different temperatures for a week. We extracted samples immediately and also after keeping them in a freezer, at room temperature, or at higher temperatures (86 or 104 degrees F) for a week.

Conclusion: After processing, these samples looked very similar to each other (Figure 2). So the answer is: yes, your microbiome profile will remain stable, regardless of shipping temperature. Our sampling buffer, the liquid in the tube that you swirl your sample in, will immediately lyse (kill) all microbes, ensuring that your results reflect your microbiome exactly how it was when you sampled it.

Figure 2: Stool samples from 8 different persons were immediately extracted or extracted after storage for a week at different temperatures (freezer, room temperature, 87F or 104F; all in duplicate). Panel A on the left shows that all 10 samples from the same person had nearly identical microbiome profiles, no matter what temperature they were stored at. On the other hand, samples from each individual were very different from one another. Panel B on the right shows the same data in a different way, and confirms that samples cluster per subject, with no apparent effect of temperature treatment.

Question 4. How reproducible is the uBiome process? If you process the same sample multiple times, how similar are those microbiome profiles going to look? What would happen if you process the same sample, say, three times? Do you get the same results, or do these three replicates look very different every time you process them?

Experiment: We extracted the DNA from 44 stool samples, all derived from different people. We then performed the same microbiome analysis test three times, using different reagents and equipment and performed by different technicians in the laboratory.

Conclusion: All three replicate samples from the same person looked very similar to each other, independent of reagents or equipment used, and technician performing the assay in the laboratory. Of the 44 samples, the 3 replicates clustered tightly together in 39 cases. The average Lin’s correlation between the 3 replicate pairs was 0.94, while that between individuals was 0.28. We could also clearly tell these samples apart, as every person has their own unique microbiome.

Figure 3: Extracted DNA from 44 stool samples, all from different people, were used to generate microbiome profiles, on three independent times. While samples from each individual were very different from one another, the three replicates looked very similar to each other.

Question 5. Sure, but what would happen if you processed the same sample hundreds of times? Would the results still all look the same?

Experiment: After testing each sample 3 times, we wanted to know what would happen if you tested the same sample hundreds of times. We took one complete poop sample (the whole sausage!), mixed it in a blender with buffer, divided that into smaller portions, and stored these in the freezer. From there, we prepared 363 even smaller portions, each of which was processed independently. The runs were done by different laboratory personnel and on different machines in our laboratory. Then, we compared all the 363 microbiome profiles.

Conclusion: The result? They still looked very similar to each other, even when processed over 300 times (Figure 4). In our paper, we also show that which technician or machine the run was performed with introduced very little variance. The average Lin’s correlation of all these 363 samples was very high: 0.93.

Figure 4. The top part of the figure shows the 363 replicates generated from the same homogenized stool (HS) sample – these look all very similar. The bottom part shows the microbial genera in samples from 400 other persons – these all look very different from each other.

Conclusions

In summary, our study shows that several aspects of microbiome analysis are reproducible. In a healthy person, there is little day-to-day variation of the microbes found in stool, and it does not matter which piece you take from toilet paper. In addition samples stored in our tubes have been experimentally verified at a wide range of temperatures for at least a week, and extracting the same sample over and over again, will give very similar results.