Science and Technology Development Journal

An official journal of Viet Nam National University Ho Chi Minh City, Viet Nam since 1997

Skip to main content Skip to main navigation menu Skip to site footer

 Section: NATURAL SCIENCES

HTML

1606

Total

424

Share

Identification of commensal Pseudomonas aeruginosa isolates using duplex PCR targeting the oprL and algD genes






 Open Access

Downloads

Download data is not yet available.

Abstract

Background: Pseudomonas aeruginosa (P. aeruginosa) is a ubiquitous bacterium which can be found in most moisture places such as soil, water, food, plants, and animals including humans. Due to genetic flexibility among strains, there is no standard molecular identification for P. aeruginosa from different sources, particularly for human colonizing P. aeruginosa or commensal isolates.


Materials and Method: Monoplex PCR using published primers of oprL, algD, nfxB gene were compared for their specificity and sensitivity in identification of commensal P. aeruginosa. Twenty- nine human colonizing Pseudomonas isolates including 16 P. aeruginosa isolates and 13 P. aeruginosa-like isolates were used in the study. To improve the detection, two new primer pairs targeting oprL gene was designed (oprL-pp1 and oprL-pp2), and oprL-algD duplex PCR was carried out with newly designed oprL primer pairs.


Result and conclusion: PCR targeting algD or nfxB genes has the same sensitivity of 93.75% and specificity of 100%, while oprL-specific PCR using published primers is more sensitive (100%) but less specific (0%). Duplex PCR yielded high sensitivity and specificity in detecting P. aeruginosa. Both oprL-pp1/algD and oprL-pp2/algD duplex-PCR had 93.75% sensitivity (15/16 P. aeruginosa isolates) and 100% specificity (0/13 P. aeruginosa-like isolates). Besides, oprL-pp2 primers were more specific than oprL-pp1 primers, with only 2 over 13 P. aeruginosa-like isolates detected, while oprL-pp1 primers detected all P. aeruginosa-like isolates. Compared to the monoplex PCR that only targeted oprL gene, the duplex-PCR utilizing oprL-pp2 and algD primer can identify 15/16 P. aeruginosa isolates (93.75% specificity) and reduce the number of oprL-positive isolates required further exact identification. Additionally, the duplex-PCR used in this study was negative for non-Pseudomonas species including E. coli, V. cholera, V. parahaemolyticus, S. aureus. In conclusion, our duplex PCR targeting oprL and algD could be a valuable tool for P. aeruginosa screening.

Introduction

Pseudomonas aeruginosa is a ubiquitous bacterium that can be found in water, soil, food, plants and animals. It can also be isolated from healthy people’s skin, throat, and stool and is considered part of the human commensal flora; thus, it is called commensal bacteria 1 . As a serious life-threatening Gram-negative pathogen, they are responsible for a wide range of minor to severe infections in burn patients, immunocompromised individuals or patients with cystic fibrosis. These infections are difficult to eliminate due to the pathogen’s intrinsic and extrinsic resistance abilities. Therefore, early and accurate detection of P. aeruginosa is critical for treating infected patients.

Many P. aeruginosa detection methods have been developed, including traditional bacterial culture methods, immunological assays, and biochemical tests 2 . However, these tests typically take days to weeks to complete confirmation, with a high rate of misidentification due to cross reactions of P. aeruginosa with other related Gram-negative bacilli. In recent years, polymerase chain reaction (PCR) has been developed as a rapid and reliable molecular method for clinical and environmental P. aeruginosa detection using a number of specific genes, such as ecfX, gyrB, algD, oprL, and fliC . 3 , 4 , 5 Multiplex PCR, which allows multiple target detection in a single PCR run, was also considered for clinical P. aeruginosa 6 . Commensal isolates, on the other hand, have been investigated even though the detection of these strains is important for tracking and analysis of the pathogen and the spread of antibiotic resistance status 7 . In this study, the application of PCR and multiplex PCR for commensal isolates was investigated.

Materials and Methods

Bacterial isolates

Twenty-nine commensal Pseudomonas isolates were used in this study. They included 16 P. aeruginosa isolates and 11 P. stuzeri , 1 P. azelaica and 1 P. nitroreducens isolates, which were identified via 16S rRNA sequencing (Nam Khoa Biotek CO., Ltd).

DNA extraction

Bacterial DNA was extracted by the organic phenol‒chloroform method 8 and quantitated using Take3 microvolume plates (Synergy HT, Biotek). For long-term storage, DNA was stored in 1X TE (Tris-EDTA) buffer at -20°C.

Primer design

Two primer sets, oprL- pp1 and oprL- pp2 ( Table 1 ), were designed based on the oprL gene sequence from NCBI [GenBank: 882991] using Primer-Blast (https://www.ncbi.nlm.nih.gov/tools/primer-blast/) and Primer3 (https://primer3.ut.ee/). An oligo analyzer (https://www.idtdna.com/calc/analyzer) was used to check primer quality, including primer dimer and cross-pairing. The melting temperature and annealing temperature of these two primers were set to be similar to the algD primer pair. The gradient PCR method was then used to confirm the optimal annealing temperature for newly designed oprL primers. Primers were submitted to BLAST (http://www.ncbi.nlm.nih.gov/BLAST) for comparison with available sequences in the nucleotide database for assessing theoretical P. aeruginosa specificity.

Monoplex and duplex PCR for the detection of P. aeruginosa

PCR was carried out in a total reaction volume of 25 µL containing 12.5 µL master mix (iStandard iVAPCR 2X Master Mix, Viet A Technology Corporation, Vietnam), 2 µL DNA template and either 0.5 µM of nfxB primers, 0.4 µM algD primers or 0.4 µM oprL primers (monoplex) or in combination of 0.4 µM of each algD and oprL primer (duplex) (PHUSA Biochem Ltd., Vietnam). PCR was carried out as described in Table 1 . PCR products were checked on a 2% agarose gel run at 100 V for 45 minutes in sodium borate (SB) buffer.

Data analysis

The sensitivity and specificity were determined by the formula from Rajul Parikj MS 9 . In short, sensitivity was calculated as true positive/(true positive+ false negative), and specificity was calculated as true negative/(true negative+ false positive). The data were analyzed using Excel (Microsoft Office, USA).

Results

Monoplex PCR using oprL -pp0 detected all commensal Pseudomonas isolates but not Pseudomonas aeruginosa specifically

Figure 1 . Representative PCR products of the oprL gene (oprL-pp0; 504 bp ) in Pseudomonas aeruginosa . Lane 1: 100 bp DNA ladder; lane 2: P. aeruginosa ATCC 9027; lane 3: Negative control; lane 4 – 11: samples with positive results; lane 12: sample with negative results.

Figure 2 . Representative PCR products of algD gene amplicons (520 bp). Lane 1: 100 bp DNA ladder; lane 2: P. aeruginosa ATCC 9027; lanes 3, 6, 8, 9, 11, 12: P. aeruginosa isolates; lanes 4, 5, 7, 10: non- P. aeruginosa isolates; lane 13: E. coli (negative control); lane 14: non-DNA (negative control).

Figure 3 . Representative PCR products of nfxB gene amplicons (673 bp). Lane 1: 100 bp DNA ladder; Lane 2: P. aeruginosa ATCC 9027; Lanes 3, 4, 5, 7, 8, 9, 10, 11: P. aeruginosa isolates; Lane 6: non- P. aeruginosa isolates; Lane 12: Negative control.

Figure 4 . Duplex PCR using oprL -pp1 (258 bp) and algD (520 bp) primers . Set A (above) and set B (below) performed PCR at the same time. Set A: Lane 1: P. aeruginosa (ATCC 9027); lanes 2-13: confirmed P. aeruginosa isolates; lanes 18 and 19: non- P. aeruginosa isolates; lane 14: V. parahaemolyticus (negative control); lane 15: V. cholerae (negative control); lane 16: master mix without DNA extract (negative control); lane 17: 100 bp DNA ladder. Set B: Lanes 20-23: confirmed P. aeruginosa isolates; lanes 25-34: non- P. aeruginosa isolates; lane 35: E. coli (negative control); lane 36: S. aureus (negative control); lane 24: 100 bp DNA ladder.

Figure 5 . Duplex PCR using oprL- pp2 (299 bp) and algD (520 bp) primers . Set A (above) and set B (below) were subjected to PCR at the same time. Set A: Lane 1: 100 bp DNA ladder; lanes 13-15: confirmed P. aeruginosa isolates; lanes 2-12: non- P. aeruginosa isolates. Set B: Lane 16: 100 bp DNA ladder lane 17: P. aeruginosa (ATCC 9027); lane 18-32: P. aeruginosa isolates lane 33: V. parahaemolyticus (negative control); lane 34: V. cholerae (negative control); lane 35: E. coli (negative control); lane 36: S. aureus (negative control); lane 37: master mix without DNA extract (negative control).

PCR using oprL -pp0 primer pairs detected all 16 P. aeruginosa isolates; thus, the sensitivity was 100%. However, it failed to differentiate P. aeruginosa isolates from other Pseudomonas species, including P. stuzeri , P. azelaica and P. nitroreducens , resulting in a specificity of 0% ( Figure 1 , Table 2 ).

Table 1 Characteristics of published and designed primers and nucleotide sequences
Table 2 Specificity and sensitivity of oprL/algD duplex PCR to detect P. aeruginosa

Monoplex PCR targeting algD and nfxB detected Pseudomonas aeruginosa specifically

PCR using algD and PCR using nfxB gave identical results. All non- P. aeruginosa samples were negative. All P. aeruginosa isolates gave positive results except one P. aeruginosa isolate (isolate 234.1) ( Figure 2 , Figure 3 ). The specificity of PCR using algD and nfxB for detecting P. aeruginosa was 100%, while the sensitivity was 93.75%.

Duplex PCR targeting oprL and algD

Two primer pairs for oprL were designed in this study with the aim of maintaining the ability to detect 234.1 but not other non-PA isolates. They were coupled with algD in a duplex PCR to design an effective identification method for commensal P. aeruginosa .

The results showed that oprL- pp1 still detected 13/13 non- P. aeruginosa isolates, while oprL- pp2 could differentiate some of the non- P. aeruginosa isolates. In addition, both primer pairs in the oprL/algD duplex were negative for all other bacterial species ( E. coli, S. aureus, V. paraheamolyticus, V. cholera) ( Table 2 ).

Both the oprL -pp2/ algD and oprL -pp1/ algD duplex PCR tests had the same detection efficiency, with 100% specificity and 93.75% sensitivity, since there was one isolate (isolate 234.1) detected by oprL primers but not by algD ( Figure 4 , Figure 5 ). The isolate 234.1 showed only one band for oprL in the electrophoresis gel in both duplex PCR assays.

More importantly, duplex PCR using oprL- pp2 primers, while maintaining 100% sensitivity to detect P. aeruginosa , such as PCR using oprL -pp0 and pp1 primer pairs, has a specificity of 84.62%, as only 2/13 P. aeruginosa -like isolates gave positive results ( Table 2 ). On the other hand, all other P. aeruginosa -like species tested positive using the oprL -pp1 primer, similar to the oprL -pp0 primers ( Table 2 ), which resulted in 0% specificity. Thus, duplex PCR using oprL- pp2/algD identified 15/16 P. aeruginosa isolates (93.75%) and narrowed the number of oprL- positive P. aeruginosa- like isolates needing further identification by other methods.

Discussion

Many studies have found that the PCR approach has a higher sensitivity than the classic culture approach in detecting P. aeruginosa , especially at the beginning of colonization 10 , 11 . Along with the advancement of PCR tests for P. aeruginosa , a number of exclusive genes have been found. Khan and Cerniglia developed the first PCR technique for detecting P. aeruginosa based on the exotoxin A gene 11 . oprL, oprI , and algD genes and other genes were also considered in monoplex or multiplex PCR tests for P. aeruginosa identification 4 , 12 , 13 .

algD and oprL are two exclusive genes that have been used in monoplex PCR to detect P. aeruginosa with high specificity and sensitivity. While the sensitivity of both genes was greater than 90%, the specificity of algD was 100% and only slightly higher than 80% for oprL 2 . algD encodes for GDP — mannose 6 — dehydrogenase of the alginate synthesis pathway 14 , while the o prL gene encodes for an outer membrane protein that plays important roles in the interaction of this pathogen with the environment 15 . nfxB, a repressor of the MexCD-oprJ efflux pump, is another potential target for P. aeruginosa detection 16 , 17 . In this study, the monoplex results showed that nfxB maintained the specificity of algD but did not improve sensitivity. Both monoplex PCRs targeting the algD and nfxB genes failed to detect 100% of the P. aeruginosa isolates. PCR targeting only a single fragment of the gene results in a lack of precision, as clinical P. aeruginosa strains display high genotypic variability 18 . In fact, several studies have reported the absence of one or more virulence genes in certain strains of P. aeruginosa 19 , 20 .

Multiplex PCR is more efficient than monoplex due to its ability to simultaneously amplify multiple PCR products in a single reaction, allowing for multiplex detection and greatly reducing the cost and time requirements. Aghamollaei et al. developed a P. aeruginosa detection assay using triplex PCR that amplifies the lasI, lasR , and gyrB genes, which successfully identified 100% of the clinical isolates tested 6 . However, the specificity of this multiplex PCR against P. aeruginosa -like isolates was not fully considered. Another PCR assay using the same genes was effective in identifying 95% of P. aeruginosa isolates from Dorper sheep milk 21 . Multiplex PCR can also be developed to allow for more in-depth diagnostics, as multiple bands and single bands can be interpreted differently, minimizing false positives and false negatives 22 .

This study considers the possibility of a duplex PCR detection method using oprL coupled with algD or nfxB genes. PCR targeting the algD or nfxB genes has a sensitivity of 93.75% and specificity of 100%, while oprL is more sensitive but less specific. Although the De Vos study demonstrated that the published oprL -pp0 primers could sufficiently detect P. aeruginosa from other Pseudomonas species 8 , our data suggested that the primer pairs also detect non- P. aeruginosa species. In a previous report, the same oprL -specific primer set also had just 70% specificity, with only 49 out of 70 oprL -positive clinical samples being P. aeruginosa 4 . Improvement of the specificity of the oprL primers might improve the specificity of this PCR assay. Duplex PCR of oprL/algD resulted in 2 bands, indicating confirmed P. aeruginosa isolates, while one oprL band indicated that further identification methods were needed to confirm the isolate identity, and no band was non- P. aeruginosa . At the same time, by designing primers with the same annealing temperature as algD or nfxB , we can develop a cost-effective duplex PCR to rapidly detect P. aeruginosa.

The designed oprL -pp2 primer pair in this study was more specific in P. aeruginosa detection than oprL -pp1 and oprL- pp0, with only 2 false-positive results (2/13), while the oprL -pp1 primer pair gave positive results for 13/13 P. aeruginosa -like isolates ( Table 2 ). With the oprL -pp2/ algD duplex, the number of suspected P. aeruginosa isolates ( oprL- positive only samples) was reduced to a minimum, reducing the amount of further testing. Thus, duplex PCR with oprL -pp2 and algD had improved specificity in detecting P. aeruginosa .

OprL -pp2 /algD duplex PCR is a simple, specific, and sensitive method for P. aeruginosa identification. It is a cost-effective and time-saving method because the processing time from sample preparation to confirmation completion is less than 3 hours. Although opr L-pp1 /algD and oprL -pp2/ algD duplex PCR have the same sensitivity and specificity, the latter produced more precise results since non- P. aeruginosa samples were less likely to be oprL positive . The 2-target system of the assay decreases the potential for sequence-related false negatives and can provide simultaneous confirmation of positive results.

Conclusion

In conclusion, the results of this study showed that, using the newly designed primers, the duplex PCR assay targeting the oprL and algD genes was able to identify P. aeruginosa with improved specificity. Although additional confirmation of the accuracy of this approach is needed, the results imply that the PCR test presented in this article is a simple, rapid and sensitive tool for the early identification of commensal P. aeruginosa isolates.

List of abbreviations

P. aeruginosa : Pseudomonas aeruginosa

P. stuzeri: Pseudomonas stuzeri

P. nitroreducens: Pseudomonas nitroreducens

V. cholerae: Vibrio cholerae

V. parahaemolyticus: Vibrio parahaemolyticus

E. coli: Escherichia coli

S. aureus: Staphylococcus aureus

PCR: Polymerase chain reaction

TE: Tris-EDTA

SB: Sodium borate

COMPETING INTERESTS

The authors declare that they have no competing interests.

Acknowledgments

This research is funded by Vietnam National University of Ho Chi Minh City under grant number C2020-28-03.

Author’s contributions

Thuc Quyen Huynh wrote the first draft; Nguyen Bao Vy Tran performed the experiments; Thi Thuy Vy Pham collected the isolates; Nguyen Huong Giang Vo performed the experiments; Lam Que Anh Nguyen collected the isolates; Ngoc My Huong Nguyen collected the isolates; Van Dung Nguyen performed the experiment; Thi Thu Hoai Nguyen designed, supervised and reviewed the work.

References

  1. Bergmans D, Bonten M. Colonization and Infection with Pseudomonas aeruginosa in Intensive Care: Endogenous or Exogenous Origin? In: Vincent JL, editor. Yearbook of Intensive Care and Emergency Medicine 1999. Berlin, Heidelberg: Springer; 1999. p. 131-40. (Yearbook of Intensive Care and Emergency Medicine). . ;:. Google Scholar
  2. Tang Y, Ali Z, Zou J, Jin G, Zhu J, Yang J, et al. Detection methods for Pseudomonas aeruginosa: history and future perspective. RSC Adv. 2017 Nov 7;7(82):51789-800. . ;:. Google Scholar
  3. Deschaght P, De Baere T, Van Simaey L, Van Daele S, De Baets F, De Vos D, et al. Comparison of the sensitivity of culture, PCR and quantitative real-time PCR for the detection of Pseudomonas aeruginosa in sputum of cystic fibrosis patients. BMC Microbiol. 2009 Nov 29;9:244. . ;:. Google Scholar
  4. Rashno Taee S, Khansarinejad B, Abtahi H, Najafimosleh M, Ghaznavi-Rad E. Detection of algD, oprL and exoA Genes by New Specific Primers as an Efficient, Rapid and Accurate Procedure for Direct Diagnosis of Pseudomonas aeruginosa Strains in Clinical Samples. Jundishapur J Microbiol. 2014 Oct;7(10):e13583. . ;:. PubMed Google Scholar
  5. Qin X, Emerson J, Stapp J, Stapp L, Abe P, Burns JL. Use of real-time PCR with multiple targets to identify Pseudomonas aeruginosa and other nonfermenting gram-negative bacilli from patients with cystic fibrosis. J Clin Microbiol. 2003 Sep;41(9):4312-7. . ;:. PubMed Google Scholar
  6. Aghamollaei H, Moghaddam MM, Kooshki H, Heiat M, Mirnejad R, Barzi NS. Detection of Pseudomonas aeruginosa by a triplex polymerase chain reaction assay based on lasI/R and gyrB genes. J Infect Public Health. 2015 Jul 1;8(4):314-22. . ;:. PubMed Google Scholar
  7. Thaller MC, Migliore L, Marquez C, Tapia W, Cedeño V, Rossolini GM, et al. Tracking Acquired Antibiotic Resistance in Commensal Bacteria of Galápagos Land Iguanas: No Man, No Resistance. PLOS ONE. 2010 Feb 1;5(2):e8989. . ;:. PubMed Google Scholar
  8. Sambrook J, Russell DW. Purification of nucleic acids by extraction with phenol:chloroform. CSH Protoc. 2006 Jun 1;2006(1):pdb.prot4455. . ;:. PubMed Google Scholar
  9. Parikh R, Mathai A, Parikh S, Chandra Sekhar G, Thomas R. Understanding and using sensitivity, specificity and predictive values. Indian J Ophthalmol. 2008 Feb;56(1):45-50. . ;:. PubMed Google Scholar
  10. Logan LK, Gandra S, Mandal S, Klein EY, Levinson J, Weinstein RA, et al. Multidrug- and Carbapenem-Resistant Pseudomonas aeruginosa in Children, United States, 1999-2012. J Pediatr Infect Dis Soc. 2017 Nov 24;6(4):352-9. . ;:. Google Scholar
  11. Khan AA, Cerniglia CE. Detection of Pseudomonas aeruginosa from clinical and environmental samples by amplification of the exotoxin A gene using PCR. Appl Environ Microbiol. 1994 Oct;60(10):3739-45. . ;:. PubMed Google Scholar
  12. DA Silva Filho LVF, Levi JE, Bento CNO, DA Silva Ramos SRT, Rozov T. PCR identification of Pseudomonas aeruginosa and direct detection in clinical samples from cystic fibrosis patients. J Med Microbiol. 1999 Apr;48(4):357-61. . ;:. PubMed Google Scholar
  13. De Vos D, Lim A, Pirnay JP, Struelens M, Vandenvelde C, Duinslaeger L, et al. Direct detection and identification of Pseudomonas aeruginosa in clinical samples such as skin biopsy specimens and expectorations by multiplex PCR based on two outer membrane lipoprotein genes, oprI and oprL. J Clin Microbiol. 1997 Jun;35(6):1295-9. . ;:. PubMed Google Scholar
  14. Ryder C, Byrd M, Wozniak DJ. Role of polysaccharides in Pseudomonas aeruginosa biofilm development. Curr Opin Microbiol. 2007 Dec;10(6):644-8. . ;:. PubMed Google Scholar
  15. Hancock REW, Siehnel R, Martin N. Outer membrane proteins of Pseudomonas. Mol Microbiol. 1990;4(7):1069-75. . ;:. Google Scholar
  16. Poole K, Gotoh N, Tsujimoto H, Zhao Q, Wada A, Yamasaki T, et al. Overexpression of the mexC-mexD-oprJ efflux operon in nfxB-type multidrug-resistant strains of Pseudomonas aeruginosa. Mol Microbiol. 1996 Aug;21(4):713-24. . ;:. PubMed Google Scholar
  17. Thai VC, Pham HV, Nguyen DNM, Lambert P, Nguyen TTH. A deletion mutation in nfxB of in vitro-induced moxifloxacin-resistant Pseudomonas aeruginosa confers multidrug resistance. Acta Microbiol Immunol Hung. 2017 May 30;64(3):245-53. . ;:. PubMed Google Scholar
  18. Vosahlikova S, Drevinek P, Cinek O, Pohunek P, Maixnerova M, Urbaskova P, et al. High genotypic diversity of Pseudomonas aeruginosa strains isolated from patients with cystic fibrosis in the Czech Republic. Res Microbiol. 2007 May 1;158(4):324-9. . ;:. PubMed Google Scholar
  19. Chugani S, Kim BS, Phattarasukol S, Brittnacher MitchellJ, Choi SH, Harwood CS, et al. Strain-dependent diversity in the Pseudomonas aeruginosa quorum-sensing regulon. Proc Natl Acad Sci. 2012 Oct 9;109(41):E2823-31. . ;:. PubMed Google Scholar
  20. Alonso B, Fernández-Barat L, Di Domenico EG, Marín M, Cercenado E, Merino I, et al. Characterization of the virulence of Pseudomonas aeruginosa strains causing ventilator-associated pneumonia. BMC Infect Dis. 2020 Dec 1;20(1):909. . ;:. PubMed Google Scholar
  21. Azemin A, Alias N, Kari A. Identification of Pseudomonas aeruginosa strains isolated from Dorper sheep milk with subclinical-mastitis infection by uniplex PCR using 16S rRNA, lasI/R, gyrB and ecfX genes. Malays J Fundam Appl Sci. 2022 Feb 28;18(1):30-42. . ;:. Google Scholar
  22. Fevraleva I, Glinshchikova O, Makarik T, Sudarikov A. How to Avoid False-Negative and False-Positive COVID-19 PCR Testing. Int J Transl Med. 2022 Jun;2(2):204-9. . ;:. Google Scholar


Author's Affiliation
Article Details

Issue: Vol 25 No 3 (2022)
Page No.: 2481-2488
Published: Nov 14, 2022
Section: Section: NATURAL SCIENCES
DOI: https://doi.org/10.32508/stdj.v25i3.3961

 Copyright Info

Creative Commons License

Copyright: The Authors. This is an open access article distributed under the terms of the Creative Commons Attribution License CC-BY 4.0., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding data


 How to Cite
Huynh, T. Q., Tran, N. B. V., Pham, T. V., Vo, N. H. G., Nguyen, L. Q. A., Nguyen, N. M. H., Nguyen, V. D., & Nguyen, T. T. H. (2022). Identification of commensal Pseudomonas aeruginosa isolates using duplex PCR targeting the oprL and algD genes. Science and Technology Development Journal, 25(3), 2481-2488. https://doi.org/https://doi.org/10.32508/stdj.v25i3.3961

 Cited by



Article level Metrics by Paperbuzz/Impactstory
Article level Metrics by Altmetrics

 Article Statistics
HTML = 1606 times
PDF   = 424 times
XML   = 0 times
Total   = 424 times