Health Status and Quality of Life in Tuberculosis: Systematic Review of Study Design, Instruments, Measuring Properties and Outcomes

Sumeera Khan1*, Ayesha Imtiaz2 and Ali Nisar Khan3

1Discipline of Clinical Pharmacy, School of Pharmaceutical Sciences, University of Science, Malaysia

2School of Biological Sciences, University of Science, Malaysia

3Discipline of Biotechnology, National Institute of Biotechnology and Genetic Engineering Faisalabad, Pakistan

*Corresponding Author:
Sumeera Khan
Discipline of Clinical Pharmacy
School of Pharmaceutical Sciences
University of Science
Pulau Pinang 11800, Penang, Malaysia
Tel: +060175226406
E-mail: khansumeera@gmail.com

Received Date: 30 November 2016; Accepted Date: 23 December 2016; Published Date: 03 January 2017

Citation: Khan S, Imtiaz A, Khan AN. Health Status and Quality of Life in Tuberculosis: Systematic Review of Study Design, Instruments, Measuring Properties and Outcomes. Health Sci J. 2017, 11: 1.

 
Visit for more related articles at Health Science Journal

Abstract

Background: Evaluation of patient reported outcome measures are important in improving the quality of life and effectiveness of care. Several measures have been developed to assess the health related quality of life of patients with tuberculosis.

Objective: To systematically identify health related quality of life outcome measures that could be used in tuberculosis care and examine their feasibility of use and psychometric properties. A systematic literature review and analysis of psychometric properties.

Method: Systematic literature research was done through PubMed, EMBASE and web search. Eligible studies assessed for unidimensional or multidimensional health related quality of life in patients with tuberculosis disease or infection using standardized instruments. Measuring psychometric properties such as internal consistency, reliability, validity and responsiveness were also examined. Results of included studies were summarized qualitatively.

Results: From 6020 articles, 99 articles were selected for full text assessment. A total of 30 studies meet or inclusion exclusion criteria and examined the psychometric properties of 6 health related quality of life measures. Evidence was limited as half of the information on psychometric properties per instrument was missing and measurement error was not analyzed in any of the included articles and responsiveness was only analyzed in one study.

Conclusion: A variety of instruments are used to assess the health related quality of life in tuberculosis patients. However, there has been only one disease specific instrument developed, making it difficult to understand the impact of illness. Most of the studies included does not required measuring properties or had problem with methodological quality. Further validation studies are required to support the use of health related quality of life measures in tuberculosis patients.

Keywords

Health related quality of life; Outcome assessment (healthcare); Psychometric properties; Tuberculosis

Introduction

Tuberculosis (TB) is a major disease infected approximately 10.4 million new cases in the world with 1.4 million deaths reported in the year 2015 [1]. The customary clinical and biological indicators often fail to characterize self-observed functions, and physical and mental well-being of patients. Thus, an area, health related quality of life (HRQOL) has increased curiosity about dreadful diseases, like TB [2]. HRQOL mainly indicate the perception of patients about their physical and mental health [3].

HRQOL is defined as “the extent to which patient’s subjective perception of physical, mental and social wellbeing are affected on a day to day basis by a disease and its treatment [3]. It is known that patients with chronic diseases, in addition to pure physical health also place high value on their mental and social wellbeing [3]. As a result, evaluation of HRQOL has become an important health outcome and an area of concern for policy makers, health care professionals and researchers [3].

Generally, HRQOL is evaluated by self-administered questionnaires filled by patients. Therefore, these questionnaires are stated to as patient reported outcome measures (PROMs). HRQOL instruments can be generic or disease specific. Generic instruments don’t need any specific situation for interpretation of results. Thus, the comparisons with healthy individuals can be made easily without accessing other diseases. Also, diseases specific instruments are more sensitive and need a specific health situation [4].

Previous studies on health relate quality of life of TB patients before 2008 indicated the two major domains of quality of life [5]. However, most studies were focused on the use of only one reported HRQOL. A detailed study was performed on impact of quality of life in TB patients based on a specific subgroup [6]. Although, various standard instruments for HRQOL measurement are available [7] but the reliability, validity and awareness of these instruments in public is still limited. This review described the present scenario of awareness and development for HRQOL measurements in the area of TB research. We aimed to evaluate the most frequently used HRQOL instrument(s) in the patients of TB to demonstrate the properties and general recovery patterns based upon the Consensus-Based Standards for the assortment of health status measurement instruments (COSMIN) checklist [8].

Methodology

This literature review was performed in accordance with Preferred Reporting Items for Systematic Reviews and Meta analyses (PRISMA) guidelines [9].

Literature search strategy

PUBMED, EMBASE and other data bases were searched for HRQOL articles of TB patients published from 1 January 2004 till December 2015. The keywords such as health status, health related quality of life (HRQOL), outcome measurements, quality of life, patient reported outcomes (PRO) in combination with terms used to find studies on measuring properties of HRQOL were searched. A reference list of the included articles was also updated for other publications. Further measurements were performed on the basis of expert’s suggestions.

Selection criteria

Inclusion/Exclusion criteria: The articles with the following details were included in the review.

• Which described the generic or disease specific HRQOL as primary or secondary outcome.

• One or more measurement properties of an instrument that measured physical, mental and social aspects of HRQOL were examined.

• Included measures could be completed by self, parent or clinician.

The articles not filling the above mentioned criteria were excluded from the review. The target population containing TB patients suffering from any type, cause and degree of the severity. All the text articles were written in English language.

Study selection

Two independent reviewers (S.K) and (A.I) thoroughly examined the full articles of all the studies and the title or abstract which meet the selection criteria were retrieved. All disagreements and discrepancies were resolved by a third reviewer (A.N.K).

Data extraction and quality assessment

The methodology quality was rated using COSMIN (Consensus based Standards for the selection of health Measurement Instruments) checklist [10]. The checklist consists of 9 boxes with a list of 5-18 items per box. This criteria list indicated whether the included study meets the standard for good methodology quality or not. The measuring property in the checklist includes reliability, validity, responsiveness and interpretability. Each item is scored on the rating scale (i.e., poor, fair, good and excellent) [11]. Table 1 gives definitions of these measuring properties.

Table 1 Definitions of these measuring properties.

Reliability Defined as the extent to which the measurement is free from measuring error and it include internal consistency and reliability [12].
Validity  Defined as the degree to which a questionnaire measures the construct to claim and it include content and structural validity [12].
Responsiveness Defined as the ability of an instrument to detect change over time and it include only one measurement property [12].
Interpretability Defined as the extent to which one can assign qualitative meaning to an instrument quantitative scores or change in scores [12].

Results synthesis

Measuring properties of each instrument can be rated as positive, negative, and indeterminate, on the basis of level of evidence. Initially this criteria was used for systematic reviews on clinical trials but it can also be used in reviews on measurement properties [11] as shown in Table 2. The assessment of the result is based on the criteria set by Terwee et al. [13] as shown in Table 3. Approval of research ethics or institutional review board was not required because this was based upon the published data.

Table 2 Degree of assessment for quality of measurement property.

Degree Assessment Measures
Strong +++ or --- Excellent and consistence finding in one or multiple studies of good methodology quality
Moderate ++ or -- Good and consistence finding in one or multiple studies of fair methodology quality
Limited + or - Single study of fair methodology quality
Conflicting ± Contradictory findings
Unknown ? Studies with poor methodology quality

Table 3 Criteria for quality measurement properties.

Measure Assessment Criteria comment
Reliability
Internal consistency + Unidimensional subscale and Cronbach’s alpha =0.07
-  Not unidimensional or Cronbach’s alpha<0.7
? Unknown dimension or Cronbach’s alpha not determined
Measuring error + MIC>SDC or MIC<LOA
- MIC =SDC or MIC = LOA
? Undefined MIC
Reliability + ICC= 0.70 or Pearson’s r= 0.80
- ICC<0.70 or Pearson’s r<0.80
? Undetermined ICC orPearson’s
Validity
Content validity + For target population all questionnaires to be relevant and complete
- Target population consider questionnaires irrelevant and incomplete
? Target population not involved
Construct validity + Factors explain 50% variance
- Factors explain <50% variance
? Unmentioned explained variance
Hypothesis testing + Correlation with instrument = 0.50 or 75% in accordance to the hypothesis
- Correlation with instrument <0.50 or <75% in accordance to the hypothesis
? Correlations with undetermined constructs
Responsiveness
Responsiveness + Correlation with instrument = 0.50 or 75% in accordance to the hypothesis or AUC =0.70
- Correlation with instrument <0.50 or <75% in accordance to the hypothesis or AUC <0.70
? Correlations with undetermined constructs

Results

Literature selection

The database search showed 6020 unique titles of relevant articles (Figure 1). We selected 99 closely related articles which meet all selection criteria. After, second round of screening (Full reading) 68 of these articles did not match the inclusion criteria. Finally, 30 articles were selected for review that measured the HRQOL in patients with TB in general and six studies with the aim to validate a HRQOL instrument in patients with TB. The main reason of exclusion was not using a HRQOL instrument or that the population under study was not TB specific.

hsj-literature

Figure 1: Flow diagram of literature search.

Summary of results

A total 30 articles were included to evaluate the 22 HRQOL measures in adult TB population (Table 4). All these articles were published in English. The data was based on comparative studies of TB population from China [14-34], two from Malaysia [18,23], five from India [14,15,29,35-38], four from Canada [5,24,25,30] one form each UK [17] Pakistan [35], Spain [36] Taiwan [28] South Africa [37] and Turkey [39]. Among the selected articles fourteen studies were cross sectional [5,14,26,27,30,33,34-40] and sixteen were prospective cohort based studies [15-26,29,31,32,41]. An article published by Chamla et al. [16] described the comparison of TB population group from general population. Another study used normative data from the Canadian population as the reference group and two studies included people with LTBI controls [20,30,39]. The sample size varied among all the studies and only one study reported statistical analysis of sample size [30]. In this review, all kinds of TB patients pulmonary TB, extra pulmonary TB, active TB disease, late TB infection, current TB and recurrent TB were included.

Table 4 Summary of included studies.

Reference Country Scale used Study design Study Population Report Time points assessed Outcomes in HRQL domain
Abdulelah [24] Malaysia FACIT-TB Longitudinal cohort Pulmonary TB patients Clinic Unspecified time duringTB treatment Impaired HRQOL improves significantly
Aggarwal[15] India WHOQOL-BREF Cross-sectional Pulmonary TB patients Clinic Unspecified treatment of time Impaired HRQOL improves significantly with anti-tuberculosis treatment. Residual impairment is noticed in some patients at the end of treatment
Atif [19] Malaysia SF-36 v2 Longitudinalcohort Pulmonary TB patients Unspecified Baseline, end of intensive phase Impaired HRQOL improves significantly with anti-tuberculosis treatment. Scores in the physical and mental health components were still impaired after end of treatment
Balgude [16] India WHOQOL-BREF Longitudinal cohort Pulmonary TB patients Self Baseline and 6 months At baseline, HRQOL is significantly affected with physical and psychological domains most affected. All domains improve after 2 and 4 month treatment.
Baure[25] Canada SF36 v2 Longitudinal cohort TB patients Unspecified Unspecified time duringTB treatment Significant negative impact of TB on HRQOL
Baure [26] Canada SF-36, Standard Gamble Longitudinal cohort Active and latent TB patients Self Treatment initiation and 2 months HRQOL is significantly affected with physical domain and is improved after anti tuberculosis treatment
Babikako [27] Uganda MOS instrument, VAS Cross-sectional Active and latent TB patients Clinic Baselineand 2 months end of 8 months HRQOL of life improves with all the domains after treatment
Chamla [17] China SF-36 Longitudinal cohort Pulmonary and extra pulmonary Self Baseline,2 months end and 6 months HRQOL is impaired at baseline with physical scales most affected and improves due to treatment.
Chung [28] Taiwan WHOQOL-BREF Cross-sectional Pulmonary TB patients Self Baseline, end of treatment Impaired HRQOL improved with all the domains
Deribew[29] Ethiopia WHOQOL-HIV Longitudinal cohort TB patients Clinic Baseline and 6 months HRQOL is significantly affected with physical domain scale
Dhingra [42] India DR-12 Longitudinal cohort TB patients Clinic Baseline, end of intensive phase Improvement is significantly seen in all the domains of HRQOL after treatment
Dhuria [43] India WHOQOL-BREF Cross-sectional TB patients Clinic Baseline, 3months and end of treatment TB patients have an impaired HRQOL with significant improvement in all domains except social domain after treatment
Dino [31] Canada VAS and Standard Gamble Cross-sectional TB patients Clinic Baseline, end of 2 months After treatment ends significant improvement is seen
Dujaili [32] Iraq FACIT-TB Longitudinal cohort Pulmonary TB patients Self Baseline,2months and end of treatment Therapeutic intervention had a positive impact on HRQOL
Fu [33] China SAS Longitudinal cohort TB patients Self Unspecified time during TB treatment HRQOL is significantly associated with all the domains
Guo [5] Canada SF-36,HUI2/3,General Health VAS Cross-sectional TB patients Self Baseline, end of 2 months Impaired HRQOL improved with all the domains
Goday [34] Brazil asthma questionnaires 20 score Cross-sectional Pulmonary and MDRTB Self Unspecified time duringTB treatment Slight improvement in all domain of HRQOL after treatment
Hasain [35] Pakistan HAD, Illness Perception Questionnaires Cross-sectional TB patients Self Unspecified timeduring treatment Impaired HRQOL improved with all the domains
Kruijshaar [18] UK SF-36, EQ-5D Longitudinal cohort TB patients Clinic 2 months of treatment Impaired HRQOL improves already after 2 month treatment, but is still below the UK norm score
Lopez-campas [36] Spain SRI Cross-sectional TB patients Clinic Unspecified time duringTB treatment After treatment ends significant improvement is seen in all the domains
Louw [37] South Africa SF-12 Cross-sectional TB patients Clinic Unspecified time duringTB treatment SIGNIFICANT improvement in mental and physical health after treatment ends
Magurie [23] Indonesia SGRQ Longitudinal cohort Pulmonary TB patients Self Baseline, 2months and 6 months Impaired HRQOL improves with treatment at 2 and 6 months
Mamani [20] Iran SF-36 Longitudinal Pulmonary and extra pulmonary Self Baseline, 2months and 6 months Impaired HRQOL improves due to treatment compared to controls
Marra [21] Canada SF-36,BDI Longitudinal cohort Active and latent TB patients Self Baseline,3 monthsand 6 months At baseline HRQOL is more affected in active than latent TB patients. Treatment improves HRQOL in active but not in latent TB. Patients with active TB have still impaired HRQOL after treatment completion compared to US norms
Masumoto [38] Philippines Short Form -8, Duke- UNC functional social support questionnaires, MRC Cross-sectional Pulmonary TB patients Clinic Unpecifiedtime duringTB treatment At baseline , HRQOL is significantly affected by physical and psychological score
Muniyandi [39] India SF-36 Cross-sectional Pulmonary and extra pulmonary Self Unspecified time duringTB treatment Improvement is significantly seen in all the domains of HRQOL after treatment
Pasipanodya[40] USA SGRQ Cross-sectional Pulmonary TB patients Clinic 6 months and 8 months of TB treatment Scores of physical and mental health are still impaired after the end of treatment
Ralph [22] Indonesia SGRQ Longitudinal  TB patients Clinic Baseline, 2 month end and 6 month Impaired HRQOL improved over treatment timebut morbidity does not end after 6 months
Rajeswari [42] India SF-36 Longitudinal cohort TB patients Self 2 months and 6 months All domains are significantly impaired and improve after 2 months of treatment
Unalan [41] Turkey SF-36,BDI Cross-sectional TB patients Self Unspecified time duringTb treatment All domains of SF36 improve over treatment expect for social functioning

The methodology quality of six studies [15,23,26,27,37,39] was validated by the use of COSMIN check list and results were demonstrated on the scale from poor to excellent (Tables 5 and 6). None of these studies included the measurement error while one study showed cross cultural validity [26]. Outcomes measures mostly focused on perceptions which are subjective and without a gold standard.

Table 5 The measurement properties of specific HRQOL measures used in TB.

HRQOL measures Internal consistency Reliability Measurement error Content validity Structural validity Hypothesis testing Responsiveness Cross cultural validity
Duke Health Profile (DUKE) [37] + + + +   + +  +    
Modified version of St. Georges Respiratory Questionnaire (SGRQ) [39] ? +     + +      
Functional Assessment of Chronic Illness Therapy- Tuberculosis (FACIT-TB) [23] + + + + + +   + + + + ++ + + +  
MOS instrument [26] +  +    + + -     + + +
SF 36 Health Survey (SF 36) [15] + + + + + +            
World Health Organizations Quality of Life-BREF (WHOQOL BREF) [27] + + + +   + + + + +    

Table 6 The methodology quality of HRQOL measurement properties as described in the original development articles.

HRQOL measure Ref Internal consistency Reliability Measurement error Content validity Structural validity Hypothesis testing Responsiveness Cross cultural validity
DUKE [37] Excellent Fair   Fair Fair Fair    
MOS instrument [26] Fair Fair   Good Poor     Excellent
FACIT-TB [23] Excellent Excellent   Good Good Good Excellent  
SF 36 Health Survey (SF 36) [15] Excellent Excellent            
SGRQ [39] Poor Fair   Good        
WHOQOL BREF [28] Good Good   Good Fair Good    

Internal consistency was excellent for Functional Assessment of Chronic Illness Therapy- Tuberculosis (FACIT-TB) in one of the study from Malaysia [23]. There results showed that the reliability of the subscales ranging from good to excellent as by the rule of thumb with a total score of 0.87 and for all the subscale ranging from 0.736-0.871 [23]. The validation study from Taiwan [27] used WHOHRQOL –BREF (World Health Organization Health Related Quality of Life- BREF) also showed an adequate reliability. However, only one HRQOL instrument showed cross culture adaptability [26].

Main findings

To the author's knowledge, this is the first systematic review of outcome measures based on TB population. The major highlights of this review are to examine the feasibility of measurement uses, and qualities of methodology. Among the fifteen HRQOL measures studied, eleven measures were generic and four diseases specific. All the measures were developed in English language and most frequently studied measure was SF-36.

Measurement properties of HRQOL instruments

Generic instrument measures were used in 10 studies with different language versions of SF-36 [5,16-20,24,25,38,40,41]. SF-36 is a generic health outcomes measure consisting of 36 items aggregated into eight sub-scales of PF (physical functioning), RP (role physical), BP (body pain), GH (general health), VT (vitality), SF (social functioning), RE (role emotion) and MH (mental health) [43]. A long medical outcome study (MOS) instrument was used in USA [39] which covered multiple dimensions including physical and emotional wellbeing [43].

SF-36 was developed from subsets from the MOS instrument [6]. In a previous study, the variance in SF-36 scores of patients with latent and active TB was described [5]. Results showed a worse mean PCS (Physical Component Summary) score of 44.8 and MCS (Mental Component Summary) score of 40.1 in the patients with active TB as compared to latent TB patients with mean PCS and MCS scores of 54.7 and 50.3 respectively [5]. Studies have reported that when patients were evaluated with SF-36, they showed significant improvements in HRQoL at the completion of intensive phase of treatment [17,20]. A validation study of SF36 was described by Chamla [16] which suggested that the validity and reliability scores were high at the end of the treatment with Cronchbachs score >0.7 [16].

Dhingra and Rajpal et al. [29] used the disease specific instruments DR-12. This instrument consists of a total 12 items, among them 7 cover TB symptoms and 5 are related to socio-psychological characteristics and exercise adaptation [29]. Response options were presented on 3 point scale and equal weightage were given to each item while calculating the 2 domain score and the total score [29]. Pasipanodya et al. [39] used St. George Questionnaires (SGRQ) in 100 patients with PTB. It is an extensively used specific instrument for measuring HRQOL in patients with COPD (chronic obstructive pulmonary disease) and other types of respiratory disorders [39]. Pasipanodya et al. had also reported a significant increase in SGRQ scores as compared to latent TB scores [39]. Magurie et al. [22] described changes in health status by using a modified version of SGRQ (Base line; 45.4) in 115 subjects diagnosed with smear TB positive from Indonesia. After 2 months of treatment, 94% improvement was recorded in at least 4 points [22].

Chung et al. [27] assessed the quality of life by using 4 domain model of WHOQOL-BREF in TB affected population of Taiwan. Internal consistency reliability coefficients scores were 0.92 and 0.93 for the subjects and control referents, correspondingly [27]. Babikako et al. [26] carried out a validation study for the feasibility to use MOS instrument. All subscales have adequate internal consistency with Cronbach’s alpha >0.7. Construct validity varied with different stages of treatment in functional status and wellbeing of TB patients [26].

Abdulelah et al. [24] used a disease specific instrument (FACIT-TB, Functional Assessment of Chronic Illness Therapytuberculosis) for the measurement of the quality of life in TB patients. FACIT-TB (Functional Assessment of Chronic Illness Therapy- tuberculosis) consist of 27 items and their subsets described the disease symptoms associated to the infection site, adverse effects, and additional QOL dimensions (fatigue, social stigma, economic burden) of the ilHOlness. Factor analysis confirmed that the FACIT-TB construct comprised of five domains which are comparatively brief, easy to manage, easy to score, and suitable for the use in clinical trials [23].

Discussion

The strategy of evaluating the HRQOL in TB patients is an important outcome to measure the efficacy of new treatments or interventions. Typically, HRQOL measures have been developed and used to describe the mean scores for specific groups. During the last decade a rapid increase in the number of measures to assess the HRQOL in TB patients was recorded. We identified 22 different HRQOL measures with internal consistency, reliability, measurement error, content validity and responsiveness depending on the information obtained from the literature. Some of the HRQOL measures showed some aspects of psychometric strength especially construct validity. However, these have different characteristics and most of them did not complete the required properties proposed by Terwee et al. [10]. Notably, the SF-36, a commonly used HRQOL measure, was not fully validated in the original study [16]. Furthermore FACIT-TB is a shorter protocol comparatively which made its use popular worldwide [23]. We used COSMIN check list to evaluate the methodology quality of the original HRQOL measures in development studies. This included evaluation of different properties such as reliability, internal consistency, content validity, structural validity, responsiveness and measurement error. Using the criteria of COSMIN checklist most of the studies were rated as fair to poor because of insufficient information or these did not match the required standards. Thus, our results suggested that high quality studies are required for the proper evaluation of the measurement quality. The use of COSMIN checklist criteria in systematic reviews of outcome measures has increased. Though, a major limitation of COSMIN is that it cannot be used for the evaluation of old measures. The inconsistency in the measurement properties is due to disagreement in the definitions and their different criteria. Moreover, the questionnaires still need to meet the criteria of validity and reliability and should be described comprehensively. The selected studies assessed by COSMIN, these showed a poor methodology quality and missing items. The measuring properties will be rated fair even if these are not well defined. Most of the HRQOL instruments developed recently and additional study is required for their validation and reliability.

The COSMIN checklist and the quality criteria for two reviewers can be different. In such cases of disagreements a third reviewer can be consulted. Our research is limited to English language only so there is possibility of missing the measure developed in other languages. As far author’s knowledge the data was included from the TB population of non-English speaking countries and we did not find any non- English HRQL measure. Previous reports of HRQOL measures in the TB patients have included a limited number of measure and only a single concept of multidimensional HRQOL [6,44].

Till date, the evaluation of the methodology quality and instrument properties of HRQOL measures has not been reviewed systematically in literature. The major strength of this systematic review is to consider the related concepts like disease burden, productivity, fatigue and social impact. We preformed the literature search in an organized manner to identify all HRQOL measurements used in TB population. To the best of our knowledge this is the first review of HRQOL measures in TB which evaluated the properties and methodological qualities using a robust and standardized approach. This also described the detailed comparison between the HRQOL measures and properties of quality measurement. This review will guide the use of HRQOL in various clinical and research studies. It will also help the clinicians, researcher and general public to assess the scientific literature on HRQOL measures easily. Several new HRQOL measures are emerging and our study showed that most of the HRQOL are supported by evidence of at least one type of reliability or validity and further validation studies might support their use. The choice of HRQOL measure in future will depend on the context for which hit will be used (e.g. social or disease burden). Until then, the FACIT-TB [23] has the strongest published evidence of reliability and validity and is well established in literature.

Conclusion

Conclusively, there is no ideal HRQOL measure for the use in TB as the validation studies. The outcome measurements in TB were hardly ever carried out and there are no specific HRQOL measures for the use in TB population. In the light of development in the field of patient reported outcomes, it is necessary to develop a combination of measures which are important to the individual, family and society. The whole purpose of this study is to improve clinical care and to evaluate services potentially for academic and research purposes [8].

Acknowledgement

We would like to thank Institute of Postgraduate Studies (IPS) of Universiti Sains Malaysia for providing fellowship support Ref.no.P-FD0009/15(R).

Conflict of Interest

None

References

Select your language of interest to view the total content in your interested language

Viewing options

Post your comment

Share This Article

Recommended Conferences

Flyer image
journal indexing image
 

Post your comment

captcha   Reload  Can't read the image? click here to refresh