Nuha A El Sharif*
Al Quds University, School of Public Health, Abu Dis Campus, Jerusalem
Received Date: 04 July 2017; Accepted Date: 20 September 2017; Published Date: 27 September 2017
Citation: Sharif NAE (2017) Reliability of Information Abstracted from Diabetic Patients' Medical Records in Palestine. Health Sci J. Vol. 11 No. 5: 526.
Copyright: © 2017 Sharif NAE. This is an open-access article distributed under the terms of the creative Commons attribution License, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.
Background: Questionnaire based surveys are an importance tool in epidemiology and public health research. Although there may be a certain degree of bias in this quality of care assessment tool since some of these measurements may be misleading, it remains an inexpensive tool for such assessments. The quality of services provided to diabetic patients in Palestine has not been studied previously. This study provides a context for the measurement of the quality of care of patients with diabetes. Objectives: To evaluate the agreement on testing at the proper time between self-report and medical record in primary care centers in the West Bank. Methods: Data from interviews and medical records were available on 882 diabetic patients attending nine primary health care centers in Palestine. Cohen's kappa coefficient (k), agreement beyond chance, between data provided in patient medical record and self-report was calculated. Results: Agreement on testing at the proper time between self-report and medical record was poor to fair in strength (k: 0.06- 0.28), and was poor to substantial (k: 0.10- 0.70) for diabetes comorbids. After adjustment, clinic location and ownership were strong determinants for agreement in the proper time of testing. Also, the personal characteristic age and gender were the most important factors in determining comorbids in those patients. Conclusion: The evaluation of the quality of care based on medical records should be treated with great caution in Palestine since it’s a manual process. However, reliance on both sources of information is useful to establish monitoring tools on which decisions can be based.
Diabetes; Reliability; Medical records
Most reliability or validation studies of self-reported chronic diseases have compared self-reports by participants with their medical records [1-3]. Studies that investigate diabetes, hypertension [2,4-6], hyper-lipidemia [2,6], and cardiovascular diseases [4,7-9] found variable agreement between these two sources of information. These studies found that agreement was most likely to be affected by the personal characteristics of patients such as age, gender, socioeconomic status, obesity, clinical characteristics of patients, patients awareness and literacy, understanding of their health condition, a family history of disease and their ability to recall [2,3,10-14]. However, similar validation studies in countries of political instability with a protracted conflict and which are suffering from fragile infrastructure of social and public health services are not published yet.
The validation studies attempted to determine whether selfreports agreed with the reliable diagnoses obtained by means of clinical examinations, biometrical measurements or structured interviews, whereas validation itself consists of measuring the actual extent of this concordance. Selfreporting of diabetes-specific data has been shown to present moderate validity and be prone to both under and overreporting. [13,15-20] These studies found variable agreement for eye examinations and eye problems, [17,19-21] diabetic foot, [13,21] erectile dysfunction, [15,16] microalbuminuria, [20,21] levels of glycosylated hemoglobin, [20,21] lipids,  blood pressure [2,21] and organ amputation .
Palestine is a country of political instability due to a protracted conflict that has been ongoing for more than 50 years. The Israeli-Palestinian conflict presents professionals with unprecedented challenges and undermines already fragile health care services. The services for diabetic patients are offered by four main care providers: Ministry of Health (MoH), United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), non-governmental organizations (NGOs), and the private sector. However, there is a lack of evidence-based practice due to very limited engagement in research; poor participation by patients and families in diabetes management; and a scarcity of specialist diabetic healthcare professionals.
A population based study (30-65 years of age) showed that the prevalence for type 2 diabetes (DM2), was 11.6% and was 10.3% for impaired glucose intolerance (IGT) [22,23]. The quality of services provided to diabetic patients has not been studied previously. This quantitative study is part of ongoing, mixed methods clinical-based national research to assess the clinical management of diabetes and inform quality improvement initiatives. Details about compliance with and awareness of diabetes guidelines among physicians and nurses in Palestine have already been published. This initial quantitative study highlighted training on diabetes guidelines as the main factor significantly associated with self-reported compliance by nurses and physicians to guidelines for the treatment of diabetic patients. Other factors influencing behaviour were time constraints, availability of guidelines and availability of laboratory tests . This study aims to evaluate the relationship between quality of diabetes care and patient satisfaction in a cohort of 882 diabetic patients using the agreement beyond chance as a tool.
Study design and population
A cross-sectional primary health care (PHC) clinic based survey was conducted in 11 governorates in the West Bank. Three governorates were selected to represent the north, middle and southern governorates. From each selected governorate, three major PHC centers were selected: MoH, UNRWA, and a non-governmental PHC (the Palestinian Medical Relief Society, PMRS).
Diabetic patients registered at the selected centers who visit general practitioners (GP) and/or diabetes care clinics were eligible to participate in the study. A calculated sample size of 900 patients was divided equally between the three governorates. This sample of 300 patients was then distributed proportionally according to the size of the selected centers in each governorate. In each center, a random sample of patients' records was selected and those patients were invited to participate on the day they visit the clinic. The study protocol and questionnaire was approved by Al Quds University Research Ethical Committee. Consent was obtained from the various stakeholders to carry out the study in their centers and to assess their services. All participating patients gave written consent for their involvement in the study.
Collection of data
A questionnaire-based interview survey was conducted. This questionnaire was based on a previously validated questionnaire [25,26], with some minor modifications as suggested by the research team and by the major stakeholders themselves. The questionnaire comprised questions to assess the background characteristics of patients (age, gender, marital status, educational level and address) and diabetes characteristics (family history of diabetes, type of diabetes, patient medical regime and ownership of a glucometer). Also, patients were asked about how their physicians cared for their diabetes in term of laboratory tests (HbA1c, lipid profile, microalbuminurea, kidney function testing); examinations (electro-cardiogram-ECG, foot examination, blood pressure measurements, and eye examination by ophthalmologist); and referral timing and the frequency of performing these tests and examinations. There were also questions on comorbid conditions: eyes problems (retinopathy), extremes numbness (neuropathy), kidney problems (nephropathy), foot ulcer (diabetic foot), heart failure, hypoglycemia, and hypertension. Some questions related to the perception of patients on the follow up of laboratory tests and examinations and diabetes comorbidities associated with their diabetic condition. The final interview format was concluded after the pilot testing of the questionnaire. Patients records data were collected from their records using a specially prepared form.
Based on the two sets of diabetes management guidelines existing in Palestine: i.e., from the Ministry of Health (MoH) and the United Nation Relief Works Agency (UNRWA), we defined the “proper time of testing” as the presence of laboratory test results of HbA1c in the previous three months, last year’s reports for lipid profile, kidney function tests, microalbuminuria, a report of an eye examination by an ophthalmologist, and an ECG report.
Total agreement, over- and under-reporting were calculated depending on patients responses by “Yes” and “No” answers in the questionnaire in comparison to what was recorded in patients files.
The level of agreement for dichotomous “proper time of testing” and the reporting of signs of diabetes comorbids was calculated using the proportion of absolute agreement and beyond chance agreement using Cohen’s kappa statistic (k). The benchmark for determining the closeness of the comparison for k was based on Landis and Koch (1977) . In their scale, the strength of agreement was as follows: 0.00– 0.20 (poor), 0.21-0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), and 0.81–1.00 (almost perfect). The measurement of the presence of diabetes comorbids was also analyzed in a similar manner. All data were entered in a database and were analyzed by SPSS 20 (Statistical Package for the Social Sciences) for Windows.
Separate binary logistic regression models were run separately for all tests and signs of diabetes comorbids. The variables: patients age, gender, marital status, educational level, center location, center supervising body, and number of years since diagnosis were entered in these models. Each variable used in the study was examined for its effect on each model. Variables were omitted from the analyses if they were not significantly associated with agreement in tests, examinations performed or comorbids at a significant level of less than 0.05. However, the year of diagnosis was excluded since it weakened the fitness of all models: the Hosmer and Lemeshow Test.
The characteristics of the cohort are described in Figure 1. Of the 882 participants, 40% were in the 20 to 39 year age group, 5% were less than 20 years of age, and 87% had been educated to school level.
In Table 1, about 70% of the study sample had been diagnosed with diabetes for five years or more, and about 73% of them reported having a family history of diabetes. Eighty patients did not know their type of diabetes and it was not stated in their medical records. Of them, 113 (14%) were diabetes type one; 46 of these patients were below the age of 20.
Table 1 Cohort diabetes characteristics (N=882) attending PHC clinics/centers in the West Bank.
|Registered as diabetic||<5||255||29.50%|
|Family history of diabetes||Positive||636||73.40%|
|Patient medical regime||Tablet||395||44.80%|
|Tablet plus insulin||159||18.00%|
|Patient owns a glucometer||Yes||378||42.90%|
|Medications as recorded in records||Diet only||19||2.20%|
|Tablets and insulin||163||18.50%|
|Type of diabetes N=803*||DM1||113||14.00%|
*79 patients did not know their diabetes type and it was not recorded in their records.
Compared to what is recorded in patients files, the proportion of observed agreement for laboratory testing was low (36.6% to 78%) versus the observed agreement for signs of diabetes comorbids (52% to 91.6%) (Table 2). According to Landis and Koch’s scale the strength of these observed agreements for performing testing at the proper time ranged from fair to substantial. However, agreement of poor to fair strength (k: 0.06 to 0.28) was shown for testing at the proper timing. The poorest agreement was seen for annual testing of microalbuminuria (k: 0.06, total agreement 36.6%) (Table 2).
Table 2 Summary of total agreement and disagreement of the “Proper time of testing”.
|Kidney function test||76.40%||14.20%||9.40%|
The agreement of proper time of testing for HbA1c in the previous three months was 73.3% (k: 0.21) and was 73.1% (k: 0.27) for having an ECG in the previous year. Lipid profiling in the previous year had similar agreement (78%), but lower kappa (0.14) (Table 2). For comorbids, the strength of observed agreement was moderate to almost perfect, but the strength of beyond chance agreement was poor to substantial (0.10 to 0.70). The strongest was for having a hypoglycemic event in the previous year (k: 0.70) (Table 2 and Figure 2).
The associations of clinic location, clinic ownership and cohort characteristics with the strength of agreement between the survey and the data collected from patients medical records are shown in Figures 3 and 4. The location of the clinic in the north and middle regions showed stronger agreement between the two sources of information than for the south for HbA1c, lipid profiling, eye examination, and kidney function testing, but not for others (Figure 3). Similarly, eye problems, kidney problems, heart failure, hypertension and hypoglycemia had doubled agreement in clinics in the north and middle regions compared with the south, but not for extreme numbness (Figure 4).
Ownership of the clinic by MoH and UNRWA showed higher agreement between the two sources of information for lipid profiling, having micro-albumin testing and proper timing of ECG testing in the previous year, but not for HbA1c or kidney function testing (Table 2). This finding was inversely associated for most diabetes comorbids, where NGO patients reported more agreement in general, but similar agreement was found between MoH and UNRWA patient’s reports. However, being educated for more than 12 years showed higher agreement between self-reported proper time of testing for lipid profile, kidney function testing, but not for HbA1c (Figure 3). Patient’s gender, marital status, and patient’s age did not show any significant association with the proper time of testing.
Figure 4 shows that younger, female and highly educated patients showed better agreement in reporting most comorbids compared with older, male and less educated patients. Marital status and clinic ownership did not have any significant agreement with any cormobid.
There is growing interest globally in measuring the performance and quality of care for non-communicable diseases using data collected from medical records. However, this source of information may have a certain level of bias. Therefore, the American Diabetes Association, in its Education Recognition Program (ERP), added patient surveys as another source of health information that can support the primary source of medical records . This study is the first of its kind in Palestine. It evaluates the quality of care of diabetes treatment using patient’s medical records, in addition to their own reports of certain indicators used for evaluation. The results of this research demonstrated substantial differences between information obtained by self-reporting and that obtained by a review of patient’s medical records.
In this study, a questionnaire-based survey showed low agreement between patients self-reports and what was stated in their medical records for the "proper time of testing" of microalbuminuria and lipid profiling, but was moderate for HbA1c, kidney function testing and ECG examination. Agreement was higher for self-reports of the diabetes comorbids of heart failure, hypertension and hypoglycemia, and was moderate for retinopathy, nephropathy, and diabetic foot. There was very low agreement for neuropathy. Despite these agreement coefficients, the limits of agreement remained considerable for all reported measures and illustrated that agreement between self-reporting and medical records is substantially low.
Overall, the reliability of diabetes follow-up indicators in this study varied from that seen in Western countries. In a Swiss community based survey, reported HbA1c measurement in the previous year was highly consistent with physicians reports (Cohen’s kappa was 0.94) . This difference could be due to the fact that, according to the Palestinian diabetes guidelines, HbA1c is recommended to be measured every three months. More than half of our study patients had been advised to measure HbA1c test in the previous year, but of those who remembered the last time they had undergone this test, 98% (452 patients) had undergone it in the previous year and 52% in the previous three months.
Non-response in following the physicians referral for this test may explain low reliability of the HbA1c test. Also, several patients may lack the information or knowledge to recognize if HbA1c had been performed or not. Patients attending the PHC clinics in the south region of the West Bank were shown to have the lowest agreement between self-report and medical records. This might suggest differences in health education for patients attending PHC in the various areas of the West Bank, or variations in the process of treatment for diabetic patients in these areas.
The overall annual self-report of eye examination and medical records in the TRIAD USA study was fair (Cohen’s kappa k=0.25) . However, agreement was moderate (k=0.37) in the Minnesota-USA study , in which findings demonstrated more over-reporting than underreporting of eye examinations by patients. This study found similar findings when comparing self-reports of an eye examination during the previous year with reported referral for eye examination (k=0.20). Also, moderate agreement was found when comparing self-reports of eye problems with a diagnosis of retinopathy in patient’s medical records (k=0.34). Referral for an eye examination was more under-reported (37%), but was over-reported (30.5%) in relation to eye problems when compared with records of retinopathy diagnosis. This might indicate that patients were referred, but did not undergo the eye examination since it is not covered by their health insurance; or patients may be examined by an ophthalmologist, but their examination results are not in their records. Another possible explanation is the physician factor, where a physician obtains an eye examination report without telling the patient their results. Older age groups reported more eye examinations and those followed up in NGOs clinics had much fewer eye examinations than those registered in UNRWA and MoH clinics, which suggests better health services. Agreement was shown to be strongest in the north compared to other regions.
High agreement was apparent between self-reports and medical records for heart failure, hypertension and hypoglycemia (Cohen’s kappa >0.60). This could be justified by good recall and understanding of the nature and effect of these illnesses. Patient age was a significant factor in determining the comparison between self-reports and records data of these comorbids. Also, such comorbids are very significant for those patients and are followed up thoroughly by physicians due to their ongoing effects on patient health. This may also be true for the moderate agreement morbidities: kidney problems and foot ulcers. Kidney function testing was over-reported by 14% of patients and was of low kappa (0.28) and microalbuminurea was over-reported by 61% of them (k=0.06). Also, 19% of patients reported having an ECG test, but the results were not present in their records; this also applied to lipid profiling. Therefore, these findings indicate that those patients might lack information about these tests, or the results of these tests are not recorded in their records. These results are consistent with the Swiss study that showed low Cohen’s kappa (k ≤ 0.50) for foot examination, but high for blood pressure (k=0.94). Collet and colleagues justified this by the possible physician factors that affect the presence of reports in patients' medical records and lead to bias such as under-reporting . This may be a serious problem in poorly developed health systems, such as in Palestine where no electronic recording exists. In addition, we emphasize that physicians in PHC centers see a high number of patients daily, both in the UNRWA and MoH clinics. Such problems are of low priority in health systems where electronic records are in operation.
In this study, many patients may have more than one type of health insurance. For example, registered refugees have free UNRWA health insurance and public sector employees have governmental health insurance. This indicates that some patients in this study may have two types of insurance and can undergo examinations and consult physicians in two locations, leading to possible fragmentation in the follow up and treatment of these patients. Therefore, we asked interviewees about this possibility prior to interviewing them and verified if we had interviewed them previously. Examination reports may not be present in the patient file in the location where we collected the medical records data. This could apply to all tests and examination results in this study, leaving considerable potential for under-reporting of these examinations. As a result, health care providers should attempt to limit any duplication in the use of services and ensure that only one physician is responsible per patient in the treatment of diabetes.
Another important issue raised by data collectors in this survey is that medical records were found to be inaccurate and discrepant in several clinics. This by itself may limit the use of medical records as a source for any decision regarding treatment or patient monitoring.
This study provides a context for the measurement of the quality of care of patients with diabetes. Questionnaire based surveys are an importance tool in epidemiology and public health research. Although there may be a certain degree of bias in this quality of care assessment tool since some of these measurements may be misleading, it remains an inexpensive tool for such assessments.
In this study we used the concept of reproducibility, i.e., reliability and agreement, as suggested by de Vet et al. . However, there was an apparent discrepancy between agreement and the reliability measures of most tests and comorbids. The reasons for these discrepancies were discussed previously in detail [20,21].
In conclusion, an evaluation of the quality of care in Palestine should be made with caution when using self-report measures and when extracting information from medical records that have been collated manually. Our findings may raise the awareness of health care professionals of which selfreported measurements they may rely on. However, both sources of information are useful for defining tools to monitor decision making. Depending solely on medical records as a source of information for monitoring, as shown in this study, may not give decision makers the full picture of the treatment process for diabetes. Therefore, development of an electronic health information system is required to monitor the treatment process, in parallel with the systematic involvement of patients in the evaluation process.
We would like to thank all the patients who participated in the study. We also thank the study team, Mrs. Ibtisam Titi and Mrs. Adan Awaratni from the Ministry of Health and Mrs. Maha Nahal from the Faculty of Health professions of Al Quds University for their professional management of the field work. We would like to thank Mrs. Karen Mann for the English editing of this articles.
None of the authors have any conflict of interest to declare.
This work was supported by a grant from the Insulin Dependent Diabetes Trust (IDDT), Northampton, United Kingdom.
All Published work is licensed under a Creative Commons Attribution 4.0 International License
Copyright © 2018 All rights reserved. iMedPub LTD Last revised : December 16, 2018