Thank you for agreeing to give back to the pharmacy profession as a pharmacy residency accreditation practitioner surveyor! In this role, you will have the opportunity to help uphold standards for the profession and help residency programs improve - while learning from the programs you visit!
You can do this program in parts:
Note your current page number at the top of the screen when you need a break.
Then start at that screen number when you get back.
There are two components to practitioner surveyor training.
1. This part: In addition to background information, this part of the program introduces a simulation of what you will actually do as a practitioner surveyor. You will do pre-survey preparation for a simulated survey. You will review selected documentation for a site you are preparing to survey and determine how well each piece of documentation meets accreditation standards. You will prepare to meet with your lead surveyor at the live workshop at the Midyear Clinical Meeting. During your Lead Surveyor meeting, you will discuss your opinions about the quality of the documentation, as you would before an actual site survey, and compare it to that of the lead surveyor and other group members. (Note: There are many case documents provided via hyperlinks in this program. You do not need to bring the documents to the workshop - they will be provided in a workbook onsite.)
2. The second part of the practitioner surveyor training is the workshop: "Essential Skills for Pharmacy Residency Accreditation Practitioner Surveyors" which will be presented at the Midyear Clinical Meeting.
You will also receive the "Manual for Pharmacy Residency Accreditation Practitioner Surveyors, 5th edition" which includes all details in a "carry-along" format. It will be mailed to you and you do not need to bring it to the workshop.
1. Define accreditation, it's purposes and major issues concerning accreditation.
2. Describe the steps of the survey process, including pre-survey preparation, the site visit and post-site visit follow-up.
3. Explain the responsibilities and challenges of both the practitioner and lead residency surveyor and how to effectively fulfill your role.
4. Review and evaluate pre-survey materials in preparation for a simulated pre-survey meeting with a lead surveyor at the live workshop.
Most practitioner surveyors find the experience rewarding and it offers a number of benefits. These include the opportunity to network with other practitioners, learn ideas to take back to improve the quality of program of your own program, as well as sharing them at other sites as you make your visits. While giving back to the profession in this role, you will also become even more familiar with the standards.
The formal definition of accreditation, from the "ASHP Regulations on Accreditation of Pharmacy Residencies," is: "the act of granting approval to a postgraduate residency program after the program has met set requirements and has been reviewed and evaluated through an official process (document review, site survey, review and evaluation by the Commission on Credentialing). An approved program is in an "ASHP-accredited" status."
Accreditation protects the public by setting standards. Being accredited lets the public know that a program meets professional standards, thereby assuring quality. Accreditation allows a profession to engage in self-regulation. While it is a voluntary process, many organizations consider it essential to the integrity of their reputation. The accreditation process upholds standards and quality and may also encourage progressive practice when standards address such practices. In pharmacy, accreditation standards have been shown to have a positive impact on the practice of pharmacy as a whole. In addition, residency programs that are eligible for CMS Medicare pass-through dollars must be ASHP-accredited.
The purpose of the survey is to determine if, and how well, a program meets accreditation standards. Surveyors need to be thoroughly familiar with the ASHP accreditation standards and regulations, skillful at identifying what evidence to access to determine if the standards and regulations are being met and reviewing this evidence to make appropriate accreditation decisions. In addition, a review of the program on the survey against specific goals and objectives within the specific standards is performed.
Different surveyors play slightly different roles on a survey but all surveyors' primary task is to determine whether or not the program meets the standards and should the program be accredited. There are lead surveyors and practitioner (guest) surveyors. The lead surveyor is either an ASHP full time staff member or on contract with ASHP to conduct surveys.
Lead Surveyor Role
As their title suggests, they are the leader of the survey team. The lead surveyor's responsibilities include:
- Determining at what day and time the survey team should meet in the survey city prior to beginning the survey in order to discuss survey strategy
- Arranging for hotel accommodations for the survey team
- Making contact with the practitioner surveyor(s) to provide guidance for travel arrangements including when to arrive, anticipated departure time, hotel information, guidance on expenses.
- Paying expenses incurred when the survey team is together (e.g., hotel bill, meals, car rental)
- Being the final decision maker on recommendation for accreditation for voting by the ASHP Commission on Credentialing (COC). The lead surveyor provides the element of consistency on the survey team. They provide consistency in the process, conduct of the review, knowledge of the standards and regulations and keep the review in order. Teams may be as small as two individuals or may have many surveyors if a site has multiple PGY1 and PGY2 programs. A guest surveyor for each PGY2 area usually is involved with the surveys with multiple guest surveyors.
Practitioner (Guest) Surveyor Role
The practitioner surveyor is usually someone who is serving or has served on the COC, has completed a practitioner surveyor workshop, or has been a program director for a residency program for an extended period of time. The practitioner surveyor for post graduate year two (PGY2) residency program surveys generally fulfills the criteria listed for PGY1 reviewers but are specific to the type of program that they lead. Practitioner surveyors must be familiar with the standards and regulation and are often called upon to conduct sessions independently in multiprogram or multisite reviews.
Each surveyor makes their own travel arrangements. ASHP will be responsible financially for surveyor's airfare, ground transportation, lodging and food expenses during the time on survey. The lead surveyor will generally pay for meals on his/her charge card. You can also submit your expenses for meals when not with the lead surveyor. Per diem rate for food is $75 per day. Mileage, if car is used, is 0.565 cents per mile and is adjusted as customary rates change by the ASHP Finance Division. ASHP will not pay for movies, phone or internet use. You will be provided with a travel expense form to complete and must provide receipts for expenses. Further details are included in your manual for reference as needed.
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
The accreditation survey can be seen as having three main phases.
1. Presurvey Preparation
2. The Site Visit
3. Post-Survey Follow-Up
Each phase has its own tasks and challenges, which are described in this program. In addition, you will find documents for your review pertaining to a case that begins here and will continue at the live workshop. It is important that you review the case documents carefully before the December 8 workshop.
Usually about 30 days before a survey, the practitioner surveyor receives from ASHP all survey-related materials provided by the program, including:
- The program's accreditation application
- Completed pre-survey questionnaire with attachments
- Survey itinerary (may be an early draft version developed with the lead surveyor)
- Roster of program's preceptors
- If survey is for reaccreditation, materials from the previous survey and progress reports.
- ASHP travel expense form
- In addition, some surveyors may ask the site for access to their ResiTrak documents so that the surveyors can review these evaluation tools in advance of the survey.
It is important that you review these documents carefully.
It is important for surveyors to be thoroughly familiar with the following resources.
Accreditation standards and regulations
Click where it says "click here" for each listed below:
New PGY1 Residency Accreditation Standards were approved by the Board on September 19, 2014 (click here). For existing programs this revision of the accreditation standard takes effect July 1, 2016. Until that time the current standard, which was approved September 23, 2005, is in force.
Supplemental outcomes, goals and objectives
You will also need to be familiar with the supplemental outcomes, goals and objectives, if applicable, for the specific program you will be surveying. To see the outcomes, goals and objectives, click on one of the following types:
PGY1
PGY1 Community
PGY1 Managed Care
For PGY2, use this link to access the list: http://www.ashp.org/menu/Accreditation/ResidencyAccreditation.aspx (Click on "Regulations and Standards" and scroll down to "PGY2 Outcomes, Goals and Objectives.")
For the new PGY1 Residency Accreditation Standards, the new terminology is "Competency areas, goals, and objectives." Click here to see this document.
Pre-Survey Questionnaire
You should compare the site's completed "Pre-Survey Questionnaire" and "Self-Assessment Checklist" to the corresponding "ASHP Accreditation Standard" for either PGY1 (pharmacy, community, managed care pharmacy) or PGY2 programs.
To see the Pre-Survey Questionnaire and Self-Assessment Checklist, click on one of the following types:
PGY1
PGY2
PGY1 Community
PGY1 Managed Care
ASHP Guidelines
For all programs, you should compare their materials to the appropriate guidelines, including the following guidelines, which are often referred to during the survey process:
In addition, you should be familiar with the Best Practices for Hospital and Health-System Pharmacy book published by ASHP, which you can access via this link: http://www.ashp.org/bestpractices. The accreditation process is looking to see if "Best Practices" are followed at a training site.
It is also important to be familiar with the "Critical Factors" for PGY1, PGY1 Community and PGY2 programs you will be surveying. Click here to see the list of the critical factors. These factors are considered critical and are weighted more heavily in determining the length of accreditation for a program. You will be working with the lead surveyor upon completion of the report onsite and when the site's report comes back to review if areas of PC/NC (partial compliance/non compliance) have been resolved.
In preparation for the initial meeting of the survey team, based on a review of the program's background information, based on your review of the program's materials and your knowledge of the appropriate standards and best practices, formulate your own opinions to share with the survey team regarding areas that are, or are potentially NC (non-compliant) or PC (partially-compliant) as well as areas where more information needs to be gathered on site before a decision can be made. An initial assessment will assist to pinpoint areas for further investigation onsite to fully establish if an area is NC or PC.
The above preparation is done individually before arriving in the survey city. The survey team continues the preparation together once everyone arrives. Come to this team meeting ready to offer your opinions about areas that you think may be NC or PC or in what areas you think more information is needed to fully assess compliance with the standard. With your input, the lead surveyor will finalize a survey strategy for the site visit, which includes the schedule and areas the team has determined need special focus.
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
Case:
In preparation for the live surveyor training workshop on December 6 at the Midyear, please review the following presurvey documents. This is a simulation of an actual survey. Assume you are preparing for a survey and receive these documents for review prior to the survey.
At the workshop, you will meet with a lead surveyor to discuss concerns and what you would want to explore further in an onsite survey. Be prepared to discuss the materials, including:
1. What are strengths and weaknesses of each document?
2. How would you rate each document: NC, PC or FC? Be prepared to justify your opinion. (Note: Definitions of the ratings are discussed later in this program.)
3. What do you think should be explored further during the site visit? How should this be done?
4. What is your overall opinion of this program based on your document review?
To access the case materials, you may either:
- Click on the hyperlink for each of the presurvey documents listed separately below and/or
- Click here to access all the presurvey documents combined in one document.
Please review the case materials prior the workshop.
Stellar Hospital Materials
Set 1:
1. Presurvey questionnaire
2. Resident Academic and Professional Record
3. Purpose statement
Set 2:
4. Structure document
5. Teach/Evaluate grid
6. Schedule of residents
7. Learning experience description - Internal Medicine
8. Assessment strategy
Set 3:
9. Resident entering interests and self evaluation
10. Customized Plan
Set 4:
11. Snapshot side-by-side
12. Summative side-by-side
13. Orientation learning experience evaluation
14. Preceptor evaluation
Set 5:
15. Preceptor roster
16. Academic and Professional Record - RPD
17. Academic and Professional Record Preceptor
Set 6:
18. Strategic plan
19. Quality improvment plan
20. Organizational chart
21. Organizational chart pharmacy department
22. General organization data collection form
23. Joint Commission report 1
24. Joint Commission report 2
25. Affiliation agreement
26. Brochure
Set 7:
27. Ambulatory and Acute Care Grids
Site Visit Tasks
During the site visit, the team must accurately identify how well the site and program meet the standards. Methods for doing this during the visit include interviews, observation during a facilities tour and further document review. As you gather more information onsite, you will be validating information that was previously provided by the site as well as giving closer examination to areas where insufficient information was provided.
Schedule
The site visit plan may vary based on what the survey team has decided needs attention. However, most surveys roughly follow these steps:
Pre-Onsite survey:
Get together with survey team the night before for dinner to discuss strategy for the survey.
Survey:
1. Short courtesy call to the CEO followed by a review of documents with the pharmacy director, RPD and sometimes members of the administrative team, residency preceptors and residents.
2. Interactive program/paperwork review with members of the pharmacy administration team, residency program director and preceptors, if available. Residents are invited to all sessions, except the preceptor session and when the surveyors write up the report.
3. Interviews with the Director of Nursing or designee and one or two line supervisory nurses, the Chairman of the Pharmacy and Therapeutics Committee, one or two members of the medical staff with whom the residents interact routinely, residency preceptors, staff pharmacists, technicians and the residents.
4. Tour of the pharmacy department, emergency room and other areas to enable the surveyors to understand the residency program and pharmacy service, interviews with representatives of the residency preceptors, pharmacist line supervisors and residents.
5. Private conference of the surveyors to discuss and verify areas of non- and partial-compliance, as well as consultative recommendations.
6. Review of findings with the Director of Pharmacy, RPD and other invited preceptors and residents.
7. Exit interview with the CEO and other health-system administrators.
Interviews
When conducting interviews, ask open-ended questions and listen attentively. The individuals being interviewed should do most of the talking. However, be sure to invite questions also. It is important to stick with the schedule and stay on time. Your manual lists questions that you may find helpful as you conduct interviews on surveys. During interviews, remember your purpose: to determine compliance of the program to the standard. Emphasize what is required by the standards rather than specific methods of meeting them. If you are asked how you meet a requirement, preface your comment by stating this is how you do it but it is just one way, not THE way. When asked for your opinions, be sure you state them as opinions rather than requirements. Avoid coming across as superior to the program being surveyed by talking excessively about "how we do it." Your lead surveyor will help to keep you on track if your are with them.
Facilities Tour
The faculties tour provides the opportunity for direct observations of current practices, patient records, documentation of pharmacist activities and talking with staff in the work environment. Opportunities to observe safety parameters (e.g., including medication storage, medication administration record reconciliation, use of technology) occur while on tour.
Onsite Document Review
Although much document review takes place before the survey, there will still be more at the site, such as the residency plan of goals/objectives, structure, learning activities, evaluation tools, customized plans and tracking of resident performance, as well as policies and procedures, P&T minutes and planning documents.
Survey Tone
The tone of the survey team should blend collegiality with professionalism. Care should be taken to make the site staff as comfortable as possible, avoiding an intimidating atmosphere. Keep in mind that you are a guest of the organization being surveyed. Use of cell phones and checking of email should be limited to breaks. It is important that the team convey that they are in accord during the visit. If there are areas of disagreement, they should be discussed privately. Consensus is sought but the lead surveyor makes the final judgment. An effort to maintain efficiency and timeliness during the survey is essential.
Giving Feedback
There are many opportunities to provide feedback during the site visit. Keep it constructive by following these guidelines:
Keep the focus on what the program can do to improve, rather than on what they are doing wrong, stressing that the survey process is about performance improvement.
Focus on the requirement being discussed and be specific in your comments.
Take unique program situations into consideration when comparing them to the standard.
Offer to put them in touch with other programs to share information when appropriate.
While it is OK to describe how other programs have met parts of the standards, avoid dwelling on "how I do it".
Conflict
While a collegial and professional survey minimizes and avoids most conflict, confrontational situations sometimes arise on surveys. Use these guidelines to diffuse conflicts quickly:
Again, the lead surveyor is in charge of the review. Make sure and identify even before you agree to come on the review that you do not have a conflict of interest in participating in the survey (see the manual section on conflicts of interest).
The Survey Report and Ratings
The survey report includes ratings of program compliance with each requirement on a checklist that is arranged like the pre-survey self-assessment questionnaire and a summary that includes areas of noncompliance and partial compliance and consultative recommendations.
Each of the standard's requirements are rated using this scale:
Fully compliant (FC): 100% implementation of the requirement
Partially compliant (PC): 1% - 99% implementation of the requirement (Note: "Critical Factors (CF)" have additional rating requirements - see below)
Noncompliant (NC): Zero implementation of the requirement
Not applicable (NA): Does not apply (e.g., standard requirement deals wish multi sites, but the program only uses one site)
Ratings must be justified by appropriate evidence and consistent among surveyors. Consistency criteria (page 85 of your manual) guide these decisions.
In addition, Critical Factors are weighted more than other areas toward the accreditation decision. You need to be thoroughly familiar with the Critical Factors. For a list of the critical factors, click here.
Critical Factors that are found to be partially compliant are rated as low, medium or high PC, based on surveyor judgement using these definitions:
Low PC (L-PC): About 1-33% compliant
Medium PC (M-PC): About 34-65% compliant
High PC (H-PC): About 66%-99% compliant
Consultative Recommendations
Consultative recommendations are not required but suggested. Practitioner surveyors are expected to write consultative recommendations. These are recommendations that programs "should consider doing" or are not in the standard that may be helpful for the residency program to have and to have their administration hear. Here is one example: "Consider forming a Residency Advisory Committee to support a team approach to the residency program." When possible, consider prefacing the recommendation with a positive comment.
Here are some typical categories of consultative recommendations:
It is useful to be aware of parts of the standard that are frequently cited. The most recent frequent findings for PGY1 and PGY2 programs are listed below.
Recent Top Areas of Partial Compliance with PGY1 Pharmacy Residencies (n=84)
Residency Program Issues:
82% Preceptors have not adequately developed and documented descriptions of learning experiences - CF
80% Not all preceptors have made adequate contributions to the total body of pharmacy knowledge and meet 4/7 preceptor criteria- CF
73% RPD does not have a plan for improving the quality of preceptors' instruction
64% Residents do not complete evaluations of preceptors and learning experiences (at least quarterly for longitudinal) - CF
64% Preceptors do not complete all aspects of the assessment plan (e.g. self-evaluation, summative plans at least quarterly for longitudinal) - CF
Service Issues at the site:
52% Services are not of a scope and quality commensurate with identified patient needs - CF
52% Pharmacists do not prospectively help develop individualized patient treatment plans - CF
37% Pharmacists do not adequately design and implement medication therapy monitoring plans for patients - CF
37% Automated medication use systems and software do not support a safe medication use system - CF
Recent Top Areas of Partial Compliance with PGY2 Pharmacy Residencies (n=43)
Residency Program Issues:
88% Preceptors do not ensure all aspects of the assessment plan are completed (e.g. self-evaluation) - CF
81% Preceptors have not adequately developed and documented descriptions of learning experiences - CF
58% RPD does not have a plan for improving the quality of preceptors' instruction
56% Customized plans for residents are not documented/implemented/updates adequately - CF
56% Residents complete all evaluations of preceptors performance (at least quarterly for longitudinal) - CF
56% Not all preceptors have made adequate contributions to the total body of pharmacy knowledge and meet 4/7 preceptor criteria) - CF
Service Issues at the site:
44% Services are not of a scope and quality commensurate with identified patient needs - CF
39% Pharmacists do not prospectively develop individualized patient treatment plans - CF
35% Professional and technical staff is insufficient to provide level of services required of all patients - CF
35% Automated systems do not support a safe and effective medication use system. - CF
32% Pharmacists do not adequately design and implement medication therapy monitoring plans for patients - CF
A draft of the survey report is completed and orally presented to the program in the exit interview, which is conducted by the lead surveyor. Sometimes, the lead surveyor asks the practitioner surveyor to kick off the exit interview with positive comments about the program. The lead surveyor presents the findings and sometimes asks the practitioner surveyor to present the consultative recommendations. The lead surveyor explains next steps, including when they will receive the final report, when and how they are to respond to the report, when the COC will act on their accreditation and when they will know the result. Survey teams can, optionally, give "Preliminary Accreditation." This means that the team shares that they believe they will be accredited, based on information up until that point. However, if the Commission on Credentialing or ASHP Board of Directors disagrees, the accreditation decision can be changed so this should only be given if the team is absolutely sure it is a strong program that will be accredited. Before the end of the exit interview, the program has the opportunity to ask questions and, in some cases, present more information if they disagree with a finding.
The last part of the survey is the exit interview with the CEO and other members of hospital administration). This interview is more high level, without the details presented to the pharmacy staff. Findings related to both direct patient care and distributive services, as opposed to program are usually the focus of this interview.
In addition, preceptor needs are frequently discussed. Detail related to evaluations, construct of the residency program (e.g., Principle 4) are not generally presented in detail at the exit interview with the health-system administrators.
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
The final survey report must be ready to be mailed to the program within 30 days of the survey. In this 30 days, the report must also go through an editing process in ASD. While different lead surveyors use different methods to complete the report, a draft is often developed during the site visit and agreed to by the team. After ASD editing, the report is sent to the site, which has 45 days to send its response. Therefore, the site has approximately 75 days from the onsite survey.
During the 45-day period that the site has to develop their response to the survey report, they may contact any member of the survey team,
usually contacting the lead surveyor. They may seek clarification of a finding or help in how to respond. This must happen within 75 days of the
survey and not after that to ensure equity for all programs. The response is sent to ASD and is forwarded to the lead surveyor and you for review.
After reviewing the response, the lead surveyor completes the "Confidential Page" on which the rating and accreditation recommendations are
recorded. These are forwarded to the survey team for consideration and the lead surveyor leads the consensus building process.
Details on ratings and accreditation recommendations can be found in your manual.
The final confidential report is sent to ASD where it is distributed to the full COC for consideration at its next meeting, providing a basis for more extensive discussion at that time.
COC meets twice a year and reviews the results of each survey. The COC's recommendations next go to the ASHP Board of Directors (BOD). Accreditation becomes official when the BOD approves the COC's recommendations and the sites are notified. ASD then forwards the decisions to the site upon ASHP BOD approval. This can be as short as 4-5 months from the time of the onsite survey, to as long as 9-10 months from the review, depending on when the survey falls within the survey cycle. Surveys conducted from December 1 - May 31st are reviewed at the August COC meeting and September ASHP BOD meeting. Surveys conducted from June 1 to November 30th are reviewed at the March COC meeting and are approved at the April ASHP BOD meeting.
At the December 8 workshop, you will see the survey report, confidential page and response to the survey report for the case and will have the opportunity to discuss them.
What most commonly challenges the surveyor's task of comparing a residency program to the standards?
1. Sometimes it is difficult to disregard the reputation of a program and just look at the evidence. It is important to remain objective. Suppose the program has a stellar reputation or maybe you consider the program director to be a colleague or a nationally recognized practice leader. If you see a discrepancy between the standard and some aspect of the program, it is incombant upon you to document this finding. Do not use "consultative" recommendations (recommended but not required) if there is a deviation from the standard. It is important to maintain consistency and not do favors in this way at one site but cite the same item as a NC or PC at another.
2. "Politics" sometimes challenge surveyors. Let's say you've had "issues" or have some other type of "baggage" with a program or staff member(s) in the program. Just remember: "Leave home without it (the baggage)!" Remain objective and adhere strictly to the task of comparing the program to the standards.
3. It may seem difficult or unfair to cite a program if/when you know that your own program does not meet the standard in the same area in question. Again, remain objective. It is your job as a surveyor to cite the program even if your own program is remiss in the same area.
4. Surveyors are often tempted to tell about what they do in their own program. However, you should refrain from talking too much about what you do in your program. Unless your hosts ask for details, keep it brief. Talking too much about your program can come across as self-righteous and be off-putting to the host program. It is better to say you have seen a particular practice than to say "at my place we do xyz."
Remember that your role is to determine if a program meets standards or not. If not, it is up the program to determine how to remedy the situation. This does not mean you can't make suggestions. However, when you make suggestions, make sure you are clear in you comments what is required and what is just a suggestion.
As a surveyor, you need to be aware of the ASHP Commission on Credentialing Conflict of Interest Policy: External Professional Business Activities and the ASHP Commission on Credentialing Policy on Disclosure of Outside Interests, which are in your training manual (pages 15-19). At the workshop at the Midyear you will be asked to sign a Confidentiality and Nondisclosure Statement and Disclosure Report.
As examples, you should not review a program if you recently worked there , did your residency there, the RPD is a former resident, etc. You may be a colleague, especially in the PGY2 areas that are "small world" but you must still maintain objectivity.
Show/hide comprehension question...
Show/hide comprehension question...
Show/hide comprehension question...
To summarize, keep these key points in mind:
- The main aim is improving quality of training
- Evaluate against standard requirements
- Be familiar with best practices
- Be objective, avoid politics or influence of reputation
- Do not cite just because you were cited
- Do not be biased by site or leader reputation
- Consultative recommendation not acceptable substitute for true NC or PC
Thanks again for your contribution as a practitioner surveyor!
We hope you benefit as much from the survey experience as the programs that you survey!