In this first blog post, I have chosen a topic for our discussion that is controversial not only in Alberta but across Canada. This is the decision by Canadian nurse regulators to negotiate the development of a new RN entry-to-practice exam with the National Council of State Boards of Nursing (NCSBN) based in Chicago. Since the decision was announced, I have responded to numerous e-mails and telephone calls from concerned members and students.
To begin, you may wonder why a change in the exam was even considered in the first place. Currently, RN candidates write the Canadian Registered Nurse Exam (CRNE), a paper and pen exam offered through the Canadian Nurses Association (CNA). This exam is a valid assessment tool for entry-level competencies. A computer-adaptive exam format is considered state-of-the art and best practice. Moving to a computer-adaptive exam will, among other things, be more accessible to candidates, continue to be based on Canadian content, be offered in English and French, be affordable and will reflect our high Canadian nursing standards.
The CEOs/EDs of the regulatory bodies for registered nursing in Canada chose to issue a global request for proposal (RFP) for a vendor to develop and administer the new computer exam. The RFP included several requirements including:
- ensuring a Canadian context exam
- involvement of Canadian RN experts in exam development and approval
- an exam offered in both English and French;
- compliance with Canada’s national and provincial/territorial privacy legislation
The NCSBN was selected as the successful vendor. Next steps in the process include moving into negotiations on an acceptable process for exam development and administration.
So what does this mean for Alberta? Alberta legislation is very clear. Provincial Council makes the final decisions on regulatory matters and approves the entry-to-practice exam for Alberta. We take this role very seriously. Council must articulate and approve the essential criteria that NCSBN must meet for development and administration of the exam. NCSBN must demonstrate to Council how they will fulfill their commitment to address all requirements identified by Council and in the RFP.
To be specific, the RFP was for development of a new exam; it will not be acceptable to simply adopt the American NCLEX-RN exam. In addition, there are significant privacy concerns to be addressed particularly given the breadth of the U.S. Patriot Act. If all criteria are met, the contract will be signed to move the development processes forward.
Our goal is a new Canadian exam that meets very high Canadian standards for entry-to-practice nursing knowledge and practice. Some concerns have been raised regarding the impact of this decision on CARNA’s future relationship with CNA. As your president, I can affirm that CARNA will continue to strongly support the work of CNA as the national and respected voice of our profession.
Now I want to hear from you. Many of you have expressed your thoughts to me already and I thank you. However, I urge you to also join this community conversation. What do you think are the most important criteria for an entry-to-practice exam? Decisions will be made at the January 2012 Provincial Council meeting. We will also be holding information sessions in Edmonton, Calgary, and Lethbridge to start.
How do you want to be kept informed on this issue as your Council proceeds?
While I agree with the reasons to move to an on-line exam & why NCSBN was selected, I think their expertise in developing and maintaining an electronic exam be utitlized (wheels haven’t changed). I’d like to see the actual exam continue to be developed and maintained by a Canadian organization. That way, as I understand it, the US Patriotic Act and Canadian Privacy concerns would be alleviated. In short, we could be renting their experience, but using it here in Canada.
I was relieved to see that the NCLEX would not be the model for the Canadian exam- having written both I strongly believe that the NCLEX is a poor model for content and process. I received a license to be a nurse in the US having answered a total of 75 multiple choice questions on a computer. I don’t feel this is adequate to ensure competent entry to practice for nursing and was alarmed (and am still cautiously observant) that an American company is being entrusted with this contract.
I agree with Karen M and KLS’s comments above. The decision to issue a global request is what concerns me. This implies that Canadian resources/ people are inadequate to provide this service. It also implies that the decision was budgetary in nature. In my opinion it is disrespectful of our people and our country. You have identified why the US firm was chosen but I would like to hear why Canadian resources were not chosen for this task. Canadian nurses have worked hard for many years to create a Canadian image and to be independent. Giving this task to any other country seems to go against everything Canadians have worked for.
I think it is important to update the way that exams are written – to optimize the efficiency and security of the process and work in a medium that our new nurses are familiar with. However,I do not think it is wise to go outside Canada in order to modernize our exam writing system. It opens the door to all kinds of issues – related to understanding our culture of health care, our values and the way that health professionals make decisions in Canada. By keeping the RN exam development and implementation in Canada we will maintain a higher standard and get a better end product in the long run.
I would like to know if there was any discussion with CNA to develop an online version of the our exam prior to making the decision to move to the American exam?? It seems to me that there is minimal transition required once a pool of exam questions is available to change the delivery system.
I believe it is imperative Alberta nurses use their voices to advocate for Canadian only content in a Canadian nursing exam. I am entrusting CARNA will listen and be the vehicle to verbalize our collective voices.
I agree with the first 4 comments above to move to an online exam and to keep the content Canadian while maintaining our high standards. I am personally offended Canadian experts have not been chosen to develop an exam similar to the American format. If CNA wants to adopt the American style exam and that is what it should be AMERICAN STYLE ONLY then what is wrong with contracting a CANADIAN company to follow that style. KEEP IT CANADIAN with content, standards, development and implementation.
Thank you LP for your response. In your comments you mention “If CNA wants to adopt an American style exam”. This is inaccurate. CNA has not made the decision to select the National State Boards of Nursing for the new exam development. This decision was made by the CEOs/EDs from 10 Colleges of Nursing through a request for proposal (RFP) process. These Colleges are entrusted with the responsibility to regulate registered nurses in the best interests of the public and to finalize the exam for their jurisdictions or province. CARNA is one of these Colleges and our Provincial Council, under Alberta legislation, has the responsibility to identify the essential requirements for exam development and to approve the final exam for Alberta.
I just watched the webinar regarding the Entry to Practice exam and it seems that a north American exam is a reality. I am wondering why we got a postcard in a recent journal requesting that we turn down this proposal by mailing the card or am I mistaken.
Thank you for your response
Thank you Maria for your question.
First of all I would like to clarify that a North American exam is not yet a reality as we are just in negotiations with NCSBN on a contract for development and administration of a new exam. Provincial Council has not signed a contract. We have certain requirements for the negotiations and if these are not addressed in the proposed contract then the negotiations will need to resume. Yes, CNA included a postcard in the most recent issue of Canadian Nurse. It is completely your choice to send this postcard however; you should be advised that the decisions related to the exam fall within the mandate of the provincial jurisdictions (i.e. CARNA) and legislation in each province.
Thanks for your questions Ming. There is a big difference between an online test and what we are proposing, which is a computer adaptive exam (CAT). CAT uses computer technology, measurement theory, algorithms and item response theory to increase the reliability and security of the exam process. It is a complicated process, and even with a large pool of exam questions, it is not a simple transition from the current pen and paper exam to the CAT. I want to assure you that that while an American organization with CAT experience was selected to develop the new exam; the agreement stipulates that it will meet Canadian standards and our essential requirements.
Diane, you have not answered my question. I am aware of the process regarding transition from paper to Computer based. My question was… Did you discuss your requirement for a computerized exam with CNA so that they could provide a possible solution to your requirement. It seems to me that converting to a computerized format would have been much less costly that the process you have identified with legal reviews, advisory committees to review content etc. I fear that the end costs to registered nurses will be considerable higher than the fees we now pay.
Hello Ming,
It is unfortunate that my first response did not address your question. There have been many discussions over the past 3 years between the CEOs/EDs that represent the regulators and CNA on these issues. The discussions focused on recommending a computer-based, not a computer-adaptive exam. The CEOs/EDs preferred to move to the state-of-the-art computer-adaptive exam. Unfortunately a consensus could not be reached between the two groups. As a result, the CEOs/EDs proceeded to issue an RFP in September 2011 and NCSBN submitted the proposal which the CEOs/EDs felt best met the requirements outlined in the RFP. As far as costs go, the new exam may cost less for regulators to purchase than the current entry-to-practice exam however; this is part of the negotiations currently underway. The requirements identified by Provincial Council are designed to advise the negotiations with the NCSBN.
I am in support in moving the exam into the computer age, but NOT having it administed by an Amercian Agency for a number of reasons.
1. Privacy – Although President Dyer assures us that information would be protected I’m not confident. How is this different than what we have as requirements for maintaining patient records, whereby provincial governments don’t permit servers to be located or managed by American companies because of the strong reaches of the Patriot Act.
2. Loss of Canadian Identity – Don’t fix it if it’s not broken. Although the CRNE exam may not be computer based it has provided a good standard for licensure for Canadian RN’s for many years.
3. Trust – Why not work with the CNA and the existing agency to develop a computer generated exam instead of farming it out to the US? What is it that has sparked regulatory bodies to seek outside Canadian involvement? I have to wonder about regulatory bodies being transparent with their registrants and the true reason for the change.
Hi SM, thanks for taking the time to respond to my post. I will try to answer your questions as best I can.
1. I hear you on the privacy concerns, and this was a major point of discussion with the CARNA Council. We have sought the advice of legal counsel, and have been advised that despite the fact that NSCBN does have a good track record of attention to privacy issues, we have added the following expectation to the negotiation process:
“Canadian experts will be used to review and validate that the proposed approach to managing private personal information is compliant with legislative
requirements and has minimized the risk of the US Patriot Act to the greatest and most reasonable extent possible.” Although we are confident that the vendor will be able to meet our expectations around this issue, the ultimate outcome is that if CARNA Council is not satisfied that the confidentiality and privacy of the information of Canadian exam writers is maintained, the negotiations will not reach a successful conclusion.
2. The current exam, the CRNE, continues to be a valid and reliable assessment of entry-level competencies. The proposed exam offers features that will make it more affordable, flexible and accessible year-round, instead of being offered only three times a year. Computer adaptive testing has also been shown to be a more precise measure of the writer’s knowledge and competence. Another advantage is that test results will be available to regulators within 48 hours leading to more timely registration of successful candidates.
3. I want to tell you that CARNA Council strongly supports the Canadian Nurses Association as our national voice of the profession, and we maintain a relationship that is mutually respectful and passionately focused on the future of the profession. As for the RFP process to select an exam vendor, I can tell you that the process was rigorous and global in reach. The EDs and CEOs of the governing Councils chose the vendor that they believed would best meet the requirements for Canadian nurse exam writers. Your Provincial Council is committed to ensuring that the new RN entry-to-practice exam will meet all of the requirements of the RFP plus additional, very specific criteria that we have identified. Although we are confident that the selected vendor can meet our stringent requirements, if for some reason our requirements are not able to be met, negotiations with NSCBN will not reach a successful conclusion. Thank you for your thoughtful questions!
According to the CNA website they put a RFP out in 2010 and was planning to deliver it. “Before this business plan was presented to the CNA board in March 2011, the 10 RN regulators communicated their intention to CNA to issue a request for information for development of a CAT exam. Because of this new development, which was at odds with the direction of the CNA board, a CNA board subcommittee was created to review the business plan; the subcommittee brought its findings and recommendations to the June 2011 board meeting. CNA then reaffirmed its desire to continue to work with the 10 RN regulators toward a solution for a computer-based exam. However, the 10 RN regulators chose to issue an RFP in September 2011 for the development and delivery of a CAT exam for entry to the profession. Assessment Strategies Inc. (ASI), a wholly owned subsidiary of CNA, responded to the RFP to develop a CAT RN entry exam.”(http://www.cna-aiic.ca/en/becoming-an-rn/rn-exam/history/)
My question is, why did the 10 RN regulator bodies decide to make an RFP on their own and not work with CNA who was already working on plans to develop a computer-based exam? Why couldn’t the 10 RN regulators see what ASI was going to create?
Thank you for this very good question Indrah. There were many discussions over the past 3 years between the CEOs/EDs that represent the regulators (i.e. exam buyers) and CNA on the entry-to-practice exam. The discussions focused on options for both the development of a computer-based exam and a computer-adaptive exam (CAT). The CEOs/EDs preferred moving towards a CAT. Unfortunately consensus could not be reached between the two groups. As a result, the CEOs/EDs proceeded to issue an RFP for a vendor that would create a CAT and meet the requirements identified by the CEOs/EDs in the RFP. ASI led a consortium that responded to the RFP with their proposal for CAT development; however, after a rigorous review of the submissions, the CEOs/EDs’ selection committee chose the NCSBN as the best option and the successful vendor.
Thank you Dianne for answering my question. I really enjoyed meeting with you today. A thought came to me about why CARNA would submit an RFP but reading your response you are saying here that CEOs/EDs submitted it. Who are the CEOs/EDs? I always thought it was CARNA who submitted the RFP and if that was the case I was going to ask, why would CARNA submit an RFP without first consulting legal counsel to find out that you would become legally bond to move forward. Maybe I misunderstood you. Is this what you said?
I have written both exams and preferred the computer exam. I believe the randomized 75 – 200 questions provided a challenging exam based on hands on nursing care. I completed my exam in 75 questions but many people had to go further. The computer is set to evaluate your competency based on your answers. I like an exam that is clear cut and not subjective. I only hope that the content for hands-on nursing care questions are based on Canadian practices.
Hi Patch, I appreciate your thoughts. The proposal by NCSBN stipulates that the development of the exam will include the following:
– a review of Canadian entry to practice competencies and other regulatory documents
- a practice analysis of Canadian RN practice
- a review of existing exam bank questions by Canadian and American RN subject matter experts for cultural sensitivity with veto over any items that demonstrate an inappropriate bias
– involvement of Canadian registered nurses in new exam question development.
As an additional measure, Provincial Council carried a motion at its meeting last week “that the negotiated contract with NCSBN shall reflect specific requirements.” These requirements include an exam that reflects the Canadian Code of Ethics, and confirms that the Canadian nursing subject matter experts will be selected and approved by CARNA Council. Please see the upcoming issue of Provincial Council Highlights will include the complete list of all the criteria included in the motion. Thanks again for your questions!
While I am not opposed the implementation of a computer based exam, I, like many others, am opposed to an American outsourcing. There has been some discussion and information released recently from the CNA that NCSBN fully intends to “bring the existing American exam to Canada”. Is this indeed the truth? I am extremely opposed to this. Many years ago I, too, wrote the NCLEX and also did not find that it evaluated the knowledge I had gained through my education in Canada. I would be very disappointed in my profession if this is the direction that is taken.
Hi HW,
CARNA and the other nine regulators would have been very pleased to have awarded the contract for the new exam to a Canadian vendor. The goal was to secure a vendor that would support development of the best possible computer-adaptive exam for future RN candidates. The request for proposals (RFP) was open world-wide and NCSBN submitted the proposal which best met the requirements defined in the RFP. The content of the new exam will be developed with direct involvement of Canadian RN subject matter experts selected by the Canadian provincial RN regulatory bodies. These experts (including both English-speaking and Canadian francophone RNs) will review all existing exam questions, develop new questions, and form expert panels to review the exam bank for cultural sensitivity and for accurate and appropriate French translation of items. Provisions to veto content or items that are biased to either Canadian or American writers must be present. The purpose of the entry-to-practice exam is to evaluate the candidate’s knowledge and decision-making skills in order to ensure the provincial regulatory bodies that the candidate can practice safely and competently as an entry-level RN. The new exam will be based on the Canadian entry to practice competencies. The Alberta educational curriculum must reflect the approved Alberta entry to practice competencies and therefore will educate students in preparation for the new exam. I hope that this information is helpful. Provincial Council will continue to keep members informed throughout the exam development process. We are also committed to ensuring that Alberta has an entry-to-practice exam that meets both the requirements of the RFP and the specific requirements established by Council.
Thank you Diane for the opportunity to comment on this important issue.
You asked “What do you think are the most important criteria for an entry-to-practice exam?”
To answer that I refer to the CNA Declaration of seven principles that are essential to any negotiation in regard to the development of a CAT Canadian RN entry examination. The declaration is on the CNA website so I will not repeat those comments here. However in my view, these principles must be met as part of this process.
I appreciate your assurances that the exam will meet all Canadian requirements and will be developed by Canadian Nurses, but am puzzled by the fact the the NCSBN continues to share the following information on its website (from its media release of December 8):
“The NCLEX-RN Examination is offered in 10 countries around the world for the purpose of domestic licensure in the U.S. The selection of NCLEX by Canadian nursing regulators marks the first time that the exam will be used for the purpose of licensure in another country. Each Canadian province (excluding Quebec) will individually make the decision to contract with NCSBN and the NCLEX-RN Examination will be offered at Pearson Professional Centers throughout Canadian provinces that have contracted with NCSBN for its delivery.”
This statement, which is one part of the entire press release makes me wonder if the NCSBN truly intends to develop an fully Canadian exam that meets all the criteria that you have indicated.
Thank you Susan for your response to this important discussion.
I can appreciate how the release on NCSBN’s website might be confusing to you and to other Canadian registered nurses. CARNA and the other 9 nursing regulators have addressed some concerns with the NCSBN regarding this release and the messaging and have agreed to work together with NCSBN on future releases related to the exam.
As regulators we are committed to the development of an entry-level RN exam that measures the competencies needed to practice as a safe newly licensed, entry-level nurse in Canada. The RFP issued by our Canadian nurse regulators was for the development of a new entry-level exam based on the Canadian entry to practice competencies. The NCSBN was selected due to their expertise in provision of state-of –the-art computer-adaptive exams and is expected to meet the criteria set out by the regulatory bodies (i.e., to provide a legally defensible, credible and accessible exam that will reliably test Canadian entry-level nursing competencies and assess candidates readiness for practice in Canada).
I want to stress that Canadian RN applicants will not be writing the current version of the NCLEX. There is a great deal of work ahead to design a new exam that addresses all of the RFP and Provincial Council’s requirements.
Within the last ten years, I have written both the CRNE and the US-based NCLEX licensing exams. I have always believed the NCLEX was a better exam – not just because of its state-of-the-art computerized format, but also when considering content. The NCLEX required much more preparation, and tested on many important concepts of pathophysiology and pharmacotherapy, knowledge I feel is essential in providing competent nursing care, no matter what country you live in.
Thank you Devon for your response. Your perspective is helpful to our understanding. Your comments on content are important as we are striving for a new exam for Canadian writers that will test both the knowledge of the writer and the critical thinking skills and judgments required to practice as a competent and safe new graduate RN. As discussed previously, the new entry to practice licensure exam for Canadian registered nurses will be a computer adaptive exam with content based on the Canadian entry to practice competencies, a national nursing practice review and the Canadian context (e.g. the metric system, Canadian medications etc.). Canadian RN content experts will be selected by the jurisdictional regulatory bodies to participate in the exam development process by reviewing questions within the existing bank and by developing new questions. These requirements were included in the RFP and by your Provincial Council for achievement of a successful negotiation process.
I have many questions and concerns that have mostly been addressed so far in this discussion. However, I would like to ask one more question.
I am a nursing student, and I have had a lot of discussions about this topic with fellow students. A common concern for us is how exactly will the computer-based exam be written?
A lot of us like to be able to write all over our exams, make notes, and be able to flip back and forth between questions and have a chance to review our answers. Our fear is that the computer-based exam might only let us view a question once, and once you have selected an answer, you cannot go back and re-read the question, which seems very unfair to a lot of students that I have talked with. Does anyone have any comments about this?
It usually does take practice to become familiar with writing an exam in a new format. Before launching the new RN entry exam in 2015, CARNA and the other regulatory bodies will ensure that nurse educators and students have the opportunity to learn about the new exam well in advance of its introduction. In addition, NCSBN will provide a series of online tools and resources, including pre-tests to help you gauge how prepared you are for the actual exam and to become familiar with the exam format. The questions on these pre-tests will relate to the same set of competencies used to develop the actual exam questions. Candidates writing the new exam will also be able to take a short tutorial at the beginning of the exam to familiarize themselves with the formats for the exam items.
During the WNRCASN conference this past February in Lethbridge, there were many nurse educators and deans that were unaware of the changes being made to the CRNE. Why were nurse educators not consulted regarding the change in the exam?
Noelle, thank you for this important question.
As you may know, CARNA is a self-regulating profession. RN members are elected to Provincial Council or participate as experts on regulatory committees such as the Registration Committee and the Competence Committee. There are times however, when members are not included as participants in a decision or activity due to the nature of the decision and/or the legislated mandate. In the case of the entry to practice exam, Provincial Council under Alberta legislation has the full responsibility and is accountable as the regulator for decisions related to the negotiations of the contract for exam development, as well as the ultimate approval of the examination. Council has set out clear requirements for the negotiations and is taking all reasonable steps to ensure that we make the right decision for the profession and the protection of the public. For example, we are ensuring that the new entry-to-practice exam reflects the Canadian Codes of Ethics (Ontario has its own code of ethics, all other jurisdictions have adopted the CNA Code of Ethics) as well as the Canadian entry-to-practice competencies. Our main goal is an exam that protects the integrity of the entry-to-practice credential in Alberta, and assists in assuring the public that our new nursing graduates are entering the profession as safe, competent and ethical practitioners.
Once the negotiations are completed and the outcome has been determined, Canadian RN experts in practice and education (selected by CARNA Council) will actively participate in the exam’s development prior to its introduction in 2015. Further, educators, students, and all registered nurses in Alberta will have opportunities to learn more about the new RN entry-to-practice exam. It is my sincere hope that the exam information sessions, FAQs and the webinar have been helpful, and have provided the information required to better understand the decisions to date and next steps. CARNA Council is committed to two-way communication on various emerging issues, and I hope we can count on your participation in those efforts in the future.