When Katherine Matthews, CAE, started working at the National Council of Architectural Registration Boards, the association had already made strides to become a data-driven organization.

“NCARB was already positioned as a thought leader in the industry,” said Matthews, who is currently NCARB’s assistant vice president of data and analytics. “But we wanted to get into conversation with our data and determine that 2.0 conversation.”

One way that NCARB used its data was to examine issues surrounding diversity, equity, inclusion, and accessibility (DEIA). By combining the data with demographics, NCARB found that candidates from under-represented groups had lower pass rates on the Architect Registration Examination (ARE) than white male candidates.

“We identified the places where the path to licensure seemed different for different groups,” Matthews said. “We use the data to inform how we manage our programs so that all candidates could get their licenses equitably.”

Digging Deeper in the Data

To start, NCARB segmented and analyzed pass rates from more than 32,700 test takers by demographic information, including race, ethnicity, gender, and age.

“NCARB has the dual mandate of protecting the public while facilitating licensure, so it’s critical to us that the rigor involved in getting licensed is backed up by data and not disproportionately affecting any group,” Matthews said.

The association found that white candidates were much more likely to pass than candidates of color and that men were more likely to pass than women. A similar trend was found when factoring race and ethnicity, although Black or African American women generally outperformed Black or African American men.

Finding Solutions

Though the demographic data revealed patterns, the patterns alone didn’t explain why certain groups were performing better than others. According to Matthews, NCARB began brainstorming what necessary information might be missing.

One area NCARB had little information on was language: Could language barriers be a contributing factor to exam performance?

“We knew from anecdotal conversations that some candidates were interested in having the exam available in other languages,” Matthews said. “We didn’t have a lot of data on this issue; we didn’t know what languages candidates spoke or were fluent in.”

In 2021, NCARB hosted licensure candidate focus groups and rolled out an Architecture Licensing Feedback survey, where the community could offer ideas for how NCARB could address disparities along the path to licensure.

Part of the survey focused on language to help NCARB determine whether candidates who speak English as a second language (ESL) had more difficultly on the exam.

“As we correlated the survey with the exam data, we found that ESL candidates took longer on the exam and omitted more questions,” Matthews said.

The findings led NCARB to develop two accommodations for ESL candidates: additional testing time and a word-to-word translation dictionary for candidates testing at a PSI Test Center.

Moving the Needle on DEIA Goals

According to Matthews, most of the architecture licensing boards in the U.S. easily adopted NCARB’s proposed accommodations, but for the state boards that were hesitant or had legislative hurdles, NCARB worked to circulate its research and partnered with peer organizations to get the message out.

“For example, the New York State licensing board didn’t allow language accommodations,” Matthew said. “But we used data to show how the accommodations were already helping ESL candidates.”

When looking at the pool of candidates who had tested before and after the language accommodations, the pass rates were higher, ranging from roughly 10 percent to 35 percent better across the individual exams.

“The New York State legislature changed its legislation, so our data not only helped our candidates but may also help other organizations proposing language accommodations on certification licensing exams,” Matthews said.

Though the accommodation has made a difference, there’s more work to do. NCARB has several task forces, committees, and working groups continuing to explore the issue.

“We not only want to support our candidates but also explore new paths to licensure,” Matthews said. “For example, most jurisdictions require three essential components to be licensed: education, experience, and examination. What if you only needed two out of three components or could offer alternatives to one of those components? These are some of the areas we’re examining as we work to move the needle on DEIA.”

[MTStock Studio/ISTOCK]